From patchwork Fri Jul 10 16:12:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 276EBC433E3 for ; Fri, 10 Jul 2020 16:20:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF6E1206F4 for ; Fri, 10 Jul 2020 16:20:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="UakEYuMf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF6E1206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvli-0006Wf-5J for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:20:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42408) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvf5-0002FK-ES; Fri, 10 Jul 2020 12:14:07 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43500) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvey-0004v0-41; Fri, 10 Jul 2020 12:14:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=MvVSycQQC8KVkYxr++0oWV9Dkuj9qnJokoUfd8+GF6A=; b=UakEYuMfn4kxEhuohvdQ5CJojRt8zLerBnkpPlnHUeWwlZIia7vPHfuIEFAKy/prlE/5UETb86c0qx47Bn4AQlO70DQGRfdyDCcKUc3Oihu3NDy/oSPMM2ngLOKRXl7/T7QJ5VPPYVfpdjY5oyShQF1rq0L57IGZlqxfbLSfTvwtzhUU7//Z3cIJKTVvePZ/wTaM6TVzh6b2DWOzB9Nforbap+13JYitz19y0bILBDE3/6zVyU8b14mZPcTsbvs/XIPdcwRGLTvXVlmIcJxK01qMFJIlyJ46njUVrJcfvQIkLdOkwe66OgWkNwqnNupAOGJZsQ+yzgjyi1D9c0K2SQ==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtveZ-0003jU-By; Fri, 10 Jul 2020 18:13:35 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001QG-M3; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 04/34] qcow2: Split cluster_needs_cow() out of count_cow_clusters() Date: Fri, 10 Jul 2020 18:12:46 +0200 Message-Id: <65e5d9627ca2ebe7e62deaeddf60949c33067d9d.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We are going to need it in other places. Signed-off-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 360da3f6b2..543f515c81 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1088,6 +1088,24 @@ static void calculate_l2_meta(BlockDriverState *bs, QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight); } +/* Returns true if writing to a cluster requires COW */ +static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry) +{ + switch (qcow2_get_cluster_type(bs, l2_entry)) { + case QCOW2_CLUSTER_NORMAL: + if (l2_entry & QCOW_OFLAG_COPIED) { + return false; + } + case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_CLUSTER_ZERO_PLAIN: + case QCOW2_CLUSTER_ZERO_ALLOC: + return true; + default: + abort(); + } +} + /* * Returns the number of contiguous clusters that can be used for an allocating * write, but require COW to be performed (this includes yet unallocated space, @@ -1100,25 +1118,11 @@ static int count_cow_clusters(BlockDriverState *bs, int nb_clusters, for (i = 0; i < nb_clusters; i++) { uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]); - QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); - - switch(cluster_type) { - case QCOW2_CLUSTER_NORMAL: - if (l2_entry & QCOW_OFLAG_COPIED) { - goto out; - } + if (!cluster_needs_cow(bs, l2_entry)) { break; - case QCOW2_CLUSTER_UNALLOCATED: - case QCOW2_CLUSTER_COMPRESSED: - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: - break; - default: - abort(); } } -out: assert(i <= nb_clusters); return i; } From patchwork Fri Jul 10 16:12:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0F77C433E1 for ; Fri, 10 Jul 2020 16:18:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A86A120657 for ; Fri, 10 Jul 2020 16:18:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Gwizkkts" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A86A120657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvjl-0002Tq-Rv for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:18:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42394) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvf3-00028e-34; Fri, 10 Jul 2020 12:14:05 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43509) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvew-0004v1-PG; Fri, 10 Jul 2020 12:14:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=3/+4+w072NPN2HtyhGIdw7rKwWLkDQ11O7JlmSXpOjI=; b=GwizkktsqKSVwD/SeBCrs8sZJZOMNvL5GUtrn3KNgXo/8KoN8tZ4myVMvSWuWfb6y7Y5JwQyakFA66lTUrtXSuW47oGXuWGEa0jDcw/N3wFQ4JGjfn/qZJDL/jAzzeTDBao2ldSQAASyaMSqPxWX13G7CkaSWFu+IrYqjSBrqfl+ponPTHTfa/LT4SL5ssugLXx7n3Vv/c7/koXz4blJBJsCHJd3T6lUpZEyWX3axaz0K7OgYxi+RpOVSnwZGwL8pUxkOWhTcgmR/+/jkdxg6nxdLfr7PtiutEYoxXKsD2U2rTuGfQTSDXI1YqOgkYA7PT8Ia83ujRTxtZT4epWs3Q==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtveZ-0003jW-Dc; Fri, 10 Jul 2020 18:13:35 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001QI-Mz; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 05/34] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied() Date: Fri, 10 Jul 2020 18:12:47 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When writing to a qcow2 file there are two functions that take a virtual offset and return a host offset, possibly allocating new clusters if necessary: - handle_copied() looks for normal data clusters that are already allocated and have a reference count of 1. In those clusters we can simply write the data and there is no need to perform any copy-on-write. - handle_alloc() looks for clusters that do need copy-on-write, either because they haven't been allocated yet, because their reference count is != 1 or because they are ZERO_ALLOC clusters. The ZERO_ALLOC case is a bit special because those are clusters that are already allocated and they could perfectly be dealt with in handle_copied() (as long as copy-on-write is performed when required). In fact, there is extra code specifically for them in handle_alloc() that tries to reuse the existing allocation if possible and frees them otherwise. This patch changes the handling of ZERO_ALLOC clusters so the semantics of these two functions are now like this: - handle_copied() looks for clusters that are already allocated and which we can overwrite (NORMAL and ZERO_ALLOC clusters with a reference count of 1). - handle_alloc() looks for clusters for which we need a new allocation (all other cases). One important difference after this change is that clusters found in handle_copied() may now require copy-on-write, but this will be necessary anyway once we add support for subclusters. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 256 +++++++++++++++++++++++------------------- 1 file changed, 141 insertions(+), 115 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 543f515c81..723a122977 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1040,13 +1040,18 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) /* * For a given write request, create a new QCowL2Meta structure, add - * it to @m and the BDRVQcow2State.cluster_allocs list. + * it to @m and the BDRVQcow2State.cluster_allocs list. If the write + * request does not need copy-on-write or changes to the L2 metadata + * then this function does nothing. * * @host_cluster_offset points to the beginning of the first cluster. * * @guest_offset and @bytes indicate the offset and length of the * request. * + * @l2_slice contains the L2 entries of all clusters involved in this + * write request. + * * If @keep_old is true it means that the clusters were already * allocated and will be overwritten. If false then the clusters are * new and we have to decrease the reference count of the old ones. @@ -1054,15 +1059,53 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) static void calculate_l2_meta(BlockDriverState *bs, uint64_t host_cluster_offset, uint64_t guest_offset, unsigned bytes, - QCowL2Meta **m, bool keep_old) + uint64_t *l2_slice, QCowL2Meta **m, bool keep_old) { BDRVQcow2State *s = bs->opaque; - unsigned cow_start_from = 0; + int l2_index = offset_to_l2_slice_index(s, guest_offset); + uint64_t l2_entry; + unsigned cow_start_from, cow_end_to; unsigned cow_start_to = offset_into_cluster(s, guest_offset); unsigned cow_end_from = cow_start_to + bytes; - unsigned cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); unsigned nb_clusters = size_to_clusters(s, cow_end_from); QCowL2Meta *old_m = *m; + QCow2ClusterType type; + + assert(nb_clusters <= s->l2_slice_size - l2_index); + + /* Return if there's no COW (all clusters are normal and we keep them) */ + if (keep_old) { + int i; + for (i = 0; i < nb_clusters; i++) { + l2_entry = be64_to_cpu(l2_slice[l2_index + i]); + if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) { + break; + } + } + if (i == nb_clusters) { + return; + } + } + + /* Get the L2 entry of the first cluster */ + l2_entry = be64_to_cpu(l2_slice[l2_index]); + type = qcow2_get_cluster_type(bs, l2_entry); + + if (type == QCOW2_CLUSTER_NORMAL && keep_old) { + cow_start_from = cow_start_to; + } else { + cow_start_from = 0; + } + + /* Get the L2 entry of the last cluster */ + l2_entry = be64_to_cpu(l2_slice[l2_index + nb_clusters - 1]); + type = qcow2_get_cluster_type(bs, l2_entry); + + if (type == QCOW2_CLUSTER_NORMAL && keep_old) { + cow_end_to = cow_end_from; + } else { + cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + } *m = g_malloc0(sizeof(**m)); **m = (QCowL2Meta) { @@ -1088,18 +1131,22 @@ static void calculate_l2_meta(BlockDriverState *bs, QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight); } -/* Returns true if writing to a cluster requires COW */ -static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry) +/* + * Returns true if writing to the cluster pointed to by @l2_entry + * requires a new allocation (that is, if the cluster is unallocated + * or has refcount > 1 and therefore cannot be written in-place). + */ +static bool cluster_needs_new_alloc(BlockDriverState *bs, uint64_t l2_entry) { switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_NORMAL: + case QCOW2_CLUSTER_ZERO_ALLOC: if (l2_entry & QCOW_OFLAG_COPIED) { return false; } case QCOW2_CLUSTER_UNALLOCATED: case QCOW2_CLUSTER_COMPRESSED: case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: return true; default: abort(); @@ -1107,20 +1154,38 @@ static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry) } /* - * Returns the number of contiguous clusters that can be used for an allocating - * write, but require COW to be performed (this includes yet unallocated space, - * which must copy from the backing file) + * Returns the number of contiguous clusters that can be written to + * using one single write request, starting from @l2_index. + * At most @nb_clusters are checked. + * + * If @new_alloc is true this counts clusters that are either + * unallocated, or allocated but with refcount > 1 (so they need to be + * newly allocated and COWed). + * + * If @new_alloc is false this counts clusters that are already + * allocated and can be overwritten in-place (this includes clusters + * of type QCOW2_CLUSTER_ZERO_ALLOC). */ -static int count_cow_clusters(BlockDriverState *bs, int nb_clusters, - uint64_t *l2_slice, int l2_index) +static int count_single_write_clusters(BlockDriverState *bs, int nb_clusters, + uint64_t *l2_slice, int l2_index, + bool new_alloc) { + BDRVQcow2State *s = bs->opaque; + uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index]); + uint64_t expected_offset = l2_entry & L2E_OFFSET_MASK; int i; for (i = 0; i < nb_clusters; i++) { - uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]); - if (!cluster_needs_cow(bs, l2_entry)) { + l2_entry = be64_to_cpu(l2_slice[l2_index + i]); + if (cluster_needs_new_alloc(bs, l2_entry) != new_alloc) { break; } + if (!new_alloc) { + if (expected_offset != (l2_entry & L2E_OFFSET_MASK)) { + break; + } + expected_offset += s->cluster_size; + } } assert(i <= nb_clusters); @@ -1191,10 +1256,10 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, } /* - * Checks how many already allocated clusters that don't require a copy on - * write there are at the given guest_offset (up to *bytes). If *host_offset is - * not INV_OFFSET, only physically contiguous clusters beginning at this host - * offset are counted. + * Checks how many already allocated clusters that don't require a new + * allocation there are at the given guest_offset (up to *bytes). + * If *host_offset is not INV_OFFSET, only physically contiguous clusters + * beginning at this host offset are counted. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -1203,12 +1268,12 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, * Returns: * 0: if no allocated clusters are available at the given offset. * *bytes is normally unchanged. It is set to 0 if the cluster - * is allocated and doesn't need COW, but doesn't have the right - * physical offset. + * is allocated and can be overwritten in-place but doesn't have + * the right physical offset. * - * 1: if allocated clusters that don't require a COW are available at - * the requested offset. *bytes may have decreased and describes - * the length of the area that can be written to. + * 1: if allocated clusters that can be overwritten in place are + * available at the requested offset. *bytes may have decreased + * and describes the length of the area that can be written to. * * -errno: in error cases */ @@ -1217,7 +1282,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, { BDRVQcow2State *s = bs->opaque; int l2_index; - uint64_t cluster_offset; + uint64_t l2_entry, cluster_offset; uint64_t *l2_slice; uint64_t nb_clusters; unsigned int keep_clusters; @@ -1238,7 +1303,8 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, l2_index = offset_to_l2_slice_index(s, guest_offset); nb_clusters = MIN(nb_clusters, s->l2_slice_size - l2_index); - assert(nb_clusters <= INT_MAX); + /* Limit total byte count to BDRV_REQUEST_MAX_BYTES */ + nb_clusters = MIN(nb_clusters, BDRV_REQUEST_MAX_BYTES >> s->cluster_bits); /* Find L2 entry for the first involved cluster */ ret = get_cluster_table(bs, guest_offset, &l2_slice, &l2_index); @@ -1246,41 +1312,39 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, return ret; } - cluster_offset = be64_to_cpu(l2_slice[l2_index]); + l2_entry = be64_to_cpu(l2_slice[l2_index]); + cluster_offset = l2_entry & L2E_OFFSET_MASK; + + if (!cluster_needs_new_alloc(bs, l2_entry)) { + if (offset_into_cluster(s, cluster_offset)) { + qcow2_signal_corruption(bs, true, -1, -1, "%s cluster offset " + "%#" PRIx64 " unaligned (guest offset: %#" + PRIx64 ")", l2_entry & QCOW_OFLAG_ZERO ? + "Preallocated zero" : "Data", + cluster_offset, guest_offset); + ret = -EIO; + goto out; + } - /* Check how many clusters are already allocated and don't need COW */ - if (qcow2_get_cluster_type(bs, cluster_offset) == QCOW2_CLUSTER_NORMAL - && (cluster_offset & QCOW_OFLAG_COPIED)) - { /* If a specific host_offset is required, check it */ - bool offset_matches = - (cluster_offset & L2E_OFFSET_MASK) == *host_offset; - - if (offset_into_cluster(s, cluster_offset & L2E_OFFSET_MASK)) { - qcow2_signal_corruption(bs, true, -1, -1, "Data cluster offset " - "%#llx unaligned (guest offset: %#" PRIx64 - ")", cluster_offset & L2E_OFFSET_MASK, - guest_offset); - ret = -EIO; - goto out; - } - - if (*host_offset != INV_OFFSET && !offset_matches) { + if (*host_offset != INV_OFFSET && cluster_offset != *host_offset) { *bytes = 0; ret = 0; goto out; } /* We keep all QCOW_OFLAG_COPIED clusters */ - keep_clusters = - count_contiguous_clusters(bs, nb_clusters, s->cluster_size, - &l2_slice[l2_index], - QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO); + keep_clusters = count_single_write_clusters(bs, nb_clusters, l2_slice, + l2_index, false); assert(keep_clusters <= nb_clusters); *bytes = MIN(*bytes, keep_clusters * s->cluster_size - offset_into_cluster(s, guest_offset)); + assert(*bytes != 0); + + calculate_l2_meta(bs, cluster_offset, guest_offset, + *bytes, l2_slice, m, true); ret = 1; } else { @@ -1294,8 +1358,7 @@ out: /* Only return a host offset if we actually made progress. Otherwise we * would make requirements for handle_alloc() that it can't fulfill */ if (ret > 0) { - *host_offset = (cluster_offset & L2E_OFFSET_MASK) - + offset_into_cluster(s, guest_offset); + *host_offset = cluster_offset + offset_into_cluster(s, guest_offset); } return ret; @@ -1356,9 +1419,10 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset, } /* - * Allocates new clusters for an area that either is yet unallocated or needs a - * copy on write. If *host_offset is not INV_OFFSET, clusters are only - * allocated if the new allocation can match the specified host offset. + * Allocates new clusters for an area that is either still unallocated or + * cannot be overwritten in-place. If *host_offset is not INV_OFFSET, + * clusters are only allocated if the new allocation can match the specified + * host offset. * * Note that guest_offset may not be cluster aligned. In this case, the * returned *host_offset points to exact byte referenced by guest_offset and @@ -1381,12 +1445,10 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, BDRVQcow2State *s = bs->opaque; int l2_index; uint64_t *l2_slice; - uint64_t entry; uint64_t nb_clusters; int ret; - bool keep_old_clusters = false; - uint64_t alloc_cluster_offset = INV_OFFSET; + uint64_t alloc_cluster_offset; trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset, *bytes); @@ -1401,10 +1463,8 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, l2_index = offset_to_l2_slice_index(s, guest_offset); nb_clusters = MIN(nb_clusters, s->l2_slice_size - l2_index); - assert(nb_clusters <= INT_MAX); - - /* Limit total allocation byte count to INT_MAX */ - nb_clusters = MIN(nb_clusters, INT_MAX >> s->cluster_bits); + /* Limit total allocation byte count to BDRV_REQUEST_MAX_BYTES */ + nb_clusters = MIN(nb_clusters, BDRV_REQUEST_MAX_BYTES >> s->cluster_bits); /* Find L2 entry for the first involved cluster */ ret = get_cluster_table(bs, guest_offset, &l2_slice, &l2_index); @@ -1412,67 +1472,32 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, return ret; } - entry = be64_to_cpu(l2_slice[l2_index]); - nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index); + nb_clusters = count_single_write_clusters(bs, nb_clusters, + l2_slice, l2_index, true); /* This function is only called when there were no non-COW clusters, so if * we can't find any unallocated or COW clusters either, something is * wrong with our code. */ assert(nb_clusters > 0); - if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC && - (entry & QCOW_OFLAG_COPIED) && - (*host_offset == INV_OFFSET || - start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK))) - { - int preallocated_nb_clusters; - - if (offset_into_cluster(s, entry & L2E_OFFSET_MASK)) { - qcow2_signal_corruption(bs, true, -1, -1, "Preallocated zero " - "cluster offset %#llx unaligned (guest " - "offset: %#" PRIx64 ")", - entry & L2E_OFFSET_MASK, guest_offset); - ret = -EIO; - goto fail; - } - - /* Try to reuse preallocated zero clusters; contiguous normal clusters - * would be fine, too, but count_cow_clusters() above has limited - * nb_clusters already to a range of COW clusters */ - preallocated_nb_clusters = - count_contiguous_clusters(bs, nb_clusters, s->cluster_size, - &l2_slice[l2_index], QCOW_OFLAG_COPIED); - assert(preallocated_nb_clusters > 0); - - nb_clusters = preallocated_nb_clusters; - alloc_cluster_offset = entry & L2E_OFFSET_MASK; - - /* We want to reuse these clusters, so qcow2_alloc_cluster_link_l2() - * should not free them. */ - keep_old_clusters = true; + /* Allocate at a given offset in the image file */ + alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET : + start_of_cluster(s, *host_offset); + ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset, + &nb_clusters); + if (ret < 0) { + goto out; } - qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - - if (alloc_cluster_offset == INV_OFFSET) { - /* Allocate, if necessary at a given offset in the image file */ - alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET : - start_of_cluster(s, *host_offset); - ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset, - &nb_clusters); - if (ret < 0) { - goto fail; - } - - /* Can't extend contiguous allocation */ - if (nb_clusters == 0) { - *bytes = 0; - return 0; - } - - assert(alloc_cluster_offset != INV_OFFSET); + /* Can't extend contiguous allocation */ + if (nb_clusters == 0) { + *bytes = 0; + ret = 0; + goto out; } + assert(alloc_cluster_offset != INV_OFFSET); + /* * Save info needed for meta data update. * @@ -1495,13 +1520,14 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, *bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset)); assert(*bytes != 0); - calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, - m, keep_old_clusters); + calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, l2_slice, + m, false); - return 1; + ret = 1; -fail: - if (*m && (*m)->nb_clusters > 0) { +out: + qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); + if (ret < 0 && *m && (*m)->nb_clusters > 0) { QLIST_REMOVE(*m, next_in_flight); } return ret; From patchwork Fri Jul 10 16:12:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 787A4C433E0 for ; Fri, 10 Jul 2020 16:21:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 319F0206F4 for ; Fri, 10 Jul 2020 16:21:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="OO4cWmYE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 319F0206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvls-0006vX-Eb for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:21:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvf4-0002Di-T6; Fri, 10 Jul 2020 12:14:06 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43516) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvew-0004v5-PN; Fri, 10 Jul 2020 12:14:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=dOSk5RtXsBiuIPpToOq43XU4r5AYUmfGVWpe4E+bRiQ=; b=OO4cWmYE8t38QTt2ubxt6lMISyB7DlzLzU6lHqnuk2Fj1enjQutYsyBdGse2vsadI7xwwNxXn//rDPNOZnaxh3EsTDd4SXtfF0byHBTdpf5g6XkFrCCZOG+hTpRM0vaQxIX5svjsESL6aGeHfaZDlVqJMXWjAojYZ4T5Gkjlkfn3v2k5HDdMzLkxq20cx28ulywLLliKnEIvom5arHBbqi9s68PHNDX5yqAF3H2hytWPr88yFXo16LYBL5uomgTm13/0lAr1pOEp75YcH8Fwxz/vopj+ZlQj4wNH2Ck2E4z9hxnAa/50xnu6hOJTISll5qavatVxd1V6Hi2TduEVZw==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtveZ-0003jX-Gx; Fri, 10 Jul 2020 18:13:35 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001QK-O4; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 06/34] qcow2: Add get_l2_entry() and set_l2_entry() Date: Fri, 10 Jul 2020 18:12:48 +0200 Message-Id: <9586363531fec125ba1386e561762d3e4224e9fc.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The size of an L2 entry is 64 bits, but if we want to have subclusters we need extended L2 entries. This means that we have to access L2 tables and slices differently depending on whether an image has extended L2 entries or not. This patch replaces all l2_slice[] accesses with calls to get_l2_entry() and set_l2_entry(). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 12 ++++++++ block/qcow2-cluster.c | 63 ++++++++++++++++++++++-------------------- block/qcow2-refcount.c | 17 ++++++------ 3 files changed, 54 insertions(+), 38 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 06475e0849..eecbadc4cb 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -510,6 +510,18 @@ typedef enum QCow2MetadataOverlap { #define INV_OFFSET (-1ULL) +static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, + int idx) +{ + return be64_to_cpu(l2_slice[idx]); +} + +static inline void set_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, + int idx, uint64_t entry) +{ + l2_slice[idx] = cpu_to_be64(entry); +} + static inline bool has_data_file(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 723a122977..b501642c56 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -383,12 +383,13 @@ fail: * cluster which may require a different handling) */ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, - int cluster_size, uint64_t *l2_slice, uint64_t stop_flags) + int cluster_size, uint64_t *l2_slice, int l2_index, uint64_t stop_flags) { + BDRVQcow2State *s = bs->opaque; int i; QCow2ClusterType first_cluster_type; uint64_t mask = stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED; - uint64_t first_entry = be64_to_cpu(l2_slice[0]); + uint64_t first_entry = get_l2_entry(s, l2_slice, l2_index); uint64_t offset = first_entry & mask; first_cluster_type = qcow2_get_cluster_type(bs, first_entry); @@ -401,7 +402,7 @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); for (i = 0; i < nb_clusters; i++) { - uint64_t l2_entry = be64_to_cpu(l2_slice[i]) & mask; + uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index + i) & mask; if (offset + (uint64_t) i * cluster_size != l2_entry) { break; } @@ -417,14 +418,16 @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, static int count_contiguous_clusters_unallocated(BlockDriverState *bs, int nb_clusters, uint64_t *l2_slice, + int l2_index, QCow2ClusterType wanted_type) { + BDRVQcow2State *s = bs->opaque; int i; assert(wanted_type == QCOW2_CLUSTER_ZERO_PLAIN || wanted_type == QCOW2_CLUSTER_UNALLOCATED); for (i = 0; i < nb_clusters; i++) { - uint64_t entry = be64_to_cpu(l2_slice[i]); + uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i); QCow2ClusterType type = qcow2_get_cluster_type(bs, entry); if (type != wanted_type) { @@ -575,7 +578,7 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, /* find the cluster offset for the given disk offset */ l2_index = offset_to_l2_slice_index(s, offset); - l2_entry = be64_to_cpu(l2_slice[l2_index]); + l2_entry = get_l2_entry(s, l2_slice, l2_index); nb_clusters = size_to_clusters(s, bytes_needed); /* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned @@ -610,7 +613,7 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, case QCOW2_CLUSTER_UNALLOCATED: /* how many empty clusters ? */ c = count_contiguous_clusters_unallocated(bs, nb_clusters, - &l2_slice[l2_index], type); + l2_slice, l2_index, type); break; case QCOW2_CLUSTER_ZERO_ALLOC: case QCOW2_CLUSTER_NORMAL: { @@ -618,7 +621,7 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, *host_offset = host_cluster_offset + offset_in_cluster; /* how many allocated clusters ? */ c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, - &l2_slice[l2_index], QCOW_OFLAG_ZERO); + l2_slice, l2_index, QCOW_OFLAG_ZERO); if (offset_into_cluster(s, host_cluster_offset)) { qcow2_signal_corruption(bs, true, -1, -1, "Cluster allocation offset %#" @@ -770,7 +773,7 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, /* Compression can't overwrite anything. Fail if the cluster was already * allocated. */ - cluster_offset = be64_to_cpu(l2_slice[l2_index]); + cluster_offset = get_l2_entry(s, l2_slice, l2_index); if (cluster_offset & L2E_OFFSET_MASK) { qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); return -EIO; @@ -799,7 +802,7 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, BLKDBG_EVENT(bs->file, BLKDBG_L2_UPDATE_COMPRESSED); qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); - l2_slice[l2_index] = cpu_to_be64(cluster_offset); + set_l2_entry(s, l2_slice, l2_index, cluster_offset); qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); *host_offset = cluster_offset & s->cluster_offset_mask; @@ -992,14 +995,14 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m) * cluster the second one has to do RMW (which is done above by * perform_cow()), update l2 table with its cluster pointer and free * old cluster. This is what this loop does */ - if (l2_slice[l2_index + i] != 0) { - old_cluster[j++] = l2_slice[l2_index + i]; + if (get_l2_entry(s, l2_slice, l2_index + i) != 0) { + old_cluster[j++] = get_l2_entry(s, l2_slice, l2_index + i); } /* The offset must fit in the offset field of the L2 table entry */ assert((offset & L2E_OFFSET_MASK) == offset); - l2_slice[l2_index + i] = cpu_to_be64(offset | QCOW_OFLAG_COPIED); + set_l2_entry(s, l2_slice, l2_index + i, offset | QCOW_OFLAG_COPIED); } @@ -1013,8 +1016,7 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m) */ if (!m->keep_old_clusters && j != 0) { for (i = 0; i < j; i++) { - qcow2_free_any_clusters(bs, be64_to_cpu(old_cluster[i]), 1, - QCOW2_DISCARD_NEVER); + qcow2_free_any_clusters(bs, old_cluster[i], 1, QCOW2_DISCARD_NEVER); } } @@ -1077,7 +1079,7 @@ static void calculate_l2_meta(BlockDriverState *bs, if (keep_old) { int i; for (i = 0; i < nb_clusters; i++) { - l2_entry = be64_to_cpu(l2_slice[l2_index + i]); + l2_entry = get_l2_entry(s, l2_slice, l2_index + i); if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) { break; } @@ -1088,7 +1090,7 @@ static void calculate_l2_meta(BlockDriverState *bs, } /* Get the L2 entry of the first cluster */ - l2_entry = be64_to_cpu(l2_slice[l2_index]); + l2_entry = get_l2_entry(s, l2_slice, l2_index); type = qcow2_get_cluster_type(bs, l2_entry); if (type == QCOW2_CLUSTER_NORMAL && keep_old) { @@ -1098,7 +1100,7 @@ static void calculate_l2_meta(BlockDriverState *bs, } /* Get the L2 entry of the last cluster */ - l2_entry = be64_to_cpu(l2_slice[l2_index + nb_clusters - 1]); + l2_entry = get_l2_entry(s, l2_slice, l2_index + nb_clusters - 1); type = qcow2_get_cluster_type(bs, l2_entry); if (type == QCOW2_CLUSTER_NORMAL && keep_old) { @@ -1171,12 +1173,12 @@ static int count_single_write_clusters(BlockDriverState *bs, int nb_clusters, bool new_alloc) { BDRVQcow2State *s = bs->opaque; - uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index]); + uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index); uint64_t expected_offset = l2_entry & L2E_OFFSET_MASK; int i; for (i = 0; i < nb_clusters; i++) { - l2_entry = be64_to_cpu(l2_slice[l2_index + i]); + l2_entry = get_l2_entry(s, l2_slice, l2_index + i); if (cluster_needs_new_alloc(bs, l2_entry) != new_alloc) { break; } @@ -1312,7 +1314,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, return ret; } - l2_entry = be64_to_cpu(l2_slice[l2_index]); + l2_entry = get_l2_entry(s, l2_slice, l2_index); cluster_offset = l2_entry & L2E_OFFSET_MASK; if (!cluster_needs_new_alloc(bs, l2_entry)) { @@ -1689,7 +1691,7 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, for (i = 0; i < nb_clusters; i++) { uint64_t old_l2_entry; - old_l2_entry = be64_to_cpu(l2_slice[l2_index + i]); + old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); /* * If full_discard is false, make sure that a discarded area reads back @@ -1729,9 +1731,9 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, /* First remove L2 entries */ qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); if (!full_discard && s->qcow_version >= 3) { - l2_slice[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO); + set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); } else { - l2_slice[l2_index + i] = cpu_to_be64(0); + set_l2_entry(s, l2_slice, l2_index + i, 0); } /* Then decrease the refcount */ @@ -1811,7 +1813,7 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, uint64_t old_offset; QCow2ClusterType cluster_type; - old_offset = be64_to_cpu(l2_slice[l2_index + i]); + old_offset = get_l2_entry(s, l2_slice, l2_index + i); /* * Minimize L2 changes if the cluster already reads back as @@ -1825,10 +1827,11 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); if (cluster_type == QCOW2_CLUSTER_COMPRESSED || unmap) { - l2_slice[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO); + set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUEST); } else { - l2_slice[l2_index + i] |= cpu_to_be64(QCOW_OFLAG_ZERO); + uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i); + set_l2_entry(s, l2_slice, l2_index + i, entry | QCOW_OFLAG_ZERO); } } @@ -1966,7 +1969,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } for (j = 0; j < s->l2_slice_size; j++) { - uint64_t l2_entry = be64_to_cpu(l2_slice[j]); + uint64_t l2_entry = get_l2_entry(s, l2_slice, j); int64_t offset = l2_entry & L2E_OFFSET_MASK; QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); @@ -1980,7 +1983,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, if (!bs->backing) { /* not backed; therefore we can simply deallocate the * cluster */ - l2_slice[j] = 0; + set_l2_entry(s, l2_slice, j, 0); l2_dirty = true; continue; } @@ -2046,9 +2049,9 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } if (l2_refcount == 1) { - l2_slice[j] = cpu_to_be64(offset | QCOW_OFLAG_COPIED); + set_l2_entry(s, l2_slice, j, offset | QCOW_OFLAG_COPIED); } else { - l2_slice[j] = cpu_to_be64(offset); + set_l2_entry(s, l2_slice, j, offset); } l2_dirty = true; } diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 0457a6060d..04546838e8 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1310,7 +1310,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, uint64_t cluster_index; uint64_t offset; - entry = be64_to_cpu(l2_slice[j]); + entry = get_l2_entry(s, l2_slice, j); old_entry = entry; entry &= ~QCOW_OFLAG_COPIED; offset = entry & L2E_OFFSET_MASK; @@ -1384,7 +1384,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, qcow2_cache_set_dependency(bs, s->l2_table_cache, s->refcount_block_cache); } - l2_slice[j] = cpu_to_be64(entry); + set_l2_entry(s, l2_slice, j, entry); qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); } @@ -1617,7 +1617,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Do the actual checks */ for(i = 0; i < s->l2_size; i++) { - l2_entry = be64_to_cpu(l2_table[i]); + l2_entry = get_l2_entry(s, l2_table, i); switch (qcow2_get_cluster_type(bs, l2_entry)) { case QCOW2_CLUSTER_COMPRESSED: @@ -1686,7 +1686,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, QCOW2_OL_INACTIVE_L2; l2_entry = QCOW_OFLAG_ZERO; - l2_table[i] = cpu_to_be64(l2_entry); + set_l2_entry(s, l2_table, i, l2_entry); ret = qcow2_pre_write_overlap_check(bs, ign, l2e_offset, sizeof(uint64_t), false); if (ret < 0) { @@ -1914,7 +1914,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, } for (j = 0; j < s->l2_size; j++) { - uint64_t l2_entry = be64_to_cpu(l2_table[j]); + uint64_t l2_entry = get_l2_entry(s, l2_table, j); uint64_t data_offset = l2_entry & L2E_OFFSET_MASK; QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); @@ -1937,9 +1937,10 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, "l2_entry=%" PRIx64 " refcount=%" PRIu64 "\n", repair ? "Repairing" : "ERROR", l2_entry, refcount); if (repair) { - l2_table[j] = cpu_to_be64(refcount == 1 - ? l2_entry | QCOW_OFLAG_COPIED - : l2_entry & ~QCOW_OFLAG_COPIED); + set_l2_entry(s, l2_table, j, + refcount == 1 ? + l2_entry | QCOW_OFLAG_COPIED : + l2_entry & ~QCOW_OFLAG_COPIED); l2_dirty++; } } From patchwork Fri Jul 10 16:12:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C864EC433E6 for ; Fri, 10 Jul 2020 16:15:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84F3620720 for ; Fri, 10 Jul 2020 16:15:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="R/OBK0dQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84F3620720 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvfy-00043z-P6 for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:15:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42384) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvf2-00027D-G3; Fri, 10 Jul 2020 12:14:04 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43525) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvex-0004v7-55; Fri, 10 Jul 2020 12:14:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=FDdJ3NUfcSN6UfYGz2xPvNrqQCi8ZzpI2orXDpb0voQ=; b=R/OBK0dQW885DtWooSx+Iw1VBnIxQgiA4Yxn9uDu3RX/oonEjzZCfxFVANb+rQ4u4zNoIekR6PDlAe0RPZo55ZHJfOep7xnvVO2/Qnk88REiS5boDhCzy0oXNWmuWE9VKlifn5QkrryNow7Gayf6+/Wl6CYpKU/hPd7ZfmIY8IcbncOMwGMTbze+tu8+OHhU4crZ7Y7rT03Id0qNLusWFA4Tuud/kUXawqQfYhL2x0BY3F7GoYCh8MtLNfqiAoREeR2ZMv/0c5Gx1LQVasSZQ8S4wHMvyyGRYyWKJ42mftvcGP6b+i/XpllEw9sVsRT3Q3KvyPp7pwwpZm3DXqUjfQ==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtveZ-0003jZ-Nj; Fri, 10 Jul 2020 18:13:35 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001QM-PD; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 07/34] qcow2: Document the Extended L2 Entries feature Date: Fri, 10 Jul 2020 18:12:49 +0200 Message-Id: <5199f2e1c717bcaa58b48142c9062b803145ff7f.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- docs/interop/qcow2.txt | 68 ++++++++++++++++++++++++++++++++++++++++-- docs/qcow2-cache.txt | 19 +++++++++++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb723463f2..64e9345fb4 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -42,6 +42,9 @@ The first cluster of a qcow2 image contains the file header: as the maximum cluster size and won't be able to open images with larger cluster sizes. + Note: if the image has Extended L2 Entries then cluster_bits + must be at least 14 (i.e. 16384 byte clusters). + 24 - 31: size Virtual disk size in bytes. @@ -117,7 +120,12 @@ the next fields through header_length. clusters. The compression_type field must be present and not zero. - Bits 4-63: Reserved (set to 0) + Bit 4: Extended L2 Entries. If this bit is set then + L2 table entries use an extended format that + allows subcluster-based allocation. See the + Extended L2 Entries section for more details. + + Bits 5-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -498,7 +506,7 @@ cannot be relaxed without an incompatible layout change). Given an offset into the virtual disk, the offset into the image file can be obtained as follows: - l2_entries = (cluster_size / sizeof(uint64_t)) + l2_entries = (cluster_size / sizeof(uint64_t)) [*] l2_index = (offset / cluster_size) % l2_entries l1_index = (offset / cluster_size) / l2_entries @@ -508,6 +516,8 @@ obtained as follows: return cluster_offset + (offset % cluster_size) + [*] this changes if Extended L2 Entries are enabled, see next section + L1 table entry: Bit 0 - 8: Reserved (set to 0) @@ -548,7 +558,8 @@ Standard Cluster Descriptor: nor is data read from the backing file if the cluster is unallocated. - With version 2, this is always 0. + With version 2 or with extended L2 entries (see the next + section), this is always 0. 1 - 8: Reserved (set to 0) @@ -585,6 +596,57 @@ file (except if bit 0 in the Standard Cluster Descriptor is set). If there is no backing file or the backing file is smaller than the image, they shall read zeros for all parts that are not covered by the backing file. +== Extended L2 Entries == + +An image uses Extended L2 Entries if bit 4 is set on the incompatible_features +field of the header. + +In these images standard data clusters are divided into 32 subclusters of the +same size. They are contiguous and start from the beginning of the cluster. +Subclusters can be allocated independently and the L2 entry contains information +indicating the status of each one of them. Compressed data clusters don't have +subclusters so they are treated the same as in images without this feature. + +The size of an extended L2 entry is 128 bits so the number of entries per table +is calculated using this formula: + + l2_entries = (cluster_size / (2 * sizeof(uint64_t))) + +The first 64 bits have the same format as the standard L2 table entry described +in the previous section, with the exception of bit 0 of the standard cluster +descriptor. + +The last 64 bits contain a subcluster allocation bitmap with this format: + +Subcluster Allocation Bitmap (for standard clusters): + + Bit 0 - 31: Allocation status (one bit per subcluster) + + 1: the subcluster is allocated. In this case the + host cluster offset field must contain a valid + offset. + 0: the subcluster is not allocated. In this case + read requests shall go to the backing file or + return zeros if there is no backing file data. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x). + + 32 - 63 Subcluster reads as zeros (one bit per subcluster) + + 1: the subcluster reads as zeros. In this case the + allocation status bit must be unset. The host + cluster offset field may or may not be set. + 0: no effect. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x - 32). + +Subcluster Allocation Bitmap (for compressed clusters): + + Bit 0 - 63: Reserved (set to 0) + Compressed clusters don't have subclusters, + so this field is not used. == Snapshots == diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index d57f409861..5f763aa6bb 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -1,6 +1,6 @@ qcow2 L2/refcount cache configuration ===================================== -Copyright (C) 2015, 2018 Igalia, S.L. +Copyright (C) 2015, 2018-2020 Igalia, S.L. Author: Alberto Garcia This work is licensed under the terms of the GNU GPL, version 2 or @@ -222,3 +222,20 @@ support this functionality, and is 0 (disabled) on other platforms. This functionality currently relies on the MADV_DONTNEED argument for madvise() to actually free the memory. This is a Linux-specific feature, so cache-clean-interval is not supported on other systems. + + +Extended L2 Entries +------------------- +All numbers shown in this document are valid for qcow2 images with normal +64-bit L2 entries. + +Images with extended L2 entries need twice as much L2 metadata, so the L2 +cache size must be twice as large for the same disk space. + + disk_size = l2_cache_size * cluster_size / 16 + +i.e. + + l2_cache_size = disk_size * 16 / cluster_size + +Refcount blocks are not affected by this. From patchwork Fri Jul 10 16:12:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B28DBC433DF for ; Fri, 10 Jul 2020 16:22:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 82D6B20657 for ; Fri, 10 Jul 2020 16:22:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="T/Ucq/XC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 82D6B20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54526 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvnZ-00027t-Oq for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:22:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfM-00030W-Nl; Fri, 10 Jul 2020 12:14:24 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43614) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfH-0004vN-Vy; Fri, 10 Jul 2020 12:14:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=iCyKhgrtnOrB9CQdlSJY9uUmfRXzZb3ZpJw7DcmH0Q0=; b=T/Ucq/XC0r80nIMtw5vuBj8eiqje6oR2BUeYUQNxz9Rhg3J6V3Tm0qaqagdbhcNGMXMRvtmWLesAOBiaWBXZdkxlm9YiXeo/lLSvr5l6oslrdjDi1VjwSqvCEK6GrEHp29a0Q8eYa39NGO1CWZMxoL7jO+tHEZ7icCv7rNai09TaKPYNyJOwoZH2Y6eID4MNPbGGdLIQVcZM8kwG9TaryHGukBoTjocSYceQw69UPsju2X0386LjJEUK2Mq1PXcMMhNumXnZScbSbsd19QtBQ6xN0GtR2SXWABxDvpFrtLwlHPMVMh1flwgB7iI9r0nm0CUAmjDejaiTSP8sfQDwzg==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jc-44; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001QQ-RK; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 09/34] qcow2: Add subcluster-related fields to BDRVQcow2State Date: Fri, 10 Jul 2020 18:12:51 +0200 Message-Id: <55bfeac86b092fa2c9d182a95cbeb479ff7eca4f.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This patch adds the following new fields to BDRVQcow2State: - subclusters_per_cluster: Number of subclusters in a cluster - subcluster_size: The size of each subcluster, in bytes - subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size Images without subclusters are treated as if they had exactly one subcluster per cluster (i.e. subcluster_size = cluster_size). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 5 +++++ block/qcow2.c | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 2064dd3d85..eee4c8de9c 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -78,6 +78,8 @@ /* The cluster reads as all zeros */ #define QCOW_OFLAG_ZERO (1ULL << 0) +#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 + #define MIN_CLUSTER_BITS 9 #define MAX_CLUSTER_BITS 21 @@ -295,6 +297,9 @@ typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; int l2_slice_size; + int subcluster_bits; + int subcluster_size; + int subclusters_per_cluster; int l2_bits; int l2_size; int l1_size; diff --git a/block/qcow2.c b/block/qcow2.c index 47f8032f89..e0349e6800 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1444,6 +1444,11 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } } + s->subclusters_per_cluster = + has_subclusters(s) ? QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER : 1; + s->subcluster_size = s->cluster_size / s->subclusters_per_cluster; + s->subcluster_bits = ctz32(s->subcluster_size); + /* Check support for various header values */ if (header.refcount_order > 6) { error_setg(errp, "Reference count entry width too large; may not " From patchwork Fri Jul 10 16:12:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7B10C433DF for ; Fri, 10 Jul 2020 16:27:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 71DC920657 for ; Fri, 10 Jul 2020 16:27:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="e0aEmokc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71DC920657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49774 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvsM-0003Gb-K5 for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:27:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfO-00035G-H6; Fri, 10 Jul 2020 12:14:26 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43678) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfI-0004vV-Pz; Fri, 10 Jul 2020 12:14:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=PswfzizGui7N0nWBDqMfs9oXoukNQIMoOin+a8N99U0=; b=e0aEmokckG/U1Vgt7x9Y+6xD5kEWuAj0cokjFdzKah08ao7wNtWDM9IpEqSlKpS1Bz2UYPPyayVdUSR4Aasajc70/XnVX9Jm4517uJHaW7/ORHvJuNdbvUSwjPvO4sIXv77FA4ccccPX2HhsmMer+7h4q34tqK+1vVZluqJMV6JiLBjYUmpUASXYF46Z/1cK8723+esgdNWio4NQhzj7qU/GC/AFxgonOm6CV0SaZtAgvCaBaROK5AG3Gwk7H8tD10+WSDCUIkvwheyX24FucVNt1Qpf1A6GF+u0nmFRQ2GdqXQ+fGge429mNYF9tiw9N+z89qjnSJpJY7qSBmMemg==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jf-CJ; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveK-0001Qb-Un; Fri, 10 Jul 2020 18:13:20 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 12/34] qcow2: Add l2_entry_size() Date: Fri, 10 Jul 2020 18:12:54 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2 images with subclusters have 128-bit L2 entries. The first 64 bits contain the same information as traditional images and the last 64 bits form a bitmap with the status of each individual subcluster. Because of that we cannot assume that L2 entries are sizeof(uint64_t) anymore. This function returns the proper value for the image. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 9 +++++++++ block/qcow2-cluster.c | 12 ++++++------ block/qcow2-refcount.c | 14 ++++++++------ block/qcow2.c | 8 ++++---- 4 files changed, 27 insertions(+), 16 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 4fe31adfd3..46b351229a 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -80,6 +80,10 @@ #define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 +/* Size of normal and extended L2 entries */ +#define L2E_SIZE_NORMAL (sizeof(uint64_t)) +#define L2E_SIZE_EXTENDED (sizeof(uint64_t) * 2) + #define MIN_CLUSTER_BITS 9 #define MAX_CLUSTER_BITS 21 @@ -521,6 +525,11 @@ static inline bool has_subclusters(BDRVQcow2State *s) return false; } +static inline size_t l2_entry_size(BDRVQcow2State *s) +{ + return has_subclusters(s) ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL; +} + static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx) { diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index b501642c56..9ba60f3fde 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -208,7 +208,7 @@ static int l2_load(BlockDriverState *bs, uint64_t offset, uint64_t l2_offset, uint64_t **l2_slice) { BDRVQcow2State *s = bs->opaque; - int start_of_slice = sizeof(uint64_t) * + int start_of_slice = l2_entry_size(s) * (offset_to_l2_index(s, offset) - offset_to_l2_slice_index(s, offset)); return qcow2_cache_get(bs, s->l2_table_cache, l2_offset + start_of_slice, @@ -281,7 +281,7 @@ static int l2_allocate(BlockDriverState *bs, int l1_index) /* allocate a new l2 entry */ - l2_offset = qcow2_alloc_clusters(bs, s->l2_size * sizeof(uint64_t)); + l2_offset = qcow2_alloc_clusters(bs, s->l2_size * l2_entry_size(s)); if (l2_offset < 0) { ret = l2_offset; goto fail; @@ -305,7 +305,7 @@ static int l2_allocate(BlockDriverState *bs, int l1_index) /* allocate a new entry in the l2 cache */ - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; trace_qcow2_l2_allocate_get_empty(bs, l1_index); @@ -369,7 +369,7 @@ fail: } s->l1_table[l1_index] = old_l2_offset; if (l2_offset > 0) { - qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t), + qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s), QCOW2_DISCARD_ALWAYS); } return ret; @@ -717,7 +717,7 @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset, /* Then decrease the refcount of the old table */ if (l2_offset) { - qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t), + qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s), QCOW2_DISCARD_OTHER); } @@ -1914,7 +1914,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, int ret; int i, j; - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; if (!is_active_l1) { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 04546838e8..770c5dbc83 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1254,7 +1254,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, l2_slice = NULL; l1_table = NULL; l1_size2 = l1_size * sizeof(uint64_t); - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; s->cache_discards = true; @@ -1605,7 +1605,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, int i, l2_size, nb_csectors, ret; /* Read L2 table from disk */ - l2_size = s->l2_size * sizeof(uint64_t); + l2_size = s->l2_size * l2_entry_size(s); l2_table = g_malloc(l2_size); ret = bdrv_pread(bs->file, l2_offset, l2_table, l2_size); @@ -1680,15 +1680,16 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", offset); if (fix & BDRV_FIX_ERRORS) { + int idx = i * (l2_entry_size(s) / sizeof(uint64_t)); uint64_t l2e_offset = - l2_offset + (uint64_t)i * sizeof(uint64_t); + l2_offset + (uint64_t)i * l2_entry_size(s); int ign = active ? QCOW2_OL_ACTIVE_L2 : QCOW2_OL_INACTIVE_L2; l2_entry = QCOW_OFLAG_ZERO; set_l2_entry(s, l2_table, i, l2_entry); ret = qcow2_pre_write_overlap_check(bs, ign, - l2e_offset, sizeof(uint64_t), false); + l2e_offset, l2_entry_size(s), false); if (ret < 0) { fprintf(stderr, "ERROR: Overlap check failed\n"); res->check_errors++; @@ -1698,7 +1699,8 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } ret = bdrv_pwrite_sync(bs->file, l2e_offset, - &l2_table[i], sizeof(uint64_t)); + &l2_table[idx], + l2_entry_size(s)); if (ret < 0) { fprintf(stderr, "ERROR: Failed to overwrite L2 " "table entry: %s\n", strerror(-ret)); @@ -1905,7 +1907,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, } ret = bdrv_pread(bs->file, l2_offset, l2_table, - s->l2_size * sizeof(uint64_t)); + s->l2_size * l2_entry_size(s)); if (ret < 0) { fprintf(stderr, "ERROR: Could not read L2 table: %s\n", strerror(-ret)); diff --git a/block/qcow2.c b/block/qcow2.c index e0349e6800..49772143b3 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -883,7 +883,7 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, uint64_t max_l2_entries = DIV_ROUND_UP(virtual_disk_size, s->cluster_size); /* An L2 table is always one cluster in size so the max cache size * should be a multiple of the cluster size. */ - uint64_t max_l2_cache = ROUND_UP(max_l2_entries * sizeof(uint64_t), + uint64_t max_l2_cache = ROUND_UP(max_l2_entries * l2_entry_size(s), s->cluster_size); combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); @@ -1042,7 +1042,7 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, } } - r->l2_slice_size = l2_cache_entry_size / sizeof(uint64_t); + r->l2_slice_size = l2_cache_entry_size / l2_entry_size(s); r->l2_table_cache = qcow2_cache_create(bs, l2_cache_size, l2_cache_entry_size); r->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size, @@ -1489,7 +1489,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, bs->encrypted = true; } - s->l2_bits = s->cluster_bits - 3; /* L2 is always one cluster */ + s->l2_bits = s->cluster_bits - ctz32(l2_entry_size(s)); s->l2_size = 1 << s->l2_bits; /* 2^(s->refcount_order - 3) is the refcount width in bytes */ s->refcount_block_bits = s->cluster_bits - (s->refcount_order - 3); @@ -4238,7 +4238,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, * preallocation. All that matters is that we will not have to allocate * new refcount structures for them.) */ nb_new_l2_tables = DIV_ROUND_UP(nb_new_data_clusters, - s->cluster_size / sizeof(uint64_t)); + s->cluster_size / l2_entry_size(s)); /* The cluster range may not be aligned to L2 boundaries, so add one L2 * table for a potential head/tail */ nb_new_l2_tables++; From patchwork Fri Jul 10 16:12:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EC0CC433E4 for ; Fri, 10 Jul 2020 16:16:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BBC220657 for ; Fri, 10 Jul 2020 16:16:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="i/yjpFBs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BBC220657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvhq-0007dz-HM for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:16:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0002rX-AK; Fri, 10 Jul 2020 12:14:21 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43617) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfH-0004vK-ES; Fri, 10 Jul 2020 12:14:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=6qQoyV/cm9Nvp5kDqRKMgyxeyG52rrfmDQLzMDkXXqo=; b=i/yjpFBsC04af9nm5uIgYWoqmf4EFHvpgILrc50C9xHWHU3LwSxhkaLtOYSM1/KViA6CbCjpqwrCXxKrrNcFJ1pHhyfQrPFH/IU+A0uKwTU5d1TKgemZVuzGI4TCWkgh7yCMJixYeFNs5hcy8FuFwnG7mFEbYBjiyrgugZw0xq/MQJAGkPK8EKjCUt7pvHyhoTJX6DY/cIvPIY3KecHgzvLraOabrcYY1kF5ajIUhnKGUyioceu7jEflud0FrBU6gbIKfxpaVvtcSu9lV6rR+DIPdRJVX2J0SDzwAss0CPioVqwcuXJKM9uYHLIGAzEiTEOb6lbgtX/uWEmlKhRS2g==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003ji-3e; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001Qk-2B; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 15/34] qcow2: Add qcow2_get_subcluster_range_type() Date: Fri, 10 Jul 2020 18:12:57 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There are situations in which we want to know how many contiguous subclusters of the same type there are in a given cluster. This can be done by simply iterating over the subclusters and repeatedly calling qcow2_get_subcluster_type() for each one of them. However once we determined the type of a subcluster we can check the rest efficiently by counting the number of adjacent ones (or zeroes) in the bitmap. This is what this function does. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 51 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 9ba60f3fde..751906c330 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -375,6 +375,57 @@ fail: return ret; } +/* + * For a given L2 entry, count the number of contiguous subclusters of + * the same type starting from @sc_from. Compressed clusters are + * treated as if they were divided into subclusters of size + * s->subcluster_size. + * + * Return the number of contiguous subclusters and set @type to the + * subcluster type. + * + * If the L2 entry is invalid return -errno and set @type to + * QCOW2_SUBCLUSTER_INVALID. + */ +G_GNUC_UNUSED +static int qcow2_get_subcluster_range_type(BlockDriverState *bs, + uint64_t l2_entry, + uint64_t l2_bitmap, + unsigned sc_from, + QCow2SubclusterType *type) +{ + BDRVQcow2State *s = bs->opaque; + uint32_t val; + + *type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_from); + + if (*type == QCOW2_SUBCLUSTER_INVALID) { + return -EINVAL; + } else if (!has_subclusters(s) || *type == QCOW2_SUBCLUSTER_COMPRESSED) { + return s->subclusters_per_cluster - sc_from; + } + + switch (*type) { + case QCOW2_SUBCLUSTER_NORMAL: + val = l2_bitmap | QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + val = (l2_bitmap | QCOW_OFLAG_SUB_ZERO_RANGE(0, sc_from)) >> 32; + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + val = ((l2_bitmap >> 32) | l2_bitmap) + & ~QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return ctz32(val) - sc_from; + + default: + g_assert_not_reached(); + } +} + /* * Checks how many clusters in a given L2 slice are contiguous in the image * file. As soon as one of the flags in the bitmask stop_flags changes compared From patchwork Fri Jul 10 16:12:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 359A8C433E3 for ; Fri, 10 Jul 2020 16:20:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0502320657 for ; Fri, 10 Jul 2020 16:20:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="afQ+Ogs+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0502320657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40750 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvkr-0004v6-9c for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:20:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfN-00032u-Is; Fri, 10 Jul 2020 12:14:25 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43751) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfI-0004vr-J0; Fri, 10 Jul 2020 12:14:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=AbBlXU+eGewve60vgPOkXaAE98FEJuDp7cRJT7oNdr4=; b=afQ+Ogs+HMDTvJ3DNWddA6KqTkwq8hjBdalCapBnfwA7wc2RXxhWNphqpbiM3tipbBWpwCq2wJ2t3QipKfFsSP0hkVJAqH5BrD0gMERmAIK4pucDUHo4TFP/XCIBlPRxUMy1nkDKfMIH81jV3qgBJ/XUmUQoDcFLj/pe1S6zZDu6owi0xdC5Mm3bVHPICmQRYOkYRUkBJze73bW42pzOpUr0Qh8D/w2l2t4xoIBjsajJowqpBZslI43ZX74l/C8/QFpVGwdyC+ZDjdK4NEohbOJ0J71w7gfIT67hifP2swFqoU0Z418bpQB3JLtm/UxSSZdgGYjZ0JNRUsSD2cK8Fw==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jk-J1; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001Qq-4f; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 17/34] qcow2: Add cluster type parameter to qcow2_get_host_offset() Date: Fri, 10 Jul 2020 18:12:59 +0200 Message-Id: <396b6eab1859a271551dcd7dcba77f8934aa3c3f.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function returns an integer that can be either an error code or a cluster type (a value from the QCow2ClusterType enum). We are going to start using subcluster types instead of cluster types in some functions so it's better to use the exact data types instead of integers for clarity and in order to detect errors more easily. This patch makes qcow2_get_host_offset() return 0 on success and puts the returned cluster type in a separate parameter. There are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 ++- block/qcow2-cluster.c | 11 +++++++---- block/qcow2.c | 37 ++++++++++++++++++++++--------------- 3 files changed, 31 insertions(+), 20 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index ea647c8bb5..74f65793bd 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -893,7 +893,8 @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, uint8_t *buf, int nb_sectors, bool enc, Error **errp); int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset); + unsigned int *bytes, uint64_t *host_offset, + QCow2ClusterType *cluster_type); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 751906c330..8e1e928852 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -565,13 +565,14 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * * On exit, *bytes is the number of bytes starting at offset that have the same * cluster type and (if applicable) are stored contiguously in the image file. + * The cluster type is stored in *cluster_type. * Compressed clusters are always returned one by one. * - * Returns the cluster type (QCOW2_CLUSTER_*) on success, -errno in error - * cases. + * Returns 0 on success, -errno in error cases. */ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset) + unsigned int *bytes, uint64_t *host_offset, + QCow2ClusterType *cluster_type) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; @@ -713,7 +714,9 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - return type; + *cluster_type = type; + + return 0; fail: qcow2_cache_put(s->l2_table_cache, (void **)&l2_slice); diff --git a/block/qcow2.c b/block/qcow2.c index 49772143b3..5122b71cf2 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2043,6 +2043,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; uint64_t host_offset; unsigned int bytes; + QCow2ClusterType type; int ret, status = 0; qemu_co_mutex_lock(&s->lock); @@ -2054,7 +2055,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, } bytes = MIN(INT_MAX, count); - ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset); + ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset, &type); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { return ret; @@ -2062,15 +2063,15 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) && + if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } - if (ret == QCOW2_CLUSTER_ZERO_PLAIN || ret == QCOW2_CLUSTER_ZERO_ALLOC) { + if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (ret != QCOW2_CLUSTER_UNALLOCATED) { + } else if (type != QCOW2_CLUSTER_UNALLOCATED) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2279,6 +2280,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t host_offset = 0; + QCow2ClusterType type; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2290,22 +2292,23 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, } qemu_co_mutex_lock(&s->lock); - ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset); + ret = qcow2_get_host_offset(bs, offset, &cur_bytes, + &host_offset, &type); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto out; } - if (ret == QCOW2_CLUSTER_ZERO_PLAIN || - ret == QCOW2_CLUSTER_ZERO_ALLOC || - (ret == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + if (type == QCOW2_CLUSTER_ZERO_PLAIN || + type == QCOW2_CLUSTER_ZERO_ALLOC || + (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { if (!aio && cur_bytes != bytes) { aio = aio_task_pool_new(QCOW2_MAX_WORKERS); } - ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret, + ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, type, host_offset, offset, cur_bytes, qiov, qiov_offset, NULL); if (ret < 0) { @@ -3834,6 +3837,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, if (head || tail) { uint64_t off; unsigned int nr; + QCow2ClusterType type; assert(head + bytes <= s->cluster_size); @@ -3849,10 +3853,11 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, offset = QEMU_ALIGN_DOWN(offset, s->cluster_size); bytes = s->cluster_size; nr = s->cluster_size; - ret = qcow2_get_host_offset(bs, offset, &nr, &off); - if (ret != QCOW2_CLUSTER_UNALLOCATED && - ret != QCOW2_CLUSTER_ZERO_PLAIN && - ret != QCOW2_CLUSTER_ZERO_ALLOC) { + ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); + if (ret < 0 || + (type != QCOW2_CLUSTER_UNALLOCATED && + type != QCOW2_CLUSTER_ZERO_PLAIN && + type != QCOW2_CLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); return -ENOTSUP; } @@ -3916,16 +3921,18 @@ qcow2_co_copy_range_from(BlockDriverState *bs, while (bytes != 0) { uint64_t copy_offset = 0; + QCow2ClusterType type; /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; - ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, ©_offset); + ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, + ©_offset, &type); if (ret < 0) { goto out; } - switch (ret) { + switch (type) { case QCOW2_CLUSTER_UNALLOCATED: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); From patchwork Fri Jul 10 16:13:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75CA7C433E0 for ; Fri, 10 Jul 2020 16:17:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F28C20657 for ; Fri, 10 Jul 2020 16:17:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="nEcEAVvY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F28C20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57832 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvia-0000QO-Cm for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:17:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42588) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfL-0002wH-85; Fri, 10 Jul 2020 12:14:23 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43691) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfH-0004vX-Cj; Fri, 10 Jul 2020 12:14:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=qS9pzaJjVclDntJ7A75IojE6NpUgVpElbrXmWtEW7MA=; b=nEcEAVvYnUWvwS7yVR6SoL+7UzOFz4k4c64VIElGuV3sVCxGOtPLVF2470zVxs1aPO0hGvenn8h5873TrtAPYwHfCen55LI7tUKJtfVkzu/TCjcWBW7K0tKlfntiB/8EstE8kw/yHLRUAPq9Z/j6MWeknq1TD7KeVn5p9w+Jbm+cMM2ThNUPgMkNxG4m3rKRG2ZEumm85CEYcMSk6cO1nRyuc17xpDyToZUZX43qB/N8YtqKuUjr8zkzkKIDv6e5/9vXurpQ+Z5oNAKgQgBNrirFapimKoP+sCZBLHC7a10QwEzEJF+iUUhgqujW0SBSfWOuDAEKhjNMRKt46gSsiw==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jl-CE; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001Qu-6E; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 18/34] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_* Date: Fri, 10 Jul 2020 18:13:00 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" In order to support extended L2 entries some functions of the qcow2 driver need to start dealing with subclusters instead of clusters. qcow2_get_host_offset() is modified to return the subcluster type instead of the cluster type, and all callers are updated to replace all values of QCow2ClusterType with their QCow2SubclusterType equivalents. This patch only changes the data types, there are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 2 +- block/qcow2-cluster.c | 10 +++---- block/qcow2.c | 70 ++++++++++++++++++++++--------------------- 3 files changed, 42 insertions(+), 40 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 74f65793bd..5df761edc3 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -894,7 +894,7 @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type); + QCow2SubclusterType *subcluster_type); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 8e1e928852..dd8a72381b 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -564,15 +564,15 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * offset that we are interested in. * * On exit, *bytes is the number of bytes starting at offset that have the same - * cluster type and (if applicable) are stored contiguously in the image file. - * The cluster type is stored in *cluster_type. - * Compressed clusters are always returned one by one. + * subcluster type and (if applicable) are stored contiguously in the image + * file. The subcluster type is stored in *subcluster_type. + * Compressed clusters are always processed one by one. * * Returns 0 on success, -errno in error cases. */ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type) + QCow2SubclusterType *subcluster_type) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; @@ -714,7 +714,7 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - *cluster_type = type; + *subcluster_type = qcow2_cluster_to_subcluster_type(type); return 0; diff --git a/block/qcow2.c b/block/qcow2.c index 5122b71cf2..6fbc73ff19 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2043,7 +2043,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; uint64_t host_offset; unsigned int bytes; - QCow2ClusterType type; + QCow2SubclusterType type; int ret, status = 0; qemu_co_mutex_lock(&s->lock); @@ -2063,15 +2063,16 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) && - !s->crypto) { + if ((type == QCOW2_SUBCLUSTER_NORMAL || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) { + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (type != QCOW2_CLUSTER_UNALLOCATED) { + } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2168,7 +2169,7 @@ typedef struct Qcow2AioTask { AioTask task; BlockDriverState *bs; - QCow2ClusterType cluster_type; /* only for read */ + QCow2SubclusterType subcluster_type; /* only for read */ uint64_t host_offset; /* or full descriptor in compressed clusters */ uint64_t offset; uint64_t bytes; @@ -2181,7 +2182,7 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); static coroutine_fn int qcow2_add_task(BlockDriverState *bs, AioTaskPool *pool, AioTaskFunc func, - QCow2ClusterType cluster_type, + QCow2SubclusterType subcluster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, @@ -2195,7 +2196,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, *task = (Qcow2AioTask) { .task.func = func, .bs = bs, - .cluster_type = cluster_type, + .subcluster_type = subcluster_type, .qiov = qiov, .host_offset = host_offset, .offset = offset, @@ -2206,7 +2207,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, func == qcow2_co_preadv_task_entry ? "read" : "write", - cluster_type, host_offset, offset, bytes, + subcluster_type, host_offset, offset, bytes, qiov, qiov_offset); if (!pool) { @@ -2219,7 +2220,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, } static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, - QCow2ClusterType cluster_type, + QCow2SubclusterType subc_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, @@ -2227,24 +2228,24 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; - switch (cluster_type) { - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + switch (subc_type) { + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: /* Both zero types are handled in qcow2_co_preadv_part */ g_assert_not_reached(); - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */ BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); return bdrv_co_preadv_part(bs->backing, offset, bytes, qiov, qiov_offset, 0); - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: return qcow2_co_preadv_compressed(bs, host_offset, offset, bytes, qiov, qiov_offset); - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: if (bs->encrypted) { return qcow2_co_preadv_encrypted(bs, host_offset, offset, bytes, qiov, qiov_offset); @@ -2267,8 +2268,9 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task) assert(!t->l2meta); - return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset, - t->offset, t->bytes, t->qiov, t->qiov_offset); + return qcow2_co_preadv_task(t->bs, t->subcluster_type, + t->host_offset, t->offset, t->bytes, + t->qiov, t->qiov_offset); } static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, @@ -2280,7 +2282,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t host_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2299,9 +2301,9 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, goto out; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || - type == QCOW2_CLUSTER_ZERO_ALLOC || - (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { @@ -2534,7 +2536,7 @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type); + assert(!t->subcluster_type); return qcow2_co_pwritev_task(t->bs, t->host_offset, t->offset, t->bytes, t->qiov, t->qiov_offset, @@ -3837,7 +3839,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, if (head || tail) { uint64_t off; unsigned int nr; - QCow2ClusterType type; + QCow2SubclusterType type; assert(head + bytes <= s->cluster_size); @@ -3855,9 +3857,9 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, nr = s->cluster_size; ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); if (ret < 0 || - (type != QCOW2_CLUSTER_UNALLOCATED && - type != QCOW2_CLUSTER_ZERO_PLAIN && - type != QCOW2_CLUSTER_ZERO_ALLOC)) { + (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); return -ENOTSUP; } @@ -3921,7 +3923,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, while (bytes != 0) { uint64_t copy_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; @@ -3933,7 +3935,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } switch (type) { - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); if (src_offset >= backing_length) { @@ -3948,16 +3950,16 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } break; - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: cur_write_flags |= BDRV_REQ_ZERO_WRITE; break; - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: ret = -ENOTSUP; goto out; - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: child = s->data_file; break; @@ -4486,7 +4488,7 @@ static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type && !t->l2meta); + assert(!t->subcluster_type && !t->l2meta); return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov, t->qiov_offset); From patchwork Fri Jul 10 16:13:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47717C433E4 for ; Fri, 10 Jul 2020 16:24:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16E4C20657 for ; Fri, 10 Jul 2020 16:24:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="BJgt/8t5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16E4C20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvpL-0005cs-9g for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:24:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42584) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfK-0002vT-Po; Fri, 10 Jul 2020 12:14:22 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43695) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfH-0004vZ-Iz; Fri, 10 Jul 2020 12:14:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=i3zbC+W1jm5Y+b2qDqKso8xaRYj01unY16Edf1znrLY=; b=BJgt/8t5HPaPgx5HDenFngC5AQIXXtTKeJsFGdKJA/Nf6Ce5I1yGnb+UKYzFvm5j6l+9jhqfufEDwgJd6OZ3sKC88virLnO8W+g7tc6F8rU3025tjXbEfIzBPCiMKTDEuOpue0hx0UDzsZ9XjMyGUDfcBMSPAQS/F/Www4IBk/sqaWDje6S8cx9DfMXah0lm++OyY96wlBfgaTE/3ZvieQEtUc2VvyAPV1qDIMpSi7536lE3xiDafhMmO1okLBXJpEB6eK+Qs+mzUQd+8t5HfWLMwowyOR2Yz3MLrgp+/PJvJvOTaxfxVMCBDrVOyXoMTNGxFUACEI+/oplOZpkK0Q==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jm-Dy; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001Qx-7a; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 19/34] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC Date: Fri, 10 Jul 2020 18:13:01 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When dealing with subcluster types there is a new value called QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC that has no equivalent in QCow2ClusterType. This patch handles that value in all places where subcluster types are processed. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6fbc73ff19..854a2fbd9b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2064,7 +2064,8 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; if ((type == QCOW2_SUBCLUSTER_NORMAL || - type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) { + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; @@ -2072,7 +2073,8 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || type == QCOW2_SUBCLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) { + } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && + type != QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2235,6 +2237,7 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, g_assert_not_reached(); case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */ BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); @@ -2303,7 +2306,8 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || type == QCOW2_SUBCLUSTER_ZERO_ALLOC || - (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing)) + (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing) || + (type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { @@ -3858,6 +3862,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); if (ret < 0 || (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && + type != QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC && type != QCOW2_SUBCLUSTER_ZERO_PLAIN && type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); @@ -3936,6 +3941,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, switch (type) { case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); if (src_offset >= backing_length) { From patchwork Fri Jul 10 16:13:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E005C433E2 for ; Fri, 10 Jul 2020 16:26:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58A32206F4 for ; Fri, 10 Jul 2020 16:26:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="V85HEBnH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58A32206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43264 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvqu-0000dp-JS for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:26:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42586) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfK-0002vq-WC; Fri, 10 Jul 2020 12:14:23 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43709) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfH-0004vc-Bo; Fri, 10 Jul 2020 12:14:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=io8Hbu29hnQVJl/hLGUl3NMzVLs9QvIxmYIu+OjGU+8=; b=V85HEBnHQvP9gv4OJTSbp9w0vIu1OV7WKwf4gEU+Yuemx6pHIeLOBCe1QK+ej5kFU+kSMuRHOJJDkSYsscSI8kQ8mt0X4iNzKkxX04PwoT5q05q2Nk2JxlH4M5Kd1WQdPucSXh9p7HQO1CRYsDNZ+OlpgKFCjlZH6UXhjF+XhH3rgBi9ehCEUADdJ/0g6Xaon32oiZBy4TrcZpLerDgXXcmsB4IRBTZhpabIqzhdzJhAJBG8WeProbBrGCmWh45F5yxK6KVzJUREmm8qyRr+mFD25ZhOu6JW1nEfOBjtFJE0WP2m3DXSin2mwtScqx1QOd5EGc6m5RPQqNoOjqfnrg==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jn-EP; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001R0-8s; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 20/34] qcow2: Add subcluster support to calculate_l2_meta() Date: Fri, 10 Jul 2020 18:13:02 +0200 Message-Id: <4292dd56e4446d386a2fe307311737a711c00708.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If an image has subclusters then there are more copy-on-write scenarios that we need to consider. Let's say we have a write request from the middle of subcluster #3 until the end of the cluster: 1) If we are writing to a newly allocated cluster then we need copy-on-write. The previous contents of subclusters #0 to #3 must be copied to the new cluster. We can optimize this process by skipping all leading unallocated or zero subclusters (the status of those skipped subclusters will be reflected in the new L2 bitmap). 2) If we are overwriting an existing cluster: 2.1) If subcluster #3 is unallocated or has the all-zeroes bit set then we need copy-on-write (on subcluster #3 only). 2.2) If subcluster #3 was already allocated then there is no need for any copy-on-write. However we still need to update the L2 bitmap to reflect possible changes in the allocation status of subclusters #4 to #31. Because of this, this function checks if all the overwritten subclusters are already allocated and in this case it returns without creating a new QCowL2Meta structure. After all these changes l2meta_cow_start() and l2meta_cow_end() are not necessarily cluster-aligned anymore. We need to update the calculation of old_start and old_end in handle_dependencies() to guarantee that no two requests try to write on the same cluster. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 163 +++++++++++++++++++++++++++++++++--------- 1 file changed, 131 insertions(+), 32 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index dd8a72381b..01495cb1c0 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -387,7 +387,6 @@ fail: * If the L2 entry is invalid return -errno and set @type to * QCOW2_SUBCLUSTER_INVALID. */ -G_GNUC_UNUSED static int qcow2_get_subcluster_range_type(BlockDriverState *bs, uint64_t l2_entry, uint64_t l2_bitmap, @@ -1111,56 +1110,148 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) * If @keep_old is true it means that the clusters were already * allocated and will be overwritten. If false then the clusters are * new and we have to decrease the reference count of the old ones. + * + * Returns 0 on success, -errno on failure. */ -static void calculate_l2_meta(BlockDriverState *bs, - uint64_t host_cluster_offset, - uint64_t guest_offset, unsigned bytes, - uint64_t *l2_slice, QCowL2Meta **m, bool keep_old) +static int calculate_l2_meta(BlockDriverState *bs, uint64_t host_cluster_offset, + uint64_t guest_offset, unsigned bytes, + uint64_t *l2_slice, QCowL2Meta **m, bool keep_old) { BDRVQcow2State *s = bs->opaque; - int l2_index = offset_to_l2_slice_index(s, guest_offset); - uint64_t l2_entry; + int sc_index, l2_index = offset_to_l2_slice_index(s, guest_offset); + uint64_t l2_entry, l2_bitmap; unsigned cow_start_from, cow_end_to; unsigned cow_start_to = offset_into_cluster(s, guest_offset); unsigned cow_end_from = cow_start_to + bytes; unsigned nb_clusters = size_to_clusters(s, cow_end_from); QCowL2Meta *old_m = *m; - QCow2ClusterType type; + QCow2SubclusterType type; + int i; + bool skip_cow = keep_old; assert(nb_clusters <= s->l2_slice_size - l2_index); - /* Return if there's no COW (all clusters are normal and we keep them) */ - if (keep_old) { - int i; - for (i = 0; i < nb_clusters; i++) { - l2_entry = get_l2_entry(s, l2_slice, l2_index + i); - if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) { - break; + /* Check the type of all affected subclusters */ + for (i = 0; i < nb_clusters; i++) { + l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + if (skip_cow) { + unsigned write_from = MAX(cow_start_to, i << s->cluster_bits); + unsigned write_to = MIN(cow_end_from, (i + 1) << s->cluster_bits); + int first_sc = offset_to_sc_index(s, write_from); + int last_sc = offset_to_sc_index(s, write_to - 1); + int cnt = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap, + first_sc, &type); + /* Is any of the subclusters of type != QCOW2_SUBCLUSTER_NORMAL ? */ + if (type != QCOW2_SUBCLUSTER_NORMAL || first_sc + cnt <= last_sc) { + skip_cow = false; } + } else { + /* If we can't skip the cow we can still look for invalid entries */ + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, 0); } - if (i == nb_clusters) { - return; + if (type == QCOW2_SUBCLUSTER_INVALID) { + int l1_index = offset_to_l1_index(s, guest_offset); + uint64_t l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK; + qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster " + "entry found (L2 offset: %#" PRIx64 + ", L2 index: %#x)", + l2_offset, l2_index + i); + return -EIO; } } + if (skip_cow) { + return 0; + } + /* Get the L2 entry of the first cluster */ l2_entry = get_l2_entry(s, l2_slice, l2_index); - type = qcow2_get_cluster_type(bs, l2_entry); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); + sc_index = offset_to_sc_index(s, guest_offset); + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); - if (type == QCOW2_CLUSTER_NORMAL && keep_old) { - cow_start_from = cow_start_to; + if (!keep_old) { + switch (type) { + case QCOW2_SUBCLUSTER_COMPRESSED: + cow_start_from = 0; + break; + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + if (has_subclusters(s)) { + /* Skip all leading zero and unallocated subclusters */ + uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC; + cow_start_from = + MIN(sc_index, ctz32(alloc_bitmap)) << s->subcluster_bits; + } else { + cow_start_from = 0; + } + break; + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + cow_start_from = sc_index << s->subcluster_bits; + break; + default: + g_assert_not_reached(); + } } else { - cow_start_from = 0; + switch (type) { + case QCOW2_SUBCLUSTER_NORMAL: + cow_start_from = cow_start_to; + break; + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_start_from = sc_index << s->subcluster_bits; + break; + default: + g_assert_not_reached(); + } } /* Get the L2 entry of the last cluster */ - l2_entry = get_l2_entry(s, l2_slice, l2_index + nb_clusters - 1); - type = qcow2_get_cluster_type(bs, l2_entry); + l2_index += nb_clusters - 1; + l2_entry = get_l2_entry(s, l2_slice, l2_index); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); + sc_index = offset_to_sc_index(s, guest_offset + bytes - 1); + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); - if (type == QCOW2_CLUSTER_NORMAL && keep_old) { - cow_end_to = cow_end_from; + if (!keep_old) { + switch (type) { + case QCOW2_SUBCLUSTER_COMPRESSED: + cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + break; + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + if (has_subclusters(s)) { + /* Skip all trailing zero and unallocated subclusters */ + uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC; + cow_end_to -= + MIN(s->subclusters_per_cluster - sc_index - 1, + clz32(alloc_bitmap)) << s->subcluster_bits; + } + break; + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size); + break; + default: + g_assert_not_reached(); + } } else { - cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + switch (type) { + case QCOW2_SUBCLUSTER_NORMAL: + cow_end_to = cow_end_from; + break; + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size); + break; + default: + g_assert_not_reached(); + } } *m = g_malloc0(sizeof(**m)); @@ -1185,6 +1276,8 @@ static void calculate_l2_meta(BlockDriverState *bs, qemu_co_queue_init(&(*m)->dependent_requests); QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight); + + return 0; } /* @@ -1273,8 +1366,8 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, uint64_t start = guest_offset; uint64_t end = start + bytes; - uint64_t old_start = l2meta_cow_start(old_alloc); - uint64_t old_end = l2meta_cow_end(old_alloc); + uint64_t old_start = start_of_cluster(s, l2meta_cow_start(old_alloc)); + uint64_t old_end = ROUND_UP(l2meta_cow_end(old_alloc), s->cluster_size); if (end <= old_start || start >= old_end) { /* No intersection */ @@ -1399,8 +1492,11 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, - offset_into_cluster(s, guest_offset)); assert(*bytes != 0); - calculate_l2_meta(bs, cluster_offset, guest_offset, - *bytes, l2_slice, m, true); + ret = calculate_l2_meta(bs, cluster_offset, guest_offset, + *bytes, l2_slice, m, true); + if (ret < 0) { + goto out; + } ret = 1; } else { @@ -1576,8 +1672,11 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, *bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset)); assert(*bytes != 0); - calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, l2_slice, - m, false); + ret = calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, + l2_slice, m, false); + if (ret < 0) { + goto out; + } ret = 1; From patchwork Fri Jul 10 16:13:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C169C433E2 for ; Fri, 10 Jul 2020 16:29:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 159C520657 for ; Fri, 10 Jul 2020 16:29:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="O7Gj1Fir" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 159C520657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvu4-0006lT-AZ for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:29:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42686) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfO-00035t-Tv; Fri, 10 Jul 2020 12:14:26 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43728) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004vg-Ai; Fri, 10 Jul 2020 12:14:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=fFliMiIZg8gQ9uYP+jhwHbbYWydz6rMSlmOIpLu6pk0=; b=O7Gj1Fir3FQ7rQ4K3OVXb7p1Lfj71yc2820GEcRSprhzP/V/W5QyF7AAQNJnUFSJiBvYpj1rFYvrPJOm07EBV6AH67yxHtXo/h8RefpdMVTPHyyyySse71hemI84N58dPn2Hcui9k8QMU/t733rUUWhv3lCcPBWvBfKkReFKVFozQFZWC2oeEmYm9yTfxWCrRefVT9dgaq0bgD5rdHoAtEXEVpb5PhPCisp0aH19LdETbgTrPNtux4B1HWDlVRYhJJppGy675m3EkrG/0rVVhekP8DgksjdtMzpH2y4ePF/6Wv9zsGqaKsk89lqpURf4UoSXisN/uGLXnSf1rY7sMQ==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003ju-H0; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RB-D5; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 24/34] qcow2: Add subcluster support to check_refcounts_l2() Date: Fri, 10 Jul 2020 18:13:06 +0200 Message-Id: <9f4ed1d0a34b0a545b032c31ecd8c14734065342.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The offset field of an uncompressed cluster's L2 entry must be aligned to the cluster size, otherwise it is invalid. If the cluster has no data then it means that the offset points to a preallocation, so we can clear the offset field without affecting the guest-visible data. This is what 'qemu-img check' does when run in repair mode. On traditional qcow2 images this can only happen when QCOW_OFLAG_ZERO is set, and repairing such entries turns the clusters from ZERO_ALLOC into ZERO_PLAIN. Extended L2 entries have no ZERO_ALLOC clusters and no QCOW_OFLAG_ZERO but the idea is the same: if none of the subclusters are allocated then we can clear the offset field and leave the bitmap untouched. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 16 +++++++++++----- tests/qemu-iotests/060.out | 2 +- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 770c5dbc83..aae52607eb 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1669,12 +1669,18 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, /* Correct offsets are cluster aligned */ if (offset_into_cluster(s, offset)) { + bool contains_data; res->corruptions++; - if (qcow2_get_cluster_type(bs, l2_entry) == - QCOW2_CLUSTER_ZERO_ALLOC) - { - fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated zero " + if (has_subclusters(s)) { + uint64_t l2_bitmap = get_l2_bitmap(s, l2_table, i); + contains_data = (l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC); + } else { + contains_data = !(l2_entry & QCOW_OFLAG_ZERO); + } + + if (!contains_data) { + fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated " "cluster is not properly aligned; L2 entry " "corrupted.\n", fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", @@ -1686,7 +1692,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, int ign = active ? QCOW2_OL_ACTIVE_L2 : QCOW2_OL_INACTIVE_L2; - l2_entry = QCOW_OFLAG_ZERO; + l2_entry = has_subclusters(s) ? 0 : QCOW_OFLAG_ZERO; set_l2_entry(s, l2_table, i, l2_entry); ret = qcow2_pre_write_overlap_check(bs, ign, l2e_offset, l2_entry_size(s), false); diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index be5f8707a3..fa3d68f0df 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -320,7 +320,7 @@ discard 65536/65536 bytes at offset 0 qcow2: Marking image as corrupt: Preallocated zero cluster offset 0x2a00 unaligned (guest offset: 0); further corruption events will be suppressed write failed: Input/output error --- Repairing --- -Repairing offset=2a00: Preallocated zero cluster is not properly aligned; L2 entry corrupted. +Repairing offset=2a00: Preallocated cluster is not properly aligned; L2 entry corrupted. The following inconsistencies were found and repaired: 0 leaked clusters From patchwork Fri Jul 10 16:13:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A63B0C433E0 for ; Fri, 10 Jul 2020 16:22:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6CC5F20657 for ; Fri, 10 Jul 2020 16:22:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="fe5EySQe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CC5F20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvn3-0000qW-K0 for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:22:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42718) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfP-00038j-Uh; Fri, 10 Jul 2020 12:14:27 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43847) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004wB-E5; Fri, 10 Jul 2020 12:14:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=kRWs8b0J2r7ro/blkkJyRPuIfQyqloPwSMTOgVtkpus=; b=fe5EySQen37iy8FIxcN7ngldNh5+3rtQ4DcyvfEr+lDsgvmUhpCseMPY08dzIaBFOXcnweSoNralvMb5XlRCtBSltW/b70a8whcZ884pQoukSeC7AsPiibe6sfnur+921RWID9z+WBBvJ7pQPNRt0tre/7aEnDiD2PKHPnhB3/Oh4ThQDIkF+SU6PcnjZ6zHap1zrwDQxZOkhemy3skXFIDwwfD61bPb8ehUOXFeeMJ9qquLVGZVc4GTSQkaQ45MZXfGSmPY56yldKbpJdtMRdx7UBWopYt9JrfoXq7trTVBYk4KLLiph7Pi9zC064E+jRHMRo7kIzGuzehhT3PAVA==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003jv-MS; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RE-EB; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 25/34] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2() Date: Fri, 10 Jul 2020 18:13:07 +0200 Message-Id: <0875620d49f44320334b6a91c73b3f301f975f38.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The L2 bitmap needs to be updated after each write to indicate what new subclusters are now allocated. This needs to happen even if the cluster was already allocated and the L2 entry was otherwise valid. In some cases however a write operation doesn't need change the L2 bitmap (because all affected subclusters were already allocated). This is detected in calculate_l2_meta(), and qcow2_alloc_cluster_link_l2() is never called in those cases. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index ea025dc531..5bdbb65e7b 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1062,6 +1062,24 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m) assert((offset & L2E_OFFSET_MASK) == offset); set_l2_entry(s, l2_slice, l2_index + i, offset | QCOW_OFLAG_COPIED); + + /* Update bitmap with the subclusters that were just written */ + if (has_subclusters(s)) { + uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + unsigned written_from = m->cow_start.offset; + unsigned written_to = m->cow_end.offset + m->cow_end.nb_bytes ?: + m->nb_clusters << s->cluster_bits; + int first_sc, last_sc; + /* Narrow written_from and written_to down to the current cluster */ + written_from = MAX(written_from, i << s->cluster_bits); + written_to = MIN(written_to, (i + 1) << s->cluster_bits); + assert(written_from < written_to); + first_sc = offset_to_sc_index(s, written_from); + last_sc = offset_to_sc_index(s, written_to - 1); + l2_bitmap |= QCOW_OFLAG_SUB_ALLOC_RANGE(first_sc, last_sc + 1); + l2_bitmap &= ~QCOW_OFLAG_SUB_ZERO_RANGE(first_sc, last_sc + 1); + set_l2_bitmap(s, l2_slice, l2_index + i, l2_bitmap); + } } From patchwork Fri Jul 10 16:13:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 001E8C433DF for ; Fri, 10 Jul 2020 16:23:57 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B8A4A20657 for ; Fri, 10 Jul 2020 16:23:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="cr75TOvm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8A4A20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvob-0004W3-0r for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:23:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42778) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfS-0003FK-7d; Fri, 10 Jul 2020 12:14:30 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43804) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004vw-MI; Fri, 10 Jul 2020 12:14:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=QjmKjTeXhBr0+DarDQOWZqlKL4Iexn7OK01kZ5bqdZk=; b=cr75TOvm4inJBSxB3aabb3GdJq/B28zqCFJx2k7AMegprZ1KRTpWXbdbi85V8fL6J+7l4T1Ejs7vUJutUIb7t+3Uk2TSmDvylCT8VQcOIvM6oITHUOrADrV3dtpkOnWT2eO9NDe0gT7slhKQcX1y8aER1AcBn3hdtKGagBeZgqn2iyH9gdnDxyfRyfXLnBGS5qmxlGqLpMjwQZJkmVFBItfMK9+DaPjr/qJTlGtazQ++CezxxLeM4M4smY/rlHJ+ZegeqoBcaxoCLF4zZfJgr6Yxkc+cnr1cWKz2qsgUZ+2xcJq7OQMJoycb1CaHtgAVHZGKEg1KbyWKlEDg2G0uyA==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003k3-LU; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RK-GA; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 27/34] qcow2: Add subcluster support to handle_alloc_space() Date: Fri, 10 Jul 2020 18:13:09 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The bdrv_co_pwrite_zeroes() call here fills complete clusters with zeroes, but it can happen that some subclusters are not part of the write request or the copy-on-write. This patch makes sure that only the affected subclusters are overwritten. A potential improvement would be to also fill with zeroes the other subclusters if we can guarantee that we are not overwriting existing data. However this would waste more disk space, so we should first evaluate if it's really worth doing. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 854a2fbd9b..b58c421c28 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2421,6 +2421,9 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) for (m = l2meta; m != NULL; m = m->next) { int ret; + uint64_t start_offset = m->alloc_offset + m->cow_start.offset; + unsigned nb_bytes = m->cow_end.offset + m->cow_end.nb_bytes - + m->cow_start.offset; if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { continue; @@ -2435,16 +2438,14 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) * efficiently zero out the whole clusters */ - ret = qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = qcow2_pre_write_overlap_check(bs, 0, start_offset, nb_bytes, true); if (ret < 0) { return ret; } BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE); - ret = bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = bdrv_co_pwrite_zeroes(s->data_file, start_offset, nb_bytes, BDRV_REQ_NO_FALLBACK); if (ret < 0) { if (ret != -ENOTSUP && ret != -EAGAIN) { From patchwork Fri Jul 10 16:13:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDBCAC433E1 for ; Fri, 10 Jul 2020 16:33:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6AF5420657 for ; Fri, 10 Jul 2020 16:33:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="g8uU4jch" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6AF5420657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42966 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvxQ-0003hL-E3 for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:33:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42780) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfS-0003G8-Go; Fri, 10 Jul 2020 12:14:30 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43854) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004wF-MK; Fri, 10 Jul 2020 12:14:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=wI3TwV908BhBisL9PFEr4SOnyuhOIobK8NYSya/vLt8=; b=g8uU4jchJn+yjNrxR+xFs4wrCiWyihqggKpauasEPp1m9NlG6kpB0oar14QglAx7DA0My3UQyziGJHdG+r7QKytNInuBeiDV/Q4c4K3wFqAnUkvUp/2kg/kJku9dGDpgZ1uYEU0ucAj1M3j85fKSv49PDj8k98vQ6+5BOAejppz5yxSL2i/jCUHvQD6YS5tMlym8aY4V/0zNHQyKRtO/Zc1ur2u4SiiNgGhqqc/dexj3/rXjVgbNumDB4N3jaG2m9+JUl0AVnKbu/oV1gvSJY73FlEdfsx97UrrMXDj1tY63yda1n76tcpxz0WRbraLowkGMkyHsT6WKGVvB94SY1g==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003k5-Nf; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RQ-IC; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 29/34] qcow2: Add subcluster support to qcow2_measure() Date: Fri, 10 Jul 2020 18:13:11 +0200 Message-Id: <7efae2efd5e36b42d2570743a12576d68ce53685.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Extended L2 entries are bigger than normal L2 entries so this has an impact on the amount of metadata needed for a qcow2 file. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 358e60a261..9ab2a91c8b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3232,28 +3232,31 @@ int64_t qcow2_refcount_metadata_size(int64_t clusters, size_t cluster_size, * @total_size: virtual disk size in bytes * @cluster_size: cluster size in bytes * @refcount_order: refcount bits power-of-2 exponent + * @extended_l2: true if the image has extended L2 entries * * Returns: Total number of bytes required for the fully allocated image * (including metadata). */ static int64_t qcow2_calc_prealloc_size(int64_t total_size, size_t cluster_size, - int refcount_order) + int refcount_order, + bool extended_l2) { int64_t meta_size = 0; uint64_t nl1e, nl2e; int64_t aligned_total_size = ROUND_UP(total_size, cluster_size); + size_t l2e_size = extended_l2 ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL; /* header: 1 cluster */ meta_size += cluster_size; /* total size of L2 tables */ nl2e = aligned_total_size / cluster_size; - nl2e = ROUND_UP(nl2e, cluster_size / sizeof(uint64_t)); - meta_size += nl2e * sizeof(uint64_t); + nl2e = ROUND_UP(nl2e, cluster_size / l2e_size); + meta_size += nl2e * l2e_size; /* total size of L1 tables */ - nl1e = nl2e * sizeof(uint64_t) / cluster_size; + nl1e = nl2e * l2e_size / cluster_size; nl1e = ROUND_UP(nl1e, cluster_size / sizeof(uint64_t)); meta_size += nl1e * sizeof(uint64_t); @@ -4838,6 +4841,8 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, PreallocMode prealloc; bool has_backing_file; bool has_luks; + bool extended_l2 = false; /* Set to false until the option is added */ + size_t l2e_size; /* Parse image creation options */ cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err); @@ -4896,8 +4901,9 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, virtual_size = ROUND_UP(virtual_size, cluster_size); /* Check that virtual disk size is valid */ + l2e_size = extended_l2 ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL; l2_tables = DIV_ROUND_UP(virtual_size / cluster_size, - cluster_size / sizeof(uint64_t)); + cluster_size / l2e_size); if (l2_tables * sizeof(uint64_t) > QCOW_MAX_L1_SIZE) { error_setg(&local_err, "The image size is too large " "(try using a larger cluster size)"); @@ -4960,9 +4966,9 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, } info = g_new0(BlockMeasureInfo, 1); - info->fully_allocated = + info->fully_allocated = luks_payload_size + qcow2_calc_prealloc_size(virtual_size, cluster_size, - ctz32(refcount_bits)) + luks_payload_size; + ctz32(refcount_bits), extended_l2); /* * Remove data clusters that are not required. This overestimates the From patchwork Fri Jul 10 16:13:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE790C433DF for ; Fri, 10 Jul 2020 16:34:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76C3D20657 for ; Fri, 10 Jul 2020 16:34:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="HWZVEe2s" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76C3D20657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52264 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvz4-0007SK-KU for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:34:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42800) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfV-0003NW-Bj; Fri, 10 Jul 2020 12:14:33 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43855) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004wJ-Ng; Fri, 10 Jul 2020 12:14:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=5rOS//XvdYLuyPAIOyXQEo7A8pGRqGa6eCbbqldP+do=; b=HWZVEe2siHoHasUqi3tfX1ocmgdwEuXw4KBfcvupXEEQCYrlwUoTnox2Gdg0qlBjeZs8hO+5LYTD4RaurqDzKEanvg8Mzqq4e6yo8uD7vu3fjFBHC9sJ1UyPOnIgfc1zxvzXL9q4Mdwry9H689BsMXb6GZgg4r5Ioy9Jb+HZ65tP9Dq9K81s9Yo/WhIcR1escCUqIYaYuvjlJoXcbnySLF6zMRd5qC1kSXBMfKIkdxO9HmenmW7aK8P6vfjzy9o+poQk7VnXkXd9Ebx9sHf5Wd4aYaMrp/scSb+CClNrB71KA6Uxz/R5qZss374zfvDOIdrTLSUAuoVaqoo2PwgdFA==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003k9-NZ; Fri, 10 Jul 2020 18:13:37 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RW-KL; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit Date: Fri, 10 Jul 2020 18:13:13 +0200 Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- qapi/block-core.json | 7 +++ block/qcow2.h | 8 ++- include/block/block_int.h | 1 + block/qcow2.c | 66 ++++++++++++++++++-- tests/qemu-iotests/031.out | 8 +-- tests/qemu-iotests/036.out | 4 +- tests/qemu-iotests/049.out | 102 +++++++++++++++---------------- tests/qemu-iotests/060.out | 1 + tests/qemu-iotests/061.out | 20 +++--- tests/qemu-iotests/065 | 12 ++-- tests/qemu-iotests/082.out | 39 +++++++++--- tests/qemu-iotests/085.out | 38 ++++++------ tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/185.out | 8 +-- tests/qemu-iotests/198 | 2 + tests/qemu-iotests/206.out | 4 ++ tests/qemu-iotests/242.out | 5 ++ tests/qemu-iotests/255.out | 8 +-- tests/qemu-iotests/274.out | 49 ++++++++------- tests/qemu-iotests/280.out | 2 +- tests/qemu-iotests/291.out | 2 + tests/qemu-iotests/common.filter | 1 + 23 files changed, 253 insertions(+), 140 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index b20332e592..0d4231ee98 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -66,6 +66,9 @@ # standalone (read-only) raw image without looking at qcow2 # metadata (since: 4.0) # +# @extended-l2: true if the image has extended L2 entries; only valid for +# compat >= 1.1 (since 5.1) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -87,6 +90,7 @@ 'compat': 'str', '*data-file': 'str', '*data-file-raw': 'bool', + '*extended-l2': 'bool', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -4318,6 +4322,8 @@ # @data-file-raw: True if the external data file must stay valid as a # standalone (read-only) raw image without looking at qcow2 # metadata (default: false; since: 4.0) +# @extended-l2 True to make the image have extended L2 entries +# (default: false; since 5.1) # @size: Size of the virtual disk in bytes # @version: Compatibility level (default: v3) # @backing-file: File name of the backing file if a backing file @@ -4338,6 +4344,7 @@ 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', '*data-file-raw': 'bool', + '*extended-l2': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/block/qcow2.h b/block/qcow2.h index f3499e53bf..065ec3df0b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -246,15 +246,18 @@ enum { QCOW2_INCOMPAT_CORRUPT_BITNR = 1, QCOW2_INCOMPAT_DATA_FILE_BITNR = 2, QCOW2_INCOMPAT_COMPRESSION_BITNR = 3, + QCOW2_INCOMPAT_EXTL2_BITNR = 4, QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, QCOW2_INCOMPAT_COMPRESSION = 1 << QCOW2_INCOMPAT_COMPRESSION_BITNR, + QCOW2_INCOMPAT_EXTL2 = 1 << QCOW2_INCOMPAT_EXTL2_BITNR, QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY | QCOW2_INCOMPAT_CORRUPT | QCOW2_INCOMPAT_DATA_FILE - | QCOW2_INCOMPAT_COMPRESSION, + | QCOW2_INCOMPAT_COMPRESSION + | QCOW2_INCOMPAT_EXTL2, }; /* Compatible feature bits */ @@ -581,8 +584,7 @@ typedef enum QCow2MetadataOverlap { static inline bool has_subclusters(BDRVQcow2State *s) { - /* FIXME: Return false until this feature is complete */ - return false; + return s->incompatible_features & QCOW2_INCOMPAT_EXTL2; } static inline size_t l2_entry_size(BDRVQcow2State *s) diff --git a/include/block/block_int.h b/include/block/block_int.h index 3d6cf88592..4c6545b950 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -58,6 +58,7 @@ #define BLOCK_OPT_DATA_FILE "data_file" #define BLOCK_OPT_DATA_FILE_RAW "data_file_raw" #define BLOCK_OPT_COMPRESSION_TYPE "compression_type" +#define BLOCK_OPT_EXTL2 "extended_l2" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2.c b/block/qcow2.c index 8ca0f22069..b7a1cae39c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1449,6 +1449,12 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->subcluster_size = s->cluster_size / s->subclusters_per_cluster; s->subcluster_bits = ctz32(s->subcluster_size); + if (s->subcluster_size < (1 << MIN_CLUSTER_BITS)) { + error_setg(errp, "Unsupported subcluster size: %d", s->subcluster_size); + ret = -EINVAL; + goto fail; + } + /* Check support for various header values */ if (header.refcount_order > 6) { error_setg(errp, "Reference count entry width too large; may not " @@ -2934,6 +2940,11 @@ int qcow2_update_header(BlockDriverState *bs) .bit = QCOW2_INCOMPAT_COMPRESSION_BITNR, .name = "compression type", }, + { + .type = QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit = QCOW2_INCOMPAT_EXTL2_BITNR, + .name = "extended L2 entries", + }, { .type = QCOW2_FEAT_TYPE_COMPATIBLE, .bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, @@ -3270,7 +3281,8 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size, return meta_size + aligned_total_size; } -static bool validate_cluster_size(size_t cluster_size, Error **errp) +static bool validate_cluster_size(size_t cluster_size, bool extended_l2, + Error **errp) { int cluster_bits = ctz32(cluster_size); if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS || @@ -3280,16 +3292,28 @@ static bool validate_cluster_size(size_t cluster_size, Error **errp) "%dk", 1 << MIN_CLUSTER_BITS, 1 << (MAX_CLUSTER_BITS - 10)); return false; } + + if (extended_l2) { + unsigned min_cluster_size = + (1 << MIN_CLUSTER_BITS) * QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER; + if (cluster_size < min_cluster_size) { + error_setg(errp, "Extended L2 entries are only supported with " + "cluster sizes of at least %u bytes", min_cluster_size); + return false; + } + } + return true; } -static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp) +static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, bool extended_l2, + Error **errp) { size_t cluster_size; cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE, DEFAULT_CLUSTER_SIZE); - if (!validate_cluster_size(cluster_size, errp)) { + if (!validate_cluster_size(cluster_size, extended_l2, errp)) { return 0; } return cluster_size; @@ -3403,7 +3427,20 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) cluster_size = DEFAULT_CLUSTER_SIZE; } - if (!validate_cluster_size(cluster_size, errp)) { + if (!qcow2_opts->has_extended_l2) { + qcow2_opts->extended_l2 = false; + } + if (qcow2_opts->extended_l2) { + if (version < 3) { + error_setg(errp, "Extended L2 entries are only supported with " + "compatibility level 1.1 and above (use version=v3 or " + "greater)"); + ret = -EINVAL; + goto out; + } + } + + if (!validate_cluster_size(cluster_size, qcow2_opts->extended_l2, errp)) { ret = -EINVAL; goto out; } @@ -3554,6 +3591,11 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) cpu_to_be64(QCOW2_INCOMPAT_COMPRESSION); } + if (qcow2_opts->extended_l2) { + header->incompatible_features |= + cpu_to_be64(QCOW2_INCOMPAT_EXTL2); + } + ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); if (ret < 0) { @@ -3731,6 +3773,7 @@ static int coroutine_fn qcow2_co_create_opts(BlockDriver *drv, { BLOCK_OPT_BACKING_FMT, "backing-fmt" }, { BLOCK_OPT_CLUSTER_SIZE, "cluster-size" }, { BLOCK_OPT_LAZY_REFCOUNTS, "lazy-refcounts" }, + { BLOCK_OPT_EXTL2, "extended-l2" }, { BLOCK_OPT_REFCOUNT_BITS, "refcount-bits" }, { BLOCK_OPT_ENCRYPT, BLOCK_OPT_ENCRYPT_FORMAT }, { BLOCK_OPT_COMPAT_LEVEL, "version" }, @@ -4847,11 +4890,14 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, PreallocMode prealloc; bool has_backing_file; bool has_luks; - bool extended_l2 = false; /* Set to false until the option is added */ + bool extended_l2; size_t l2e_size; /* Parse image creation options */ - cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err); + extended_l2 = qemu_opt_get_bool_del(opts, BLOCK_OPT_EXTL2, false); + + cluster_size = qcow2_opt_get_cluster_size_del(opts, extended_l2, + &local_err); if (local_err) { goto err; } @@ -5047,6 +5093,8 @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .corrupt = s->incompatible_features & QCOW2_INCOMPAT_CORRUPT, .has_corrupt = true, + .has_extended_l2 = true, + .extended_l2 = has_subclusters(s), .refcount_bits = s->refcount_bits, .has_bitmaps = !!bitmaps, .bitmaps = bitmaps, @@ -5769,6 +5817,12 @@ static QemuOptsList qcow2_create_opts = { .help = "qcow2 cluster size", \ .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \ }, \ + { \ + .name = BLOCK_OPT_EXTL2, \ + .type = QEMU_OPT_BOOL, \ + .help = "Extended L2 tables", \ + .def_value_str = "off" \ + }, \ { \ .name = BLOCK_OPT_PREALLOC, \ .type = QEMU_OPT_STRING, \ diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index 4b21d6a9ba..0054c2ed97 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -117,7 +117,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data Header extension: @@ -150,7 +150,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data Header extension: @@ -164,7 +164,7 @@ No errors were found on the image. magic 0x514649fb version 3 -backing_file_offset 0x210 +backing_file_offset 0x240 backing_file_size 0x17 cluster_bits 16 size 67108864 @@ -188,7 +188,7 @@ data 'host_device' Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data Header extension: diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index a9bed828e5..1fa7cad28d 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -26,7 +26,7 @@ compatible_features [] autoclear_features [63] Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data @@ -38,7 +38,7 @@ compatible_features [] autoclear_features [] Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data *** done diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out index e77966446b..fa5595653c 100644 --- a/tests/qemu-iotests/049.out +++ b/tests/qemu-iotests/049.out @@ -4,90 +4,90 @@ QA output created by 049 == 1. Traditional size parameter == qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16 == 2. Specifying size via -o == qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16 == 3. Invalid sizes == @@ -129,84 +129,84 @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once == Check correct interpretation of suffixes for cluster size == qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 == Check compat level option == qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16 == Check preallocation option == qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 == Check encryption option == qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 == Check lazy_refcounts option (only with v3) == qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater) -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16 *** done diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index fa3d68f0df..bb98fa5db8 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -21,6 +21,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: true + extended l2: false qemu-io: can't open device TEST_DIR/t.IMGFMT: IMGFMT: Image is corrupt; cannot be opened read/write no file open, try 'help open' read 512/512 bytes at offset 0 diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index b0f8befe30..caa53e87cc 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -26,7 +26,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data magic 0x514649fb @@ -84,7 +84,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data magic 0x514649fb @@ -140,7 +140,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data ERROR cluster 5 refcount=0 reference=1 @@ -195,7 +195,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data magic 0x514649fb @@ -264,7 +264,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data read 65536/65536 bytes at offset 44040192 @@ -326,7 +326,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data ERROR cluster 5 refcount=0 reference=1 @@ -355,7 +355,7 @@ header_length 112 Header extension: magic 0x6803f857 (Feature table) -length 336 +length 384 data read 131072/131072 bytes at offset 0 @@ -529,6 +529,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. === Try changing the external data file === @@ -550,6 +551,7 @@ Format specific information: data file: foo data file raw: false corrupt: false + extended l2: false qemu-img: Could not open 'TEST_DIR/t.IMGFMT': 'data-file' is required for this image image: TEST_DIR/t.IMGFMT @@ -563,6 +565,7 @@ Format specific information: refcount bits: 16 data file raw: false corrupt: false + extended l2: false === Clearing and setting data-file-raw === @@ -579,6 +582,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: true corrupt: false + extended l2: false No errors were found on the image. image: TEST_DIR/t.IMGFMT file format: IMGFMT @@ -592,6 +596,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. qemu-img: data-file-raw cannot be set on existing images image: TEST_DIR/t.IMGFMT @@ -606,5 +611,6 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. *** done diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 index 18dc488c7a..29a7f7ad60 100755 --- a/tests/qemu-iotests/065 +++ b/tests/qemu-iotests/065 @@ -98,20 +98,20 @@ class TestQCow3NotLazy(TestQemuImgInfo): img_options = 'compat=1.1,lazy_refcounts=off' json_compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: false', 'refcount bits: 16', - 'corrupt: false' ] + 'corrupt: false', 'extended l2: false' ] class TestQCow3Lazy(TestQemuImgInfo): '''Testing a qcow2 version 3 image with lazy refcounts enabled''' img_options = 'compat=1.1,lazy_refcounts=on' json_compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: true', 'refcount bits: 16', - 'corrupt: false' ] + 'corrupt: false', 'extended l2: false' ] class TestQCow3NotLazyQMP(TestQMP): '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening @@ -120,7 +120,7 @@ class TestQCow3NotLazyQMP(TestQMP): qemu_options = 'lazy-refcounts=on' compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } class TestQCow3LazyQMP(TestQMP): @@ -130,7 +130,7 @@ class TestQCow3LazyQMP(TestQMP): qemu_options = 'lazy-refcounts=off' compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } TestImageInfoSpecific = None TestQemuImgInfo = None diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index a4a2b69030..f9948a854f 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -3,14 +3,14 @@ QA output created by 082 === create: Options specified more than once === Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) cluster_size: 65536 Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -21,9 +21,10 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -34,9 +35,10 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -62,6 +64,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -86,6 +89,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -110,6 +114,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -134,6 +139,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -158,6 +164,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -182,6 +189,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -206,6 +214,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -230,6 +239,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -237,10 +247,10 @@ Supported options: size= - Virtual disk size Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help lazy_refcounts=off refcount_bits=16 Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? lazy_refcounts=off refcount_bits=16 Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2, @@ -269,6 +279,7 @@ Supported qcow2 options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -290,7 +301,7 @@ qemu-img: Format driver 'bochs' does not support image creation === convert: Options specified more than once === Testing: create -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -314,6 +325,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: convert -O qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -326,6 +338,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: convert -O qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -353,6 +366,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -377,6 +391,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -401,6 +416,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -425,6 +441,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -449,6 +466,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -473,6 +491,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -497,6 +516,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -521,6 +541,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -560,6 +581,7 @@ Supported qcow2 options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -605,6 +627,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=130M -o lazy_refcounts=off TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT @@ -617,6 +640,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=8M -o lazy_refcounts=on -o size=132M TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT @@ -629,6 +653,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=4M,size=148M TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out index d68c06efdf..a248900aa5 100644 --- a/tests/qemu-iotests/085.out +++ b/tests/qemu-iotests/085.out @@ -13,7 +13,7 @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728 === Create a single snapshot on virtio0 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Invalid command - missing device and nodename === @@ -30,40 +30,40 @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compress === Create several transactional group snapshots === { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Create a couple of snapshots using blockdev-snapshot === diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out index a2172a1308..13e0c4f5a7 100644 --- a/tests/qemu-iotests/144.out +++ b/tests/qemu-iotests/144.out @@ -9,7 +9,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912 { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Performing block-commit on active layer === @@ -31,6 +31,6 @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=z === Performing Live Snapshot 2 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} *** done diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out index 29e9db3497..ce23340670 100644 --- a/tests/qemu-iotests/182.out +++ b/tests/qemu-iotests/182.out @@ -13,7 +13,7 @@ Is another process using the image [TEST_DIR/t.qcow2]? {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}} {'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } } -Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16 {"return": {}} {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}} diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out index 62d1ab74d3..339438ac68 100644 --- a/tests/qemu-iotests/185.out +++ b/tests/qemu-iotests/185.out @@ -9,14 +9,14 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 === Creating backing chain === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } } wrote 4194304/4194304 bytes at offset 0 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Start commit job and exit qemu === @@ -48,7 +48,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zli { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} @@ -62,7 +62,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_typ { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198 index aeb059d5ea..9362d21513 100755 --- a/tests/qemu-iotests/198 +++ b/tests/qemu-iotests/198 @@ -94,6 +94,7 @@ echo "== checking image base ==" $QEMU_IMG info --image-opts $IMGSPECBASE | _filter_img_info --format-specific \ | sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \ -e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \ + -e '/extended l2:/ D' \ | _filter_json_filename echo @@ -101,6 +102,7 @@ echo "== checking image layer ==" $QEMU_IMG info --image-opts $IMGSPECLAYER | _filter_img_info --format-specific \ | sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \ -e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \ + -e '/extended l2:/ D' \ | _filter_json_filename diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out index 1a14255a83..363c5abe35 100644 --- a/tests/qemu-iotests/206.out +++ b/tests/qemu-iotests/206.out @@ -22,6 +22,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false === Successful image creation (inline blockdev-add, explicit defaults) === @@ -45,6 +46,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false === Successful image creation (v3 non-default options) === @@ -68,6 +70,7 @@ Format specific information: lazy refcounts: true refcount bits: 1 corrupt: false + extended l2: false === Successful image creation (v2 non-default options) === @@ -146,6 +149,7 @@ Format specific information: payload offset: 528384 master key iters: XXX corrupt: false + extended l2: false === Invalid BlockdevRef === diff --git a/tests/qemu-iotests/242.out b/tests/qemu-iotests/242.out index 091b9126ce..3759c99284 100644 --- a/tests/qemu-iotests/242.out +++ b/tests/qemu-iotests/242.out @@ -16,6 +16,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false No bitmap in JSON format output @@ -42,6 +43,7 @@ Format specific information: granularity: 32768 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -80,6 +82,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -123,6 +126,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -167,5 +171,6 @@ Format specific information: granularity: 16384 refcount bits: 16 corrupt: false + extended l2: false Test complete diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out index d74903db99..33b7f22de3 100644 --- a/tests/qemu-iotests/255.out +++ b/tests/qemu-iotests/255.out @@ -3,9 +3,9 @@ Finishing a commit job with background reads === Create backing chain and start VM === -Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 === Start background read requests === @@ -23,9 +23,9 @@ Closing the VM while a job is being cancelled === Create images and start VM === -Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 wrote 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out index d248a1e21b..3bc8570304 100644 --- a/tests/qemu-iotests/274.out +++ b/tests/qemu-iotests/274.out @@ -1,9 +1,9 @@ == Commit tests == -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -56,6 +56,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -64,11 +65,11 @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) === Testing HMP commit (top -> mid) === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -86,6 +87,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -94,11 +96,11 @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) === Testing QMP active commit (top -> mid) === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16 wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -122,6 +124,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -131,9 +134,9 @@ read 1048576/1048576 bytes at offset 1048576 == Resize tests == === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 5368709120 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -150,9 +153,9 @@ read 65536/65536 bytes at offset 5368709120 { "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}] === preallocation=metadata === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 33285996544 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -174,9 +177,9 @@ read 65536/65536 bytes at offset 33285996544 { "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}] === preallocation=falloc === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 9437184 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -193,9 +196,9 @@ read 65536/65536 bytes at offset 9437184 { "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}] === preallocation=full === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 11534336 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -212,9 +215,9 @@ read 65536/65536 bytes at offset 11534336 { "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 259072 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -232,9 +235,9 @@ read 65536/65536 bytes at offset 259072 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 344064 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -251,9 +254,9 @@ read 65536/65536 bytes at offset 344064 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16 wrote 65536/65536 bytes at offset 446464 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out index fc59b9bc5c..09a0f1a7cb 100644 --- a/tests/qemu-iotests/280.out +++ b/tests/qemu-iotests/280.out @@ -1,4 +1,4 @@ -Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 === Launch VM === Enabling migration QMP events on VM... diff --git a/tests/qemu-iotests/291.out b/tests/qemu-iotests/291.out index 9f661515b4..ee89a72885 100644 --- a/tests/qemu-iotests/291.out +++ b/tests/qemu-iotests/291.out @@ -41,6 +41,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 10 MiB (10485760 bytes) @@ -65,6 +66,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false === Check bitmap contents === diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index d967adc59a..72453e36b1 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -250,6 +250,7 @@ _filter_img_info() -e "/adapter_type: '[^']*'/d" \ -e "/hwversion: '[^']*'/d" \ -e "/lazy_refcounts: \\(on\\|off\\)/d" \ + -e "/extended_l2=\\(on\\|off\\)/d" \ -e "/block_size: [0-9]\\+/d" \ -e "/block_state_zero: \\(on\\|off\\)/d" \ -e "/log_size: [0-9]\\+/d" \ From patchwork Fri Jul 10 16:13:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E076DC433EA for ; Fri, 10 Jul 2020 16:31:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC0F120657 for ; Fri, 10 Jul 2020 16:31:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="sMV4HH7T" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC0F120657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvvv-00016B-SL for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:31:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfP-00036U-52; Fri, 10 Jul 2020 12:14:27 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43818) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004w5-3T; Fri, 10 Jul 2020 12:14:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Nd8wD7TYfXytYxR8lRZ8t5ifgIxXWdbTW6J+QjApulg=; b=sMV4HH7TAaAgxxMaCkn5YQ8ZNVEKiaNFzwm8e/znRiZFDIbLJxs0Se7cQWAl2B2GdCiP6fgz8p4p7Drs2dxXU1ptEMlVs0dXS4ioWMkZZkiD5694TO8LlyP3l6SFkr3/M3USv4GyBBvSZcdbQxyBmQ5S4yUD14FLtjNYGHpYnzJx83C8vTfvVK3/YmCwTtT4jd1baoUgE+R6pY3aHnKyri5lzlM5PEsm4bMa6zy0Ym8c7DSoAETwY+i7sE7nfPqQXeNJElELQedbJhpiD4rYL5GJkB2YVWPT1dGoxNH0M/5P8/JMe9XezASdlRGfebTJr30tKlUjYkJv/nxKXzVnqA==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003kA-MY; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001RZ-LQ; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 32/34] qcow2: Allow preallocation and backing files if extended_l2 is set Date: Fri, 10 Jul 2020 18:13:14 +0200 Message-Id: <6d5b0f38e7dc5f2f31d8cab1cb92044e9909aece.1594396418.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Traditional qcow2 images don't allow preallocation if a backing file is set. This is because once a cluster is allocated there is no way to tell that its data should be read from the backing file. Extended L2 entries have individual allocation bits for each subcluster, and therefore it is perfectly possible to have an allocated cluster with all its subclusters unallocated. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2.c | 7 ++++--- tests/qemu-iotests/206.out | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index b7a1cae39c..806b194761 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3449,10 +3449,11 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) qcow2_opts->preallocation = PREALLOC_MODE_OFF; } if (qcow2_opts->has_backing_file && - qcow2_opts->preallocation != PREALLOC_MODE_OFF) + qcow2_opts->preallocation != PREALLOC_MODE_OFF && + !qcow2_opts->extended_l2) { - error_setg(errp, "Backing file and preallocation cannot be used at " - "the same time"); + error_setg(errp, "Backing file and preallocation can only be used at " + "the same time if extended_l2 is on"); ret = -EINVAL; goto out; } diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out index 363c5abe35..a100849fcb 100644 --- a/tests/qemu-iotests/206.out +++ b/tests/qemu-iotests/206.out @@ -203,7 +203,7 @@ Job failed: Different refcount widths than 16 bits require compatibility level 1 === Invalid backing file options === {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"backing-file": "/dev/null", "driver": "qcow2", "file": "node0", "preallocation": "full", "size": 67108864}}} {"return": {}} -Job failed: Backing file and preallocation cannot be used at the same time +Job failed: Backing file and preallocation can only be used at the same time if extended_l2 is on {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} From patchwork Fri Jul 10 16:13:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 278074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CF08C433DF for ; Fri, 10 Jul 2020 16:29:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D9EC220657 for ; Fri, 10 Jul 2020 16:29:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="sb414Aqn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9EC220657 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtvth-0005sg-5K for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 12:29:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42808) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtvfV-0003Oa-Ni; Fri, 10 Jul 2020 12:14:33 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:43849) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtvfJ-0004wC-MH; Fri, 10 Jul 2020 12:14:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=OxviijHyK8TVhWv5hZ2rzo6rpZNC2nXRJo7+cof5oXE=; b=sb414Aqn0lyQcq5rTLKLlio/epBQvzGj1ieXmQOBRpWPHtGCfGiidcaNIZLuDKaRjDwAtgswx5+FaaRtiZQTazD9J2GP0Y7JmThCKtVCy00TmqbbWnM9BnRfwDxa2FMG5ks8hTKsWPIfn7hahsOKnjz2ZBSTu/tXjpBRVF0gR6RCVBovkMmpbl09qp8VGNix0Ql5oXkhNePrYJYk1LiJ98hPfnDavAGYUcUDustYVT8VrU2EMqNl17du72UioEpbBmmJJdcuMF7KFuy5Sw9aYS9qU1dmUNVsBJw8aFClDtWBW28PLNdhxArMwsIUbn7lDwEzxruU9s+3clyV3Hducw==; Received: from [81.0.43.0] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jtvea-0003kC-MB; Fri, 10 Jul 2020 18:13:36 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jtveL-0001Rf-Nc; Fri, 10 Jul 2020 18:13:21 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v11 34/34] iotests: Add tests for qcow2 images with extended L2 entries Date: Fri, 10 Jul 2020 18:13:16 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 12:13:35 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- tests/qemu-iotests/271 | 901 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/271.out | 726 ++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 1628 insertions(+) create mode 100755 tests/qemu-iotests/271 create mode 100644 tests/qemu-iotests/271.out diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 new file mode 100755 index 0000000000..39ff462328 --- /dev/null +++ b/tests/qemu-iotests/271 @@ -0,0 +1,901 @@ +#!/bin/bash +# +# Test qcow2 images with extended L2 entries +# +# Copyright (C) 2019-2020 Igalia, S.L. +# Author: Alberto Garcia +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=berto@igalia.com + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_IMG.raw" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file nfs +_supported_os Linux +_unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file refcount_bits=1[^0-9] + +l2_offset=$((0x40000)) + +_verify_img() +{ + $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.raw" | grep -v 'Images are identical' + $QEMU_IMG check "$TEST_IMG" | _filter_qemu_img_check | \ + grep -v 'No errors were found on the image' +} + +# Compare the bitmap of an extended L2 entry against an expected value +_verify_l2_bitmap() +{ + entry_no="$1" # L2 entry number, starting from 0 + expected_alloc="$alloc" # Space-separated list of allocated subcluster indexes + expected_zero="$zero" # Space-separated list of zero subcluster indexes + + offset=$(($l2_offset + $entry_no * 16)) + entry=$(peek_file_be "$TEST_IMG" $offset 8) + offset=$(($offset + 8)) + bitmap=$(peek_file_be "$TEST_IMG" $offset 8) + + expected_bitmap=0 + for bit in $expected_alloc; do + expected_bitmap=$(($expected_bitmap | (1 << $bit))) + done + for bit in $expected_zero; do + expected_bitmap=$(($expected_bitmap | (1 << (32 + $bit)))) + done + printf -v expected_bitmap "%u" $expected_bitmap # Convert to unsigned + + printf "L2 entry #%d: 0x%016x %016x\n" "$entry_no" "$entry" "$bitmap" + if [ "$bitmap" != "$expected_bitmap" ]; then + printf "ERROR: expecting bitmap 0x%016x\n" "$expected_bitmap" + fi +} + +# This should be called as _run_test c=XXX sc=XXX off=XXX len=XXX cmd=XXX +# c: cluster number (0 if unset) +# sc: subcluster number inside cluster @c (0 if unset) +# off: offset inside subcluster @sc, in kilobytes (0 if unset) +# len: request length, passed directly to qemu-io (e.g: 256, 4k, 1M, ...) +# cmd: the command to pass to qemu-io, must be one of +# write -> write +# zero -> write -z +# unmap -> write -z -u +# compress -> write -c +# discard -> discard +_run_test() +{ + unset c sc off len cmd + for var in "$@"; do eval "$var"; done + case "${cmd:-write}" in + zero) + cmd="write -q -z";; + unmap) + cmd="write -q -z -u";; + compress) + pat=$((${pat:-0} + 1)) + cmd="write -q -c -P ${pat}";; + write) + pat=$((${pat:-0} + 1)) + cmd="write -q -P ${pat}";; + discard) + cmd="discard -q";; + *) + echo "Unknown option $cmd" + exit 1;; + esac + c="${c:-0}" + sc="${sc:-0}" + off="${off:-0}" + offset="$(($c * 64 + $sc * 2 + $off))" + [ "$offset" != 0 ] && offset="${offset}k" + cmd="$cmd ${offset} ${len}" + raw_cmd=$(echo $cmd | sed s/-c//) # Raw images don't support -c + echo $cmd | sed 's/-P [0-9][0-9]\?/-P PATTERN/' + $QEMU_IO -c "$cmd" "$TEST_IMG" | _filter_qemu_io + $QEMU_IO -c "$raw_cmd" -f raw "$TEST_IMG.raw" | _filter_qemu_io + _verify_img + _verify_l2_bitmap "$c" +} + +_reset_img() +{ + size="$1" + $QEMU_IMG create -f raw "$TEST_IMG.raw" "$size" | _filter_img_create + if [ "$use_backing_file" = "yes" ]; then + $QEMU_IMG create -f raw "$TEST_IMG.base" "$size" | _filter_img_create + $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.base" | _filter_qemu_io + $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.raw" | _filter_qemu_io + _make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" "$size" + else + _make_test_img -o extended_l2=on "$size" + fi +} + +############################################################ +############################################################ +############################################################ + +# Test that writing to an image with subclusters produces the expected +# results, in images with and without backing files +for use_backing_file in yes no; do + echo + echo "### Standard write tests (backing file: $use_backing_file) ###" + echo + _reset_img 1M + ### Write subcluster #0 (beginning of subcluster) ### + alloc="0"; zero="" + _run_test sc=0 len=1k + + ### Write subcluster #1 (middle of subcluster) ### + alloc="0 1"; zero="" + _run_test sc=1 off=1 len=512 + + ### Write subcluster #2 (end of subcluster) ### + alloc="0 1 2"; zero="" + _run_test sc=2 off=1 len=1k + + ### Write subcluster #3 (full subcluster) ### + alloc="0 1 2 3"; zero="" + _run_test sc=3 len=2k + + ### Write subclusters #4-6 (full subclusters) ### + alloc="$(seq 0 6)"; zero="" + _run_test sc=4 len=6k + + ### Write subclusters #7-9 (partial subclusters) ### + alloc="$(seq 0 9)"; zero="" + _run_test sc=7 off=1 len=4k + + ### Write subcluster #16 (partial subcluster) ### + alloc="$(seq 0 9) 16"; zero="" + _run_test sc=16 len=1k + + ### Write subcluster #31-#33 (cluster overlap) ### + alloc="$(seq 0 9) 16 31"; zero="" + _run_test sc=31 off=1 len=4k + alloc="0 1" ; zero="" + _verify_l2_bitmap 1 + + ### Zero subcluster #1 + alloc="0 $(seq 2 9) 16 31"; zero="1" + _run_test sc=1 len=2k cmd=zero + + ### Zero cluster #0 + alloc=""; zero="$(seq 0 31)" + _run_test sc=0 len=64k cmd=zero + + ### Fill cluster #0 with data + alloc="$(seq 0 31)"; zero="" + _run_test sc=0 len=64k + + ### Zero and unmap half of cluster #0 (this won't unmap it) + alloc="$(seq 16 31)"; zero="$(seq 0 15)" + _run_test sc=0 len=32k cmd=unmap + + ### Zero and unmap cluster #0 + alloc=""; zero="$(seq 0 31)" + _run_test sc=0 len=64k cmd=unmap + + ### Write subcluster #1 (middle of subcluster) + alloc="1"; zero="0 $(seq 2 31)" + _run_test sc=1 off=1 len=512 + + ### Fill cluster #0 with data + alloc="$(seq 0 31)"; zero="" + _run_test sc=0 len=64k + + ### Discard cluster #0 + alloc=""; zero="$(seq 0 31)" + _run_test sc=0 len=64k cmd=discard + + ### Write compressed data to cluster #0 + alloc=""; zero="" + _run_test sc=0 len=64k cmd=compress + + ### Write subcluster #1 (middle of subcluster) + alloc="$(seq 0 31)"; zero="" + _run_test sc=1 off=1 len=512 +done + +############################################################ +############################################################ +############################################################ + +# calculate_l2_meta() checks if none of the clusters affected by a +# write operation need COW or changes to their L2 metadata and simply +# returns when they don't. This is a test for that optimization. +# Here clusters #0-#3 are overwritten but only #1 and #2 need changes. +echo +echo '### Overwriting several clusters without COW ###' +echo +use_backing_file="no" _reset_img 1M +# Write cluster #0, subclusters #12-#31 +alloc="$(seq 12 31)"; zero="" +_run_test sc=12 len=40k + +# Write cluster #1, subcluster #13 +alloc="13"; zero="" +_run_test c=1 sc=13 len=2k + +# Zeroize cluster #2, subcluster #14 +alloc="14"; zero="" +_run_test c=2 sc=14 len=2k +alloc=""; zero="14" +_run_test c=2 sc=14 len=2k cmd=zero + +# Write cluster #3, subclusters #0-#16 +alloc="$(seq 0 16)"; zero="" +_run_test c=3 sc=0 len=34k + +# Write from cluster #0, subcluster #12 to cluster #3, subcluster #11 +alloc="$(seq 12 31)"; zero="" +_run_test sc=12 len=192k +alloc="$(seq 0 31)"; zero="" +_verify_l2_bitmap 1 +_verify_l2_bitmap 2 + +alloc="$(seq 0 16)"; zero="" +_verify_l2_bitmap 3 + +############################################################ +############################################################ +############################################################ + +# Test different patterns of writing zeroes +for use_backing_file in yes no; do + echo + echo "### Writing zeroes 1: unallocated clusters (backing file: $use_backing_file) ###" + echo + # Note that the image size is not a multiple of the cluster size + _reset_img 2083k + + # Cluster-aligned request from clusters #0 to #2 + alloc=""; zero="$(seq 0 31)" + _run_test c=0 sc=0 len=192k cmd=zero + _verify_l2_bitmap 1 + _verify_l2_bitmap 2 + + # Subcluster-aligned request from clusters #3 to #5 + alloc=""; zero="$(seq 16 31)" + _run_test c=3 sc=16 len=128k cmd=zero + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 4 + alloc=""; zero="$(seq 0 15)" + _verify_l2_bitmap 5 + + # Unaligned request from clusters #6 to #8 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="$(seq 16 31)" # copy-on-write happening here + else + alloc=""; zero="$(seq 15 31)" + fi + _run_test c=6 sc=15 off=1 len=128k cmd=zero + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 7 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here + else + alloc=""; zero="$(seq 0 15)" + fi + _verify_l2_bitmap 8 + + echo + echo "### Writing zeroes 2: allocated clusters (backing file: $use_backing_file) ###" + echo + alloc="$(seq 0 31)"; zero="" + _run_test c=9 sc=0 len=576k + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + _verify_l2_bitmap 12 + _verify_l2_bitmap 13 + _verify_l2_bitmap 14 + _verify_l2_bitmap 15 + _verify_l2_bitmap 16 + _verify_l2_bitmap 17 + + # Cluster-aligned request from clusters #9 to #11 + alloc=""; zero="$(seq 0 31)" + _run_test c=9 sc=0 len=192k cmd=zero + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + + # Subcluster-aligned request from clusters #12 to #14 + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=12 sc=16 len=128k cmd=zero + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 13 + alloc="$(seq 16 31)"; zero="$(seq 0 15)" + _verify_l2_bitmap 14 + + # Unaligned request from clusters #15 to #17 + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=15 sc=15 off=1 len=128k cmd=zero + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 16 + alloc="$(seq 15 31)"; zero="$(seq 0 14)" + _verify_l2_bitmap 17 + + echo + echo "### Writing zeroes 3: compressed clusters (backing file: $use_backing_file) ###" + echo + alloc=""; zero="" + for c in $(seq 18 28); do + _run_test c=$c sc=0 len=64k cmd=compress + done + + # Cluster-aligned request from clusters #18 to #20 + alloc=""; zero="$(seq 0 31)" + _run_test c=18 sc=0 len=192k cmd=zero + _verify_l2_bitmap 19 + _verify_l2_bitmap 20 + + # Subcluster-aligned request from clusters #21 to #23. + # We cannot partially zero a compressed cluster so the code + # returns -ENOTSUP, which means copy-on-write of the compressed + # data and fill the rest with actual zeroes on disk. + # TODO: cluster #22 should use the 'all zeroes' bits. + alloc="$(seq 0 31)"; zero="" + _run_test c=21 sc=16 len=128k cmd=zero + _verify_l2_bitmap 22 + _verify_l2_bitmap 23 + + # Unaligned request from clusters #24 to #26 + # In this case QEMU internally sends a 1k request followed by a + # subcluster-aligned 128k request. The first request decompresses + # cluster #24, but that's not enough to perform the second request + # efficiently because it partially writes to cluster #26 (which is + # compressed) so we hit the same problem as before. + alloc="$(seq 0 31)"; zero="" + _run_test c=24 sc=15 off=1 len=129k cmd=zero + _verify_l2_bitmap 25 + _verify_l2_bitmap 26 + + # Unaligned request from clusters #27 to #29 + # Similar to the previous case, but this time the tail of the + # request does not correspond to a compressed cluster, so it can + # be zeroed efficiently. + # Note that the very last subcluster is partially written, so if + # there's a backing file we need to perform cow. + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=27 sc=15 off=1 len=128k cmd=zero + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 28 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here + else + alloc=""; zero="$(seq 0 15)" + fi + _verify_l2_bitmap 29 + + echo + echo "### Writing zeroes 4: other tests (backing file: $use_backing_file) ###" + echo + # Unaligned request in the middle of cluster #30. + # If there's a backing file we need to allocate and do + # copy-on-write on the partially zeroed subclusters. + # If not we can set the 'all zeroes' bit on them. + if [ "$use_backing_file" = "yes" ]; then + alloc="15 19"; zero="$(seq 16 18)" # copy-on-write happening here + else + alloc=""; zero="$(seq 15 19)" + fi + _run_test c=30 sc=15 off=1 len=8k cmd=zero + + # Fill the last cluster with zeroes, up to the end of the image + # (the image size is not a multiple of the cluster or subcluster size). + alloc=""; zero="$(seq 0 17)" + _run_test c=32 sc=0 len=35k cmd=zero +done + +############################################################ +############################################################ +############################################################ + +# Zero + unmap +for use_backing_file in yes no; do + echo + echo "### Zero + unmap 1: allocated clusters (backing file: $use_backing_file) ###" + echo + # Note that the image size is not a multiple of the cluster size + _reset_img 2083k + alloc="$(seq 0 31)"; zero="" + _run_test c=9 sc=0 len=576k + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + _verify_l2_bitmap 12 + _verify_l2_bitmap 13 + _verify_l2_bitmap 14 + _verify_l2_bitmap 15 + _verify_l2_bitmap 16 + _verify_l2_bitmap 17 + + # Cluster-aligned request from clusters #9 to #11 + alloc=""; zero="$(seq 0 31)" + _run_test c=9 sc=0 len=192k cmd=unmap + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + + # Subcluster-aligned request from clusters #12 to #14 + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=12 sc=16 len=128k cmd=unmap + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 13 + alloc="$(seq 16 31)"; zero="$(seq 0 15)" + _verify_l2_bitmap 14 + + # Unaligned request from clusters #15 to #17 + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=15 sc=15 off=1 len=128k cmd=unmap + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 16 + alloc="$(seq 15 31)"; zero="$(seq 0 14)" + _verify_l2_bitmap 17 + + echo + echo "### Zero + unmap 2: compressed clusters (backing file: $use_backing_file) ###" + echo + alloc=""; zero="" + for c in $(seq 18 28); do + _run_test c=$c sc=0 len=64k cmd=compress + done + + # Cluster-aligned request from clusters #18 to #20 + alloc=""; zero="$(seq 0 31)" + _run_test c=18 sc=0 len=192k cmd=unmap + _verify_l2_bitmap 19 + _verify_l2_bitmap 20 + + # Subcluster-aligned request from clusters #21 to #23. + # We cannot partially zero a compressed cluster so the code + # returns -ENOTSUP, which means copy-on-write of the compressed + # data and fill the rest with actual zeroes on disk. + # TODO: cluster #22 should use the 'all zeroes' bits. + alloc="$(seq 0 31)"; zero="" + _run_test c=21 sc=16 len=128k cmd=unmap + _verify_l2_bitmap 22 + _verify_l2_bitmap 23 + + # Unaligned request from clusters #24 to #26 + # In this case QEMU internally sends a 1k request followed by a + # subcluster-aligned 128k request. The first request decompresses + # cluster #24, but that's not enough to perform the second request + # efficiently because it partially writes to cluster #26 (which is + # compressed) so we hit the same problem as before. + alloc="$(seq 0 31)"; zero="" + _run_test c=24 sc=15 off=1 len=129k cmd=unmap + _verify_l2_bitmap 25 + _verify_l2_bitmap 26 + + # Unaligned request from clusters #27 to #29 + # Similar to the previous case, but this time the tail of the + # request does not correspond to a compressed cluster, so it can + # be zeroed efficiently. + # Note that the very last subcluster is partially written, so if + # there's a backing file we need to perform cow. + alloc="$(seq 0 15)"; zero="$(seq 16 31)" + _run_test c=27 sc=15 off=1 len=128k cmd=unmap + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 28 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here + else + alloc=""; zero="$(seq 0 15)" + fi + _verify_l2_bitmap 29 +done + +############################################################ +############################################################ +############################################################ + +# Test qcow2_cluster_discard() with full and normal discards +for use_backing_file in yes no; do + echo + echo "### Discarding clusters with non-zero bitmaps (backing file: $use_backing_file) ###" + echo + if [ "$use_backing_file" = "yes" ]; then + _make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 1M + else + _make_test_img -o extended_l2=on 1M + fi + # Write clusters #0-#2 and then discard them + $QEMU_IO -c 'write -q 0 128k' "$TEST_IMG" + $QEMU_IO -c 'discard -q 0 128k' "$TEST_IMG" + # 'qemu-io discard' doesn't do a full discard, it zeroizes the + # cluster, so both clusters have all zero bits set now + alloc=""; zero="$(seq 0 31)" + _verify_l2_bitmap 0 + _verify_l2_bitmap 1 + # Now mark the 2nd half of the subclusters from cluster #0 as unallocated + poke_file "$TEST_IMG" $(($l2_offset+8)) "\x00\x00" + # Discard cluster #0 again to see how the zero bits have changed + $QEMU_IO -c 'discard -q 0 64k' "$TEST_IMG" + # And do a full discard of cluster #1 by shrinking and growing the image + $QEMU_IMG resize --shrink "$TEST_IMG" 64k + $QEMU_IMG resize "$TEST_IMG" 1M + # A normal discard sets all 'zero' bits only if the image has a + # backing file, otherwise it won't touch them. + if [ "$use_backing_file" = "yes" ]; then + alloc=""; zero="$(seq 0 31)" + else + alloc=""; zero="$(seq 0 15)" + fi + _verify_l2_bitmap 0 + # A full discard should clear the L2 entry completely. However + # when growing an image with a backing file the new clusters are + # zeroized to hide the stale data from the backing file + if [ "$use_backing_file" = "yes" ]; then + alloc=""; zero="$(seq 0 31)" + else + alloc=""; zero="" + fi + _verify_l2_bitmap 1 +done + +############################################################ +############################################################ +############################################################ + +# Test that corrupted L2 entries are detected in both read and write +# operations +for corruption_test_cmd in read write; do + echo + echo "### Corrupted L2 entries - $corruption_test_cmd test (allocated) ###" + echo + echo "# 'cluster is zero' bit set on the standard cluster descriptor" + echo + # We actually don't consider this a corrupted image. + # The 'cluster is zero' bit is unused in extended L2 entries so + # QEMU ignores it. + # TODO: maybe treat the image as corrupted and make qemu-img check fix it? + _make_test_img -o extended_l2=on 1M + $QEMU_IO -c 'write -q -P 0x11 0 2k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01" + alloc="0"; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -q -P 0x11 0 1k" "$TEST_IMG" + if [ "$corruption_test_cmd" = "write" ]; then + alloc="0"; zero="" + fi + _verify_l2_bitmap 0 + + echo + echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set" + echo + _make_test_img -o extended_l2=on 1M + # Write from the middle of cluster #0 to the middle of cluster #2 + $QEMU_IO -c 'write -q 32k 128k' "$TEST_IMG" + # Corrupt the L2 entry from cluster #1 + poke_file_be "$TEST_IMG" $(($l2_offset+24)) 4 1 + alloc="$(seq 0 31)"; zero="0" + _verify_l2_bitmap 1 + $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG" + + echo + echo "### Corrupted L2 entries - $corruption_test_cmd test (unallocated) ###" + echo + echo "# 'cluster is zero' bit set on the standard cluster descriptor" + echo + # We actually don't consider this a corrupted image. + # The 'cluster is zero' bit is unused in extended L2 entries so + # QEMU ignores it. + # TODO: maybe treat the image as corrupted and make qemu-img check fix it? + _make_test_img -o extended_l2=on 1M + # We want to modify the (empty) L2 entry from cluster #0, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01" + alloc=""; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -q 0 1k" "$TEST_IMG" + if [ "$corruption_test_cmd" = "write" ]; then + alloc="0"; zero="" + fi + _verify_l2_bitmap 0 + + echo + echo "# 'subcluster is allocated' bit set" + echo + _make_test_img -o extended_l2=on 1M + # We want to corrupt the (empty) L2 entry from cluster #0, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+15)) "\x01" + alloc="0"; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd 0 1k" "$TEST_IMG" + + echo + echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set" + echo + _make_test_img -o extended_l2=on 1M + # We want to corrupt the (empty) L2 entry from cluster #1, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + # Corrupt the L2 entry from cluster #1 + poke_file_be "$TEST_IMG" $(($l2_offset+24)) 8 $(((1 << 32) | 1)) + alloc="0"; zero="0" + _verify_l2_bitmap 1 + $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG" + + echo + echo "### Compressed cluster with subcluster bitmap != 0 - $corruption_test_cmd test ###" + echo + # We actually don't consider this a corrupted image. + # The bitmap in compressed clusters is unused so QEMU should just ignore it. + _make_test_img -o extended_l2=on 1M + $QEMU_IO -c 'write -q -P 11 -c 0 64k' "$TEST_IMG" + # Change the L2 bitmap to allocate subcluster #31 and zeroize subcluster #0 + poke_file "$TEST_IMG" $(($l2_offset+11)) "\x01\x80" + alloc="31"; zero="0" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -P 11 0 64k" "$TEST_IMG" | _filter_qemu_io + # Writing allocates a new uncompressed cluster so we get a new bitmap + if [ "$corruption_test_cmd" = "write" ]; then + alloc="$(seq 0 31)"; zero="" + fi + _verify_l2_bitmap 0 +done + +############################################################ +############################################################ +############################################################ + +echo +echo "### Detect and repair unaligned clusters ###" +echo +# Create a backing file and fill it with data +$QEMU_IMG create -f raw "$TEST_IMG.base" 128k | _filter_img_create +$QEMU_IO -c "write -q -P 0xff 0 128k" -f raw "$TEST_IMG.base" | _filter_qemu_io + +echo "# Corrupted L2 entry, allocated subcluster #" +# Create a new image, allocate a cluster and write some data to it +_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" +$QEMU_IO -c 'write -q -P 1 4k 2k' "$TEST_IMG" +# Corrupt the L2 entry by making the offset unaligned +poke_file "$TEST_IMG" "$(($l2_offset+6))" "\x02" +# This cannot be repaired, qemu-img check will fail to fix it +_check_test_img -r all +# Attempting to read the image will still show that it's corrupted +$QEMU_IO -c 'read -q 0 2k' "$TEST_IMG" + +echo "# Corrupted L2 entry, no allocated subclusters #" +# Create a new image, allocate a cluster and zeroize subcluster #2 +_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" +$QEMU_IO -c 'write -q -P 1 4k 2k' "$TEST_IMG" +$QEMU_IO -c 'write -q -z 4k 2k' "$TEST_IMG" +# Corrupt the L2 entry by making the offset unaligned +poke_file "$TEST_IMG" "$(($l2_offset+6))" "\x02" +# This time none of the subclusters are allocated so we can repair the image +_check_test_img -r all +# And the data can be read normally +$QEMU_IO -c 'read -q -P 0xff 0 4k' "$TEST_IMG" +$QEMU_IO -c 'read -q -P 0x00 4k 2k' "$TEST_IMG" +$QEMU_IO -c 'read -q -P 0xff 6k 122k' "$TEST_IMG" + +############################################################ +############################################################ +############################################################ + +echo +echo "### Image creation options ###" +echo +echo "# cluster_size < 16k" +_make_test_img -o extended_l2=on,cluster_size=8k 1M + +echo "# backing file and preallocation=metadata" +# For preallocation with backing files, create a backing file first +$QEMU_IMG create -f raw "$TEST_IMG.base" 1M | _filter_img_create +$QEMU_IO -c "write -q -P 0xff 0 1M" -f raw "$TEST_IMG.base" | _filter_qemu_io + +_make_test_img -o extended_l2=on,preallocation=metadata -F raw -b "$TEST_IMG.base" 512k +$QEMU_IMG resize "$TEST_IMG" 1M +$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map "$TEST_IMG" | _filter_testdir + +echo "# backing file and preallocation=falloc" +_make_test_img -o extended_l2=on,preallocation=falloc -F raw -b "$TEST_IMG.base" 512k +$QEMU_IMG resize "$TEST_IMG" 1M +$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map "$TEST_IMG" | _filter_testdir + +echo "# backing file and preallocation=full" +_make_test_img -o extended_l2=on,preallocation=full -F raw -b "$TEST_IMG.base" 512k +$QEMU_IMG resize "$TEST_IMG" 1M +$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map "$TEST_IMG" | _filter_testdir + +echo +echo "### Image resizing with preallocation and backing files ###" +echo +# In this case the new subclusters must have the 'all zeroes' bit set +echo "# resize --preallocation=metadata" +_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k +$QEMU_IMG resize --preallocation=metadata "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +# In this case and the next one the new subclusters must be allocated +echo "# resize --preallocation=falloc" +_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k +$QEMU_IMG resize --preallocation=falloc "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +echo "# resize --preallocation=full" +_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k +$QEMU_IMG resize --preallocation=full "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +echo +echo "### Image resizing with preallocation without backing files ###" +echo +# In this case the new subclusters must have the 'all zeroes' bit set +echo "# resize --preallocation=metadata" +_make_test_img -o extended_l2=on 503k +$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG resize --preallocation=metadata "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +# In this case and the next one the new subclusters must be allocated +echo "# resize --preallocation=falloc" +_make_test_img -o extended_l2=on 503k +$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG resize --preallocation=falloc "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +echo "# resize --preallocation=full" +_make_test_img -o extended_l2=on 503k +$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG resize --preallocation=full "$TEST_IMG" 1013k +$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io + +echo +echo "### qemu-img measure ###" +echo +echo "# 512MB, extended_l2=off" # This needs one L2 table +$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=off +echo "# 512MB, extended_l2=on" # This needs two L2 tables +$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=on + +echo "# 16K clusters, 64GB, extended_l2=off" # This needs one full L1 table cluster +$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=off +echo "# 16K clusters, 64GB, extended_l2=on" # This needs two full L2 table clusters +$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=on + +echo "# 8k clusters" # This should fail +$QEMU_IMG measure --size 1M -O qcow2 -o cluster_size=8k,extended_l2=on + +echo "# 1024 TB" # Maximum allowed size with extended_l2=on and 64K clusters +$QEMU_IMG measure --size 1024T -O qcow2 -o extended_l2=on +echo "# 1025 TB" # This should fail +$QEMU_IMG measure --size 1025T -O qcow2 -o extended_l2=on + +echo +echo "### qemu-img amend ###" +echo +_make_test_img -o extended_l2=on 1M +$QEMU_IMG amend -o extended_l2=off "$TEST_IMG" && echo "Unexpected pass" + +_make_test_img -o extended_l2=off 1M +$QEMU_IMG amend -o extended_l2=on "$TEST_IMG" && echo "Unexpected pass" + +echo +echo "### Test copy-on-write on an image with snapshots ###" +echo +_make_test_img -o extended_l2=on 1M + +# For each cluster from #0 to #9 this loop zeroes subcluster #7 +# and allocates subclusters #13 and #18. +alloc="13 18"; zero="7" +for c in $(seq 0 9); do + $QEMU_IO -c "write -q -z $((64*$c+14))k 2k" \ + -c "write -q -P $((0xd0+$c)) $((64*$c+26))k 2k" \ + -c "write -q -P $((0xe0+$c)) $((64*$c+36))k 2k" "$TEST_IMG" + _verify_l2_bitmap "$c" +done + +# Create a snapshot and set l2_offset to the new L2 table +$QEMU_IMG snapshot -c snap1 "$TEST_IMG" +l2_offset=$((0x110000)) + +# Write different patterns to each one of the clusters +# in order to see how copy-on-write behaves in each case. +$QEMU_IO -c "write -q -P 0xf0 $((64*0+30))k 1k" \ + -c "write -q -P 0xf1 $((64*1+20))k 1k" \ + -c "write -q -P 0xf2 $((64*2+40))k 1k" \ + -c "write -q -P 0xf3 $((64*3+26))k 1k" \ + -c "write -q -P 0xf4 $((64*4+14))k 1k" \ + -c "write -q -P 0xf5 $((64*5+1))k 1k" \ + -c "write -q -z $((64*6+30))k 3k" \ + -c "write -q -z $((64*7+26))k 2k" \ + -c "write -q -z $((64*8+26))k 1k" \ + -c "write -q -z $((64*9+12))k 1k" \ + "$TEST_IMG" +alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 0 +alloc="$(seq 10 18)"; zero="7" _verify_l2_bitmap 1 +alloc="$(seq 13 20)"; zero="7" _verify_l2_bitmap 2 +alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 3 +alloc="$(seq 7 18)"; zero="" _verify_l2_bitmap 4 +alloc="$(seq 0 18)"; zero="" _verify_l2_bitmap 5 +alloc="13 18"; zero="7 15 16" _verify_l2_bitmap 6 +alloc="18"; zero="7 13" _verify_l2_bitmap 7 +alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 8 +alloc="13 18"; zero="6 7" _verify_l2_bitmap 9 + +echo +echo "### Test concurrent requests ###" +echo + +_concurrent_io() +{ +# Allocate three subclusters in the same cluster. +# This works because handle_dependencies() checks whether the requests +# allocate the same cluster, even if the COW regions don't overlap (in +# this case they don't). +cat <