From patchwork Mon Jul 13 07:27:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 277994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9209EC433DF for ; Mon, 13 Jul 2020 07:33:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6811F206F0 for ; Mon, 13 Jul 2020 07:33:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6811F206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jusxh-0007gF-Ld for qemu-devel@archiver.kernel.org; Mon, 13 Jul 2020 03:33:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussx-0008Nt-MG; Mon, 13 Jul 2020 03:28:23 -0400 Received: from relay.sw.ru ([185.231.240.75]:35072 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussw-0007TK-5T; Mon, 13 Jul 2020 03:28:23 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jussP-0001Vx-Ja; Mon, 13 Jul 2020 10:27:49 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v9 01/10] qcow2: Fix capitalization of header extension constant. Date: Mon, 13 Jul 2020 10:27:47 +0300 Message-Id: <1594625276-134500-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 03:27:59 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Make the capitalization of the hexadecimal numbers consistent for the QCOW2 header extension constants in docs/interop/qcow2.txt. Suggested-by: Eric Blake Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 2 +- docs/interop/qcow2.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 0cd2e67..80dfe5f 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -66,7 +66,7 @@ typedef struct { } QEMU_PACKED QCowExtension; #define QCOW2_EXT_MAGIC_END 0 -#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA +#define QCOW2_EXT_MAGIC_BACKING_FORMAT 0xe2792aca #define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857 #define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77 #define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb72346..f072e27 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -231,7 +231,7 @@ be stored. Each extension has a structure like the following: Byte 0 - 3: Header extension type: 0x00000000 - End of the header extension area - 0xE2792ACA - Backing file format name string + 0xe2792aca - Backing file format name string 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer From patchwork Mon Jul 13 07:27:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 277997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74E8DC433E7 for ; Mon, 13 Jul 2020 07:29:31 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4990C206F0 for ; Mon, 13 Jul 2020 07:29:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4990C206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jusu2-0001m6-HU for qemu-devel@archiver.kernel.org; Mon, 13 Jul 2020 03:29:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59506) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussf-0007fl-Tp; Mon, 13 Jul 2020 03:28:05 -0400 Received: from relay.sw.ru ([185.231.240.75]:35078 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussc-0007TF-EG; Mon, 13 Jul 2020 03:28:05 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jussP-0001Vx-M2; Mon, 13 Jul 2020 10:27:49 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v9 03/10] qcow2_format.py: change Qcow2BitmapExt initialization method Date: Mon, 13 Jul 2020 10:27:49 +0300 Message-Id: <1594625276-134500-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 03:27:59 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There are two ways to initialize a class derived from Qcow2Struct: 1. Pass a block of binary data to the constructor. 2. Pass the file descriptor to allow reading the file from constructor. Let's change the Qcow2BitmapExt initialization method from 1 to 2 to support a scattered reading in the initialization chain. The implementation comes with the patch that follows. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_format.py index 2f3681b..9eb6b5b 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -62,8 +62,8 @@ class Qcow2StructMeta(type): class Qcow2Struct(metaclass=Qcow2StructMeta): - - """Qcow2Struct: base class for qcow2 data structures + """ + Qcow2Struct: base class for qcow2 data structures Successors should define fields class variable, which is: list of tuples, each of three elements: @@ -173,7 +173,13 @@ class QcowHeaderExtension(Qcow2Struct): self.data_str = data_str if self.magic == QCOW2_EXT_MAGIC_BITMAPS: - self.obj = Qcow2BitmapExt(data=self.data) + assert fd is not None + position = fd.tell() + # Step back to reread data + padded = (self.length + 7) & ~7 + fd.seek(-padded, 1) + self.obj = Qcow2BitmapExt(fd=fd) + fd.seek(position) else: self.obj = None From patchwork Mon Jul 13 07:27:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 277995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3147C433DF for ; Mon, 13 Jul 2020 07:32:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84AC1206F0 for ; Mon, 13 Jul 2020 07:32:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84AC1206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37952 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jusx8-0006fK-N0 for qemu-devel@archiver.kernel.org; Mon, 13 Jul 2020 03:32:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussh-0007i3-6X; Mon, 13 Jul 2020 03:28:07 -0400 Received: from relay.sw.ru ([185.231.240.75]:35082 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussc-0007TL-Hm; Mon, 13 Jul 2020 03:28:06 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jussP-0001Vx-Pj; Mon, 13 Jul 2020 10:27:49 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v9 06/10] qcow2_format.py: pass cluster size to substructures Date: Mon, 13 Jul 2020 10:27:52 +0300 Message-Id: <1594625276-134500-7-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 03:27:59 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The cluster size of an image is the QcowHeader class member and may be obtained by dependent extension structures such as Qcow2BitmapExt for further bitmap table details print. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_format.py index 1438792..d7198a9 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -129,14 +129,16 @@ class Qcow2BitmapExt(Qcow2Struct): ('u64', '{:#x}', 'bitmap_directory_offset') ) - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=fd) + self.cluster_size = cluster_size self.read_bitmap_directory(fd) def read_bitmap_directory(self, fd): fd.seek(self.bitmap_directory_offset) self.bitmap_directory = \ - [Qcow2BitmapDirEntry(fd) for _ in range(self.nb_bitmaps)] + [Qcow2BitmapDirEntry(fd, cluster_size=self.cluster_size) + for _ in range(self.nb_bitmaps)] def dump(self): super().dump() @@ -157,8 +159,9 @@ class Qcow2BitmapDirEntry(Qcow2Struct): ('u32', '{}', 'extra_data_size') ) - def __init__(self, fd): + def __init__(self, fd, cluster_size): super().__init__(fd=fd) + self.cluster_size = cluster_size # Seek relative to the current position in the file fd.seek(self.extra_data_size, 1) bitmap_name = fd.read(self.name_size) @@ -198,11 +201,13 @@ class QcowHeaderExtension(Qcow2Struct): # then padding to next multiply of 8 ) - def __init__(self, magic=None, length=None, data=None, fd=None): + def __init__(self, magic=None, length=None, data=None, fd=None, + cluster_size=None): """ Support both loading from fd and creation from user data. For fd-based creation current position in a file will be used to read the data. + The cluster_size value may be obtained by dependent structures. This should be somehow refactored and functionality should be moved to superclass (to allow creation of any qcow2 struct), but then, fields @@ -241,7 +246,7 @@ class QcowHeaderExtension(Qcow2Struct): # Step back to reread data padded = (self.length + 7) & ~7 fd.seek(-padded, 1) - self.obj = Qcow2BitmapExt(fd=fd) + self.obj = Qcow2BitmapExt(fd=fd, cluster_size=cluster_size) fd.seek(position) else: self.obj = None @@ -317,7 +322,7 @@ class QcowHeader(Qcow2Struct): end = self.cluster_size while fd.tell() < end: - ext = QcowHeaderExtension(fd=fd) + ext = QcowHeaderExtension(fd=fd, cluster_size=self.cluster_size) if ext.magic == 0: break else: From patchwork Mon Jul 13 07:27:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 277998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE623C433E3 for ; Mon, 13 Jul 2020 07:29:12 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C84E120702 for ; Mon, 13 Jul 2020 07:29:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C84E120702 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52102 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1justk-0000xX-1M for qemu-devel@archiver.kernel.org; Mon, 13 Jul 2020 03:29:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59486) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussf-0007fY-4j; Mon, 13 Jul 2020 03:28:05 -0400 Received: from relay.sw.ru ([185.231.240.75]:35088 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussc-0007TM-B9; Mon, 13 Jul 2020 03:28:04 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jussP-0001Vx-S9; Mon, 13 Jul 2020 10:27:49 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v9 08/10] qcow2.py: Introduce '-j' key to dump in JSON format Date: Mon, 13 Jul 2020 10:27:54 +0300 Message-Id: <1594625276-134500-9-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 03:27:59 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add the command key to the qcow2.py arguments list to dump QCOW2 metadata in JSON format. Here is the suggested way to do that. The implementation of the dump in JSON format is in the patch that follows. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2.py | 19 +++++++++++++++---- tests/qemu-iotests/qcow2_format.py | 16 ++++++++-------- 2 files changed, 23 insertions(+), 12 deletions(-) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index 0910e6a..7402279 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -26,16 +26,19 @@ from qcow2_format import ( ) +dump_json = False + + def cmd_dump_header(fd): h = QcowHeader(fd) - h.dump() + h.dump(dump_json) print() - h.dump_extensions() + h.dump_extensions(dump_json) def cmd_dump_header_exts(fd): h = QcowHeader(fd) - h.dump_extensions() + h.dump_extensions(dump_json) def cmd_set_header(fd, name, value): @@ -134,6 +137,11 @@ cmds = [ def main(filename, cmd, args): + global dump_json + dump_json = '-j' in sys.argv + if dump_json: + sys.argv.remove('-j') + args.remove('-j') fd = open(filename, "r+b") try: for name, handler, num_args, desc in cmds: @@ -151,11 +159,14 @@ def main(filename, cmd, args): def usage(): - print("Usage: %s [, ...]" % sys.argv[0]) + print("Usage: %s [, ...] [, ...]" % sys.argv[0]) print("") print("Supported commands:") for name, handler, num_args, desc in cmds: print(" %-20s - %s" % (name, desc)) + print("") + print("Supported keys:") + print(" %-20s - %s" % ('-j', 'Dump in JSON format')) if __name__ == '__main__': diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_format.py index 22be3ee..a9bd5a8 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -109,7 +109,7 @@ class Qcow2Struct(metaclass=Qcow2StructMeta): self.__dict__ = dict((field[2], values[i]) for i, field in enumerate(self.fields)) - def dump(self): + def dump(self, dump_json=None): for f in self.fields: value = self.__dict__[f[2]] if isinstance(f[1], str): @@ -140,8 +140,8 @@ class Qcow2BitmapExt(Qcow2Struct): [Qcow2BitmapDirEntry(fd, cluster_size=self.cluster_size) for _ in range(self.nb_bitmaps)] - def dump(self): - super().dump() + def dump(self, dump_json=None): + super().dump(dump_json) for entry in self.bitmap_directory: print() entry.dump() @@ -185,7 +185,7 @@ class Qcow2BitmapDirEntry(Qcow2Struct): self.bitmap_table = Qcow2BitmapTable(raw_table=table, cluster_size=self.cluster_size) - def dump(self): + def dump(self, dump_json=None): print(f'{"Bitmap name":<25} {self.name}') super(Qcow2BitmapDirEntry, self).dump() self.bitmap_table.dump() @@ -293,13 +293,13 @@ class QcowHeaderExtension(Qcow2Struct): else: self.obj = None - def dump(self): + def dump(self, dump_json=None): super().dump() if self.obj is None: print(f'{"data":<25} {self.data_str}') else: - self.obj.dump() + self.obj.dump(dump_json) @classmethod def create(cls, magic, data): @@ -398,8 +398,8 @@ class QcowHeader(Qcow2Struct): buf = buf[0:header_bytes-1] fd.write(buf) - def dump_extensions(self): + def dump_extensions(self, dump_json=None): for ex in self.extensions: print('Header extension:') - ex.dump() + ex.dump(dump_json) print() From patchwork Mon Jul 13 07:27:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 277996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6480FC433E3 for ; Mon, 13 Jul 2020 07:30:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3EE66206F0 for ; Mon, 13 Jul 2020 07:30:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3EE66206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59886 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jusvA-000432-Dr for qemu-devel@archiver.kernel.org; Mon, 13 Jul 2020 03:30:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussg-0007g8-Ah; Mon, 13 Jul 2020 03:28:06 -0400 Received: from relay.sw.ru ([185.231.240.75]:35090 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jussc-0007TN-Fs; Mon, 13 Jul 2020 03:28:06 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jussP-0001Vx-TO; Mon, 13 Jul 2020 10:27:49 +0300 From: Andrey Shinkevich To: qemu-block@nongnu.org Subject: [PATCH v9 09/10] qcow2_format.py: collect fields to dump in JSON format Date: Mon, 13 Jul 2020 10:27:55 +0300 Message-Id: <1594625276-134500-10-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1594625276-134500-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/13 03:27:59 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" As __dict__ is being extended with class members we do not want to print, make a light copy of the initial __dict__ and extend the copy by adding lists we have to print in the JSON output. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/qcow2_format.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/qcow2_format.py b/tests/qemu-iotests/qcow2_format.py index a9bd5a8..4ce6d95 100644 --- a/tests/qemu-iotests/qcow2_format.py +++ b/tests/qemu-iotests/qcow2_format.py @@ -109,6 +109,8 @@ class Qcow2Struct(metaclass=Qcow2StructMeta): self.__dict__ = dict((field[2], values[i]) for i, field in enumerate(self.fields)) + self.fields_dict = self.__dict__.copy() + def dump(self, dump_json=None): for f in self.fields: value = self.__dict__[f[2]] @@ -139,6 +141,7 @@ class Qcow2BitmapExt(Qcow2Struct): self.bitmap_directory = \ [Qcow2BitmapDirEntry(fd, cluster_size=self.cluster_size) for _ in range(self.nb_bitmaps)] + self.fields_dict.update(bitmap_directory=self.bitmap_directory) def dump(self, dump_json=None): super().dump(dump_json) @@ -184,6 +187,7 @@ class Qcow2BitmapDirEntry(Qcow2Struct): table = [e[0] for e in struct.iter_unpack('>Q', fd.read(table_size))] self.bitmap_table = Qcow2BitmapTable(raw_table=table, cluster_size=self.cluster_size) + self.fields_dict.update(bitmap_table=self.bitmap_table) def dump(self, dump_json=None): print(f'{"Bitmap name":<25} {self.name}')