From patchwork Tue Jul 14 22:21:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02B16C433E0 for ; Tue, 14 Jul 2020 22:25:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA6482068F for ; Tue, 14 Jul 2020 22:25:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OUsSq+Id" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA6482068F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTMJ-0005Cv-U1 for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:25:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJ5-0000Bk-5p for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:47 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:27838 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJ3-0004VG-41 for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lFvJAvABHJhn9aUbqkhpwxA2JbVZYnPkXyaT9KSO/8A=; b=OUsSq+Id9uj1AfGr+Cz2wqjD3DhH/91gRDpLm4AxMndgGMVwmdZq085QZ0v9BJ31Eu1TNe kUHa1RVNSXeqbaT2cIgqjG4p97TMJgivcT5F/zJMw8QYpdPteAofpnCEOkOk21PcdHJsAh Zb2J190AyZ2vXDllW7sNsbrvuXla9lw= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-120-q_MIO3efN6mOND-FbL76Mw-1; Tue, 14 Jul 2020 18:21:41 -0400 X-MC-Unique: q_MIO3efN6mOND-FbL76Mw-1 Received: by mail-wm1-f70.google.com with SMTP id l5so90068wml.7 for ; Tue, 14 Jul 2020 15:21:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lFvJAvABHJhn9aUbqkhpwxA2JbVZYnPkXyaT9KSO/8A=; b=aEfrwfyfMaMQUPauGZ5h8ZxtS5xifMXwkSyRC/VZNLSxgARDVa073C1juLImmKTJDo fbFHey9i5q36/pQufQz2g+1O+JR4QbFGtgq9QsyQOc3u98q2QgLvJKZcUtmaEoT3ZVFg OaTmrnTEluw5Lg/P5nrljOh0Bu31ybV1JMLe7777U849EjN1b1FnfhXOZMtLdrcwScu4 pwiKxxYDelJwoBulNkB7cRaVzIVzMoz8gepbfEw5L3kcccAsm5oXGevv/ZujjdIgevcE a6vAght/cxIHjFfUw+Y2Rumo07NCBqzUL7QFmu5hQpkpI5xo1o5IMFpHj/hTfjS6a/J5 F7nA== X-Gm-Message-State: AOAM5312L/96iZtiiO6bRT1yk1R3V4aCD5Hd5ATABILNlEoUY9cEc/GT YOvdh1C1jfVC37i/Qlmr4aIJY137bcL/E0Ul2Z7MijgcY2SAAWNyM45Ot20XZxo9uXML/L6wo4V pfm12VRHH5BEBbTQ= X-Received: by 2002:a5d:43d2:: with SMTP id v18mr8036550wrr.196.1594765299326; Tue, 14 Jul 2020 15:21:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyf5GQIef7IlQoICvopUEpJloqBEI51Q+7z/cOPUBnJBo96Jsm9gt1x42ZLYOv1EnHqyav7bg== X-Received: by 2002:a5d:43d2:: with SMTP id v18mr8036527wrr.196.1594765298999; Tue, 14 Jul 2020 15:21:38 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id z6sm277928wmf.33.2020.07.14.15.21.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:21:38 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 01/19] scripts/performance: Add dissect.py script Date: Wed, 15 Jul 2020 00:21:14 +0200 Message-Id: <20200714222132.10815-2-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=philmd@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 01:42:04 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , John Snow , Ahmed Karaman , Aleksandar Markovic , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Ahmed Karaman Python script that dissects QEMU execution into three main phases: code generation, JIT execution and helpers execution. Syntax: dissect.py [-h] -- [] \ [] [-h] - Print the script arguments help message. Example of usage: dissect.py -- qemu-arm coulomb_double-arm Example output: Total Instructions: 4,702,865,362 Code Generation: 115,819,309 2.463% JIT Execution: 1,081,980,528 23.007% Helpers: 3,505,065,525 74.530% Signed-off-by: Ahmed Karaman Reviewed-by: Aleksandar Markovic Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200709052055.2650-2-ahmedkhaledkaraman@gmail.com> Signed-off-by: Philippe Mathieu-Daudé --- scripts/performance/dissect.py | 166 +++++++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100755 scripts/performance/dissect.py diff --git a/scripts/performance/dissect.py b/scripts/performance/dissect.py new file mode 100755 index 0000000000..bf24f50922 --- /dev/null +++ b/scripts/performance/dissect.py @@ -0,0 +1,166 @@ +#!/usr/bin/env python3 + +# Print the percentage of instructions spent in each phase of QEMU +# execution. +# +# Syntax: +# dissect.py [-h] -- [] \ +# [] +# +# [-h] - Print the script arguments help message. +# +# Example of usage: +# dissect.py -- qemu-arm coulomb_double-arm +# +# This file is a part of the project "TCG Continuous Benchmarking". +# +# Copyright (C) 2020 Ahmed Karaman +# Copyright (C) 2020 Aleksandar Markovic +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import argparse +import os +import subprocess +import sys +import tempfile + + +def get_JIT_line(callgrind_data): + """ + Search for the first instance of the JIT call in + the callgrind_annotate output when ran using --tree=caller + This is equivalent to the self number of instructions of JIT. + + Parameters: + callgrind_data (list): callgrind_annotate output + + Returns: + (int): Line number + """ + line = -1 + for i in range(len(callgrind_data)): + if callgrind_data[i].strip('\n') and \ + callgrind_data[i].split()[-1] == "[???]": + line = i + break + if line == -1: + sys.exit("Couldn't locate the JIT call ... Exiting.") + return line + + +def main(): + # Parse the command line arguments + parser = argparse.ArgumentParser( + usage='dissect.py [-h] -- ' + ' [] ' + ' []') + + parser.add_argument('command', type=str, nargs='+', help=argparse.SUPPRESS) + + args = parser.parse_args() + + # Extract the needed variables from the args + command = args.command + + # Insure that valgrind is installed + check_valgrind = subprocess.run( + ["which", "valgrind"], stdout=subprocess.DEVNULL) + if check_valgrind.returncode: + sys.exit("Please install valgrind before running the script.") + + # Save all intermediate files in a temporary directory + with tempfile.TemporaryDirectory() as tmpdirname: + # callgrind output file path + data_path = os.path.join(tmpdirname, "callgrind.data") + # callgrind_annotate output file path + annotate_out_path = os.path.join(tmpdirname, "callgrind_annotate.out") + + # Run callgrind + callgrind = subprocess.run((["valgrind", + "--tool=callgrind", + "--callgrind-out-file=" + data_path] + + command), + stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE) + if callgrind.returncode: + sys.exit(callgrind.stderr.decode("utf-8")) + + # Save callgrind_annotate output + with open(annotate_out_path, "w") as output: + callgrind_annotate = subprocess.run( + ["callgrind_annotate", data_path, "--tree=caller"], + stdout=output, + stderr=subprocess.PIPE) + if callgrind_annotate.returncode: + sys.exit(callgrind_annotate.stderr.decode("utf-8")) + + # Read the callgrind_annotate output to callgrind_data[] + callgrind_data = [] + with open(annotate_out_path, 'r') as data: + callgrind_data = data.readlines() + + # Line number with the total number of instructions + total_instructions_line_number = 20 + # Get the total number of instructions + total_instructions_line_data = \ + callgrind_data[total_instructions_line_number] + total_instructions = total_instructions_line_data.split()[0] + total_instructions = int(total_instructions.replace(',', '')) + + # Line number with the JIT self number of instructions + JIT_self_instructions_line_number = get_JIT_line(callgrind_data) + # Get the JIT self number of instructions + JIT_self_instructions_line_data = \ + callgrind_data[JIT_self_instructions_line_number] + JIT_self_instructions = JIT_self_instructions_line_data.split()[0] + JIT_self_instructions = int(JIT_self_instructions.replace(',', '')) + + # Line number with the JIT self + inclusive number of instructions + # It's the line above the first JIT call when running with --tree=caller + JIT_total_instructions_line_number = JIT_self_instructions_line_number-1 + # Get the JIT self + inclusive number of instructions + JIT_total_instructions_line_data = \ + callgrind_data[JIT_total_instructions_line_number] + JIT_total_instructions = JIT_total_instructions_line_data.split()[0] + JIT_total_instructions = int(JIT_total_instructions.replace(',', '')) + + # Calculate number of instructions in helpers and code generation + helpers_instructions = JIT_total_instructions-JIT_self_instructions + code_generation_instructions = total_instructions-JIT_total_instructions + + # Print results (Insert commas in large numbers) + # Print total number of instructions + print('{:<20}{:>20}\n'. + format("Total Instructions:", + format(total_instructions, ','))) + # Print code generation instructions and percentage + print('{:<20}{:>20}\t{:>6.3f}%'. + format("Code Generation:", + format(code_generation_instructions, ","), + (code_generation_instructions / total_instructions) * 100)) + # Print JIT instructions and percentage + print('{:<20}{:>20}\t{:>6.3f}%'. + format("JIT Execution:", + format(JIT_self_instructions, ","), + (JIT_self_instructions / total_instructions) * 100)) + # Print helpers instructions and percentage + print('{:<20}{:>20}\t{:>6.3f}%'. + format("Helpers:", + format(helpers_instructions, ","), + (helpers_instructions/total_instructions)*100)) + + +if __name__ == "__main__": + main() From patchwork Tue Jul 14 22:21:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18A10C433E0 for ; Tue, 14 Jul 2020 22:27:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D6003206D9 for ; Tue, 14 Jul 2020 22:26:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YXKTSTLd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D6003206D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42568 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTO6-0000Jk-W8 for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:26:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49394) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJA-0000EQ-Ka for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:56 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:55051 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJ7-0004VZ-5a for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765307; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8FhXd9jo3ZykzUIegSI6TSb8wRrSAAOgjzX+n7PAM3E=; b=YXKTSTLdNTip89ad/saQckhVTe7tUqq6Xt0YNSYR3QeUkLH+KTsOxQwb5I0Kyo71fFQypn okxGdwsBgcK1PJMUI/vevpaPzmyXIc4eEvT47I1E95pN8YZ+QcYznOeg/PrO+Yo9WnS51y KAkDpX8wyVIqaqG2kJrUoDB0AVNvaqE= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-213-RryxhGZ0PHyIl952Tn3-gQ-1; Tue, 14 Jul 2020 18:21:45 -0400 X-MC-Unique: RryxhGZ0PHyIl952Tn3-gQ-1 Received: by mail-wr1-f71.google.com with SMTP id v3so19812wrq.10 for ; Tue, 14 Jul 2020 15:21:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8FhXd9jo3ZykzUIegSI6TSb8wRrSAAOgjzX+n7PAM3E=; b=IPHoVHYi+E6SWhYkYpDQampYrawK+APcck3/Frg+DVvTW2Z7EOw425oyWbKgvppW0o 3PguIkXMhMIN7Y1ePY1SBu5V76z9NgZlfMpTeQX8aTCshS/R1wsquE9ApEJl/ehrmIZB u4k80aUQeM69RL81tl3eyswcEKfBS048dZmiC1EftljnafdrwRDBDQc5PsGiGEDyxOIU 42T7SCijjINCeEEmnmy5P2FUquAEU2MP5WjOLejqBCwG4tKWF9eN6Ze+tKJ8X0K1r7A+ ntLOXL8XKvO7S+vwL90UvCGGGc9Ne+N+xEcwxnXtOLsm12sX857m4ziJpDTVz4FFrrHM ryuw== X-Gm-Message-State: AOAM5315lHBst/6txZFzgqxUfUEL7pW/QRhnzFQOlrljjX2G561BHxCg qbv1K9drVjRpLDJMlY3l9hDeGXyyX4YSoktKxWPeR0z7ntM+cqV+ZTTy2eb2eJzHt4oJH+TWdPD PqhFWQ0acdcjHwD0= X-Received: by 2002:a5d:628b:: with SMTP id k11mr8159969wru.107.1594765304163; Tue, 14 Jul 2020 15:21:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyzzUFgb07Cm3+x5Qosv7eEcZqNJKwtfLwh3QJmI2BohpotIatfLAm2GmCtHEnR79ejMueHeQ== X-Received: by 2002:a5d:628b:: with SMTP id k11mr8159951wru.107.1594765303957; Tue, 14 Jul 2020 15:21:43 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id j145sm351404wmj.7.2020.07.14.15.21.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:21:43 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 02/19] python/machine.py: consolidate _post_shutdown() Date: Wed, 15 Jul 2020 00:21:15 +0200 Message-Id: <20200714222132.10815-3-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=philmd@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 12:02:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ahmed Karaman , =?utf-8?q?Philippe_Mathi?= =?utf-8?b?ZXUtRGF1ZMOp?= , John Snow , Eduardo Habkost , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow Move more cleanup actions into _post_shutdown. As a change, if QEMU should so happen to be terminated during a call to wait(), that event will now be logged. This is not likely to occur during normative use. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200710050649.32434-2-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index c25f0b42cf..ca1f2114e6 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -294,6 +294,8 @@ def _post_launch(self): self._qmp.accept() def _post_shutdown(self): + self._load_io_log() + if self._qemu_log_file is not None: self._qemu_log_file.close() self._qemu_log_file = None @@ -307,6 +309,17 @@ def _post_shutdown(self): while len(self._remove_files) > 0: self._remove_if_exists(self._remove_files.pop()) + exitcode = self.exitcode() + if exitcode is not None and exitcode < 0: + msg = 'qemu received signal %i; command: "%s"' + if self._qemu_full_args: + command = ' '.join(self._qemu_full_args) + else: + command = '' + LOG.warning(msg, -int(exitcode), command) + + self._launched = False + def launch(self): """ Launch the VM and make sure we cleanup and expose the @@ -355,7 +368,6 @@ def wait(self): self._popen.wait() if self._qmp: self._qmp.close() - self._load_io_log() self._post_shutdown() def shutdown(self, has_quit=False, hard=False): @@ -382,21 +394,8 @@ def shutdown(self, has_quit=False, hard=False): self._popen.kill() self._popen.wait() - self._load_io_log() self._post_shutdown() - exitcode = self.exitcode() - if exitcode is not None and exitcode < 0 and \ - not (exitcode == -9 and hard): - msg = 'qemu received signal %i: %s' - if self._qemu_full_args: - command = ' '.join(self._qemu_full_args) - else: - command = '' - LOG.warning(msg, -int(exitcode), command) - - self._launched = False - def kill(self): self.shutdown(hard=True) From patchwork Tue Jul 14 22:21:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B476DC433DF for ; Tue, 14 Jul 2020 22:28:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7BBB82072D for ; Tue, 14 Jul 2020 22:28:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="I4oPX68c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7BBB82072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTPt-0003kP-Pp for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:28:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJE-0000F1-J4 for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:56 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:47098 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJA-0004W1-NJ for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:21:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765311; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f4PIu0HMQQfwQmcqNhOsRKcF3CqOiR6vgF4LSRdEoL8=; b=I4oPX68cWplyaskyWsF8q1BOyKyOatqEFQuarlHi1KuBQ+rVS9T+2qOMXwBl2+WlZX2DbW FWd6gkxC/a9W5bbTsuoxSAwzkviRsEUSk2UFGZYQl9R2GlyzdhZFrIhAoUUMMvVnA4biVS z+aeYFWLe4LKcFV13uvYl+/U1/frJ3k= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-226-qxrP7g1zO42P-jKnvr-Rvw-1; Tue, 14 Jul 2020 18:21:50 -0400 X-MC-Unique: qxrP7g1zO42P-jKnvr-Rvw-1 Received: by mail-wr1-f72.google.com with SMTP id o25so17927wro.16 for ; Tue, 14 Jul 2020 15:21:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f4PIu0HMQQfwQmcqNhOsRKcF3CqOiR6vgF4LSRdEoL8=; b=jASlVRxaHYrxxZHForxYuacahVLR2hQa2RcZdTkJpKrImfb8IhZgNAqa3WetnDNaSo sfTWCcyHGuh4F1Ox1OOYkBlL9K4ksF+S5SVjxjVWtUhqrS8fAwRSU9x5vR0LqQZd5ify g/mcTeXobd/U8yMJzwTD3FXzZGnYJhnHa9ii+bX/UMYDKiu0HJ5itkB3NZz/MPFC/2Q4 QqcXDIbtZ3hm4XGJSBSwRqfzLTZk9btr468OcModEbaLKg9FmDHPm22s3ntb37Vauw3A NdXkEsgYzuZKzf0kzzgnLVZB5oC2ChDV0COJAiOAJWNB8nrzPmrG7zjLAQ0kD3P80eCC bcew== X-Gm-Message-State: AOAM531UAY87lNIP+8ovZasRU97PJdNgGuUnuXfK9bMU1DgPJyWG64mJ rjx7tUBplYJjlXwUjP8xRcO3Mk+KqWo6vWu+aNrwQeLJ+sZ/rzHdbNM43ZUgaNYsoEUw6Ow+9oi VSnbRpuItjDnXbQM= X-Received: by 2002:a1c:3bc1:: with SMTP id i184mr5729269wma.119.1594765308880; Tue, 14 Jul 2020 15:21:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyKDnGybAjeG5dJqWW0wSiVBJBdnKyUwvcrZOYtDNIFnAYbHL6Th6v+//SXR1HN3sCe/buyCQ== X-Received: by 2002:a1c:3bc1:: with SMTP id i184mr5729252wma.119.1594765308665; Tue, 14 Jul 2020 15:21:48 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id j6sm181798wro.25.2020.07.14.15.21.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:21:48 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 03/19] python/machine.py: Close QMP socket in cleanup Date: Wed, 15 Jul 2020 00:21:16 +0200 Message-Id: <20200714222132.10815-4-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=philmd@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 03:57:32 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ahmed Karaman , =?utf-8?q?Philippe_Mathi?= =?utf-8?b?ZXUtRGF1ZMOp?= , John Snow , Eduardo Habkost , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow It's not important to do this before waiting for the process to exit, so it can be done during generic post-shutdown cleanup. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-Id: <20200710050649.32434-3-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index ca1f2114e6..d3faa9a84c 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -294,6 +294,10 @@ def _post_launch(self): self._qmp.accept() def _post_shutdown(self): + if self._qmp: + self._qmp.close() + self._qmp = None + self._load_io_log() if self._qemu_log_file is not None: @@ -366,8 +370,6 @@ def wait(self): Wait for the VM to power off """ self._popen.wait() - if self._qmp: - self._qmp.close() self._post_shutdown() def shutdown(self, has_quit=False, hard=False): @@ -388,7 +390,6 @@ def shutdown(self, has_quit=False, hard=False): try: if not has_quit: self._qmp.cmd('quit') - self._qmp.close() self._popen.wait(timeout=3) except: self._popen.kill() From patchwork Tue Jul 14 22:21:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADDCAC433E0 for ; Tue, 14 Jul 2020 22:23:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 77F4F2068F for ; Tue, 14 Jul 2020 22:23:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R/A3ea93" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 77F4F2068F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTKr-00025s-Ix for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:23:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49550) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJQ-0000QC-VQ for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:08 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:36500 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJL-0004YW-Uo for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765323; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5KOJiRNNgCKXxXF3PAojUBMhMfHmXxvQ94Tix5b7+Js=; b=R/A3ea93+FwISTxZVqFOq65GnTZ4Kl5jo0Y0wAmvVLTcXkoRjcKRRFo1d6aVWvokrb5UYU gE8BnCiax6+gwh4NdMCDp6LN2H4TpZbnUnOH1vgvAy9VHBRbWTxg3eCRppZNRvKGb2ZyDh cQTGIran13IqKF3JRz31NCh1wqduwWw= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-318-REd_f3inPm6Lk3IcPssU6A-1; Tue, 14 Jul 2020 18:21:59 -0400 X-MC-Unique: REd_f3inPm6Lk3IcPssU6A-1 Received: by mail-wm1-f70.google.com with SMTP id c124so105702wme.0 for ; Tue, 14 Jul 2020 15:21:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5KOJiRNNgCKXxXF3PAojUBMhMfHmXxvQ94Tix5b7+Js=; b=UCZvluu8BM/ha9Vh1JwEbsBHjez1AeuloBWnQ4eJvMECWpcSz6LqVie+eiliG4lrYQ VspNmtnDiKK4XmrAOTXFxYwqUWh3LB/5GptAV7ZpCWqDpNvpffg0pJ7ikNJl4AHPHnEB IAhYDxNl9NMlYDrRgR7sgrcxDXvOVJRS8zRcPIi/AQyfZ+l+lxaDGbHBZRgcupoIrsRA uVIXpO5NoT5WeVYfllw01txYHmnZuU4RM3/Yjwny5mrS/PpEy5cgagH7elnc8BVq5Qkl Y0mLKBrUzNhbXgiSA+9S+wUqwYhGDgYZCGEgothorE0xF9ggU6IHQCg9YEN7VUWEnhRX yZ5Q== X-Gm-Message-State: AOAM5301O3xzJJHTEENZbF/aqQ+f3/v7LkyiTNdMpnB8MkGjMAHBNVhP CKj2OFIJ4PF9P+mi0SbHotdizbb3Y5ZErn0rHzF22g2vaPGmXh+07JJKmhGyVFCOF2ARquiJiGW Q6SsoZh1N3KJJe4w= X-Received: by 2002:a5d:554d:: with SMTP id g13mr7730002wrw.98.1594765318138; Tue, 14 Jul 2020 15:21:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxIl8VT+T+TJSXcjMJX3LIWDdwurgfpY6HH5UdPHk5kiuwk0HdcZ1OVeT6zU3JEJIr4Aa5Vnw== X-Received: by 2002:a5d:554d:: with SMTP id g13mr7729983wrw.98.1594765317930; Tue, 14 Jul 2020 15:21:57 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id f12sm177045wrw.53.2020.07.14.15.21.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:21:57 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 05/19] python/machine.py: Perform early cleanup for wait() calls, too Date: Wed, 15 Jul 2020 00:21:18 +0200 Message-Id: <20200714222132.10815-6-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=philmd@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 01:42:04 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ahmed Karaman , =?utf-8?q?Philippe_Mathi?= =?utf-8?b?ZXUtRGF1ZMOp?= , John Snow , Eduardo Habkost , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow This is primarily for consistency, and is a step towards wait() and shutdown() sharing the same implementation so that the two cleanup paths cannot diverge. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-Id: <20200710050649.32434-5-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 1 + 1 file changed, 1 insertion(+) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 127926b276..63e40879e2 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -380,6 +380,7 @@ def wait(self): """ Wait for the VM to power off """ + self._early_cleanup() self._popen.wait() self._post_shutdown() From patchwork Tue Jul 14 22:21:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90C58C433E3 for ; Tue, 14 Jul 2020 22:27:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 573772072D for ; Tue, 14 Jul 2020 22:27:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cJ5sk1ls" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 573772072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43372 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTOF-0000ed-H6 for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:27:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50160) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJs-0001Ae-MO for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:36 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:48214 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJh-0004j9-Ro for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765344; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yI/gwTOC/ILtlhkZf0FnguyCxPWQjiV56qaIviljiXg=; b=cJ5sk1ls3gkj15Zy30hgCk6IQc2XfL1howXf40k0hXnE/hNUZkGXnMOOuQdM6RejPHiqlU Z7brF453PuoUdOj3SSQC2hjfxmGO0ZqrojnwacJeDRyQovQYqpKjd/c6l1I3hprqbx4gKK hM8SjbxW0GAtYju9HAEFZS5+mdjyL2I= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-5miYbdudO9CRUfyIytE0fA-1; Tue, 14 Jul 2020 18:22:23 -0400 X-MC-Unique: 5miYbdudO9CRUfyIytE0fA-1 Received: by mail-wr1-f72.google.com with SMTP id i12so20685wrx.11 for ; Tue, 14 Jul 2020 15:22:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yI/gwTOC/ILtlhkZf0FnguyCxPWQjiV56qaIviljiXg=; b=L70WkW7lD7rR8Y8Q+WtWfuc9nDVIbry/XWwXD36lK1iv2z88tRz16qyjH7vduWswoL R0+5p4ZJtYnnAJll8GHaXJNZ9MpmmAjbl4SI7idb4NCcGSgCj2iP1gQZGLOxAW2lRLhu 07ff9kP/Ez9OKQJVY8PgCDH5sgW9hoGoSE1KlbfZ1wtihWM7Gt+Wn3x+Vqm4t/bgxYPK BxDxBCGB/eLjP8l4h6L/lws23o7JbDJDQ3Vm4pgg96GlWLRqyWt/41KqFw4KsFPQWnHm iE03gR5PkaKceW2QJM+cqFJYrAY/0woOuVmbSsNB8b12+mfQuvDwlTW805dU1U5ZKDh/ cc9g== X-Gm-Message-State: AOAM5322zeoiH3eF8kYxEnh75c15Z45zuNVgzVB21MiLPCT+nT97itlp 8swucyYwHvw4mie4G1WFftTnm6RfsxZTA3sAgqV5ZpnyDSssYyZB/o4kxFNSmZnXHvKsr49urNd xc+JxaFbWaVCeCDo= X-Received: by 2002:a1c:e209:: with SMTP id z9mr5861731wmg.153.1594765341877; Tue, 14 Jul 2020 15:22:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzjmKxmbIJW0/cPl2BPl6yPDtNyf1lHyHgquft37qp4ZS8FqBv/eUDsxQxvcesIaxdOKwQPrA== X-Received: by 2002:a1c:e209:: with SMTP id z9mr5861718wmg.153.1594765341691; Tue, 14 Jul 2020 15:22:21 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id k18sm186277wrx.34.2020.07.14.15.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:22:21 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 10/19] tests/acceptance: Don't test reboot on cubieboard Date: Wed, 15 Jul 2020 00:21:23 +0200 Message-Id: <20200714222132.10815-11-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=philmd@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 03:57:32 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , John Snow , Wainer dos Santos Moschetta , Ahmed Karaman , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow cubieboard does not have a functioning reboot, it halts and QEMU does not exit. vm.shutdown() is modified in a forthcoming patch that makes it less tolerant of race conditions on shutdown; tests should consciously decide to WAIT or to SHUTDOWN qemu. So long as this test is attempting to reboot, the correct choice would be to WAIT for the VM to exit. However, since that's broken, we should SHUTDOWN instead. SHUTDOWN is indeed what already happens when the test performs teardown, however, if anyone fixes cubieboard reboot in the future, this test will develop a new race condition that might be hard to debug. Therefore: remove the reboot test and make it obvious that the VM is still running when the test concludes, where the test teardown will do the right thing. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-Id: <20200710050649.32434-10-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- tests/acceptance/boot_linux_console.py | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py index 5867ef760c..8b8b828bc5 100644 --- a/tests/acceptance/boot_linux_console.py +++ b/tests/acceptance/boot_linux_console.py @@ -508,9 +508,7 @@ def test_arm_cubieboard_initrd(self): 'Allwinner sun4i/sun5i') exec_command_and_wait_for_pattern(self, 'cat /proc/iomem', 'system-control@1c00000') - exec_command_and_wait_for_pattern(self, 'reboot', - 'reboot: Restarting system') - # NB: Do not issue vm.wait() here, cubieboard's reboot does not exit! + # cubieboard's reboot is not functioning; omit reboot test. def test_arm_cubieboard_sata(self): """ @@ -553,9 +551,7 @@ def test_arm_cubieboard_sata(self): 'Allwinner sun4i/sun5i') exec_command_and_wait_for_pattern(self, 'cat /proc/partitions', 'sda') - exec_command_and_wait_for_pattern(self, 'reboot', - 'reboot: Restarting system') - # NB: Do not issue vm.wait() here, cubieboard's reboot does not exit! + # cubieboard's reboot is not functioning; omit reboot test. def test_arm_orangepi(self): """ From patchwork Tue Jul 14 22:21:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECEF4C433E3 for ; Tue, 14 Jul 2020 22:25:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B565B20658 for ; Tue, 14 Jul 2020 22:25:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NzH+JV/O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B565B20658 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTMi-0006Ah-VO for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:25:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50128) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTJr-00018N-IM for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:49781 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTJm-0004kS-Vb for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fxpPD25l94H6jH9waRJgbDvpErEb7UHCQOGbx0fD5YA=; b=NzH+JV/OeyOmDwfCMRFP6in31PtVI7IkeWspg1rMY5Htsujd/xDhixjIvWAoBLLFKRBldM 5LJrQi/TE3KG7ve6ad9R9FZomwk/Qbg5GTIZwyj0GpArs/wmYMlRnAmeTBcgeXY8lp1wv+ h4Q+FsrKHvEYhrwb5hQaqmPjWy1OtkM= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-279-KkgddlGiPQC-_tDG0pLbDw-1; Tue, 14 Jul 2020 18:22:28 -0400 X-MC-Unique: KkgddlGiPQC-_tDG0pLbDw-1 Received: by mail-wr1-f69.google.com with SMTP id o25so18488wro.16 for ; Tue, 14 Jul 2020 15:22:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fxpPD25l94H6jH9waRJgbDvpErEb7UHCQOGbx0fD5YA=; b=f1nu1Cz/NMljK7PDhGXAUOQclcthHm4aLlUezAW6U8RncBWBbJNxhiKTH9IusqMJFT M/+KVPhbhQeP+TszYEALExbU/VEddkpRFX5znkmyo+ImuJQY7PAnA9Gq4XQrezWuwJgD qhd9WNmsedkf2A5+KGMo9bVZCBEhJqhCmwFWON6XTGyDjAi1SAGOsSB/BWD2wWkCWSa5 Vdz9Z5xmcxo4Y+xKTMpJjlyNNHbwRmpZKENyXHbSoXymqFKlrCyoFRyvqrCvqGAK/wq0 6DUjk5bIQQovoVyDuzOuANWhWhfiSPCnFqQIAkyXqT7nvDI8cKcU7oUgxd2VF1E1wqWU g+Ng== X-Gm-Message-State: AOAM530sd8ooIsb1NM5iZtHUNFE/84bLs4byfs/A5bsHJP6L3JSUGo84 S8ezS+i+q30oF/dwtJ1vcALESRMgyVlontl+wBws3pK/TBX2zdwZ5dcvq5GAWQms2iZecaoaU50 m8rjOjvZNwsFj8kY= X-Received: by 2002:a5d:40cb:: with SMTP id b11mr8264546wrq.263.1594765346765; Tue, 14 Jul 2020 15:22:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzwanRA/LIdrlHXnRYIrv61pWu7j68fnofevncVhh1VvW3D1TOPE3ae+3SWppY5GjxrIbmNkQ== X-Received: by 2002:a5d:40cb:: with SMTP id b11mr8264521wrq.263.1594765346477; Tue, 14 Jul 2020 15:22:26 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id u10sm289156wml.29.2020.07.14.15.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:22:25 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 11/19] python/machine.py: split shutdown into hard and soft flavors Date: Wed, 15 Jul 2020 00:21:24 +0200 Message-Id: <20200714222132.10815-12-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=philmd@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 18:21:58 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ahmed Karaman , =?utf-8?q?Philippe_Mathi?= =?utf-8?b?ZXUtRGF1ZMOp?= , John Snow , Eduardo Habkost , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow This is done primarily to avoid the 'bare except' pattern, which suppresses all exceptions during shutdown and can obscure errors. Replace this with a pattern that isolates the different kind of shutdown paradigms (_hard_shutdown and _soft_shutdown), and a new fallback shutdown handler (_do_shutdown) that gracefully attempts one before the other. This split now also ensures that no matter what happens, _post_shutdown() is always invoked. shutdown() changes in behavior such that if it attempts to do a graceful shutdown and is unable to, it will now always raise an exception to indicate this. This can be avoided by the test writer in three ways: 1. If the VM is expected to have already exited or is in the process of exiting, wait() can be used instead of shutdown() to clean up resources instead. This helps avoid race conditions in shutdown. 2. If a test writer is expecting graceful shutdown to fail, shutdown should be called in a try...except block. 3. If the test writer has no interest in performing a graceful shutdown at all, kill() can be used instead. Handling shutdown in this way makes it much more explicit which type of shutdown we want and allows the library to report problems with this process. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-Id: <20200710050649.32434-11-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 98 +++++++++++++++++++++++++++++++++++------- 1 file changed, 83 insertions(+), 15 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 3f0b873f58..a955e3f221 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -49,6 +49,12 @@ class QEMUMachineAddDeviceError(QEMUMachineError): """ +class AbnormalShutdown(QEMUMachineError): + """ + Exception raised when a graceful shutdown was requested, but not performed. + """ + + class MonitorResponseError(qmp.QMPError): """ Represents erroneous QMP monitor reply @@ -376,6 +382,7 @@ def _early_cleanup(self) -> None: """ Perform any cleanup that needs to happen before the VM exits. + May be invoked by both soft and hard shutdown in failover scenarios. Called additionally by _post_shutdown for comprehensive cleanup. """ # If we keep the console socket open, we may deadlock waiting @@ -385,32 +392,93 @@ def _early_cleanup(self) -> None: self._console_socket.close() self._console_socket = None + def _hard_shutdown(self) -> None: + """ + Perform early cleanup, kill the VM, and wait for it to terminate. + + :raise subprocess.Timeout: When timeout is exceeds 60 seconds + waiting for the QEMU process to terminate. + """ + self._early_cleanup() + self._popen.kill() + self._popen.wait(timeout=60) + + def _soft_shutdown(self, has_quit: bool = False, + timeout: Optional[int] = 3) -> None: + """ + Perform early cleanup, attempt to gracefully shut down the VM, and wait + for it to terminate. + + :param has_quit: When True, don't attempt to issue 'quit' QMP command + :param timeout: Optional timeout in seconds for graceful shutdown. + Default 3 seconds, A value of None is an infinite wait. + + :raise ConnectionReset: On QMP communication errors + :raise subprocess.TimeoutExpired: When timeout is exceeded waiting for + the QEMU process to terminate. + """ + self._early_cleanup() + + if self._qmp is not None: + if not has_quit: + # Might raise ConnectionReset + self._qmp.cmd('quit') + + # May raise subprocess.TimeoutExpired + self._popen.wait(timeout=timeout) + + def _do_shutdown(self, has_quit: bool = False, + timeout: Optional[int] = 3) -> None: + """ + Attempt to shutdown the VM gracefully; fallback to a hard shutdown. + + :param has_quit: When True, don't attempt to issue 'quit' QMP command + :param timeout: Optional timeout in seconds for graceful shutdown. + Default 3 seconds, A value of None is an infinite wait. + + :raise AbnormalShutdown: When the VM could not be shut down gracefully. + The inner exception will likely be ConnectionReset or + subprocess.TimeoutExpired. In rare cases, non-graceful termination + may result in its own exceptions, likely subprocess.TimeoutExpired. + """ + try: + self._soft_shutdown(has_quit, timeout) + except Exception as exc: + self._hard_shutdown() + raise AbnormalShutdown("Could not perform graceful shutdown") \ + from exc + def shutdown(self, has_quit: bool = False, hard: bool = False, timeout: Optional[int] = 3) -> None: """ - Terminate the VM and clean up + Terminate the VM (gracefully if possible) and perform cleanup. + Cleanup will always be performed. + + If the VM has not yet been launched, or shutdown(), wait(), or kill() + have already been called, this method does nothing. + + :param has_quit: When true, do not attempt to issue 'quit' QMP command. + :param hard: When true, do not attempt graceful shutdown, and + suppress the SIGKILL warning log message. + :param timeout: Optional timeout in seconds for graceful shutdown. + Default 3 seconds, A value of None is an infinite wait. """ if not self._launched: return - self._early_cleanup() - - if self.is_running(): + try: if hard: - self._popen.kill() - elif self._qmp: - try: - if not has_quit: - self._qmp.cmd('quit') - self._popen.wait(timeout=timeout) - except: - self._popen.kill() - self._popen.wait(timeout=timeout) - - self._post_shutdown() + self._hard_shutdown() + else: + self._do_shutdown(has_quit, timeout=timeout) + finally: + self._post_shutdown() def kill(self): + """ + Terminate the VM forcefully, wait for it to exit, and perform cleanup. + """ self.shutdown(hard=True) def wait(self, timeout: Optional[int] = None) -> None: From patchwork Tue Jul 14 22:21:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F2DFC433E2 for ; Tue, 14 Jul 2020 22:32:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 48DDF2072D for ; Tue, 14 Jul 2020 22:32:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QxDOnFwM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 48DDF2072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTTp-0000ii-Ik for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:32:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTKD-0001oR-E6 for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:57 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:49304 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTK7-0004w7-N5 for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rq8+AkQ2f65ZI8kNrmi3lYHC8ESMu5ga2yLy+oSnBcw=; b=QxDOnFwMUuzrxwRLSGizeAxDPFnDgbpMM95M6Tj2uR/viYQG25+KlQv62E9nUPmpi8JOP6 s20w44LSrxjky0cKYrHUYPPYSAGGLU+yhuc/7S3DXoUpeVlMZQtfRD/Uvwa+g/4ZtA9Nex ENOiD/h9IU7e0R1ydX3eHe+8h8JBP0Q= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-175-LVXPlCMdNomaVdBRg7awvg-1; Tue, 14 Jul 2020 18:22:47 -0400 X-MC-Unique: LVXPlCMdNomaVdBRg7awvg-1 Received: by mail-wm1-f72.google.com with SMTP id b13so88718wme.9 for ; Tue, 14 Jul 2020 15:22:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rq8+AkQ2f65ZI8kNrmi3lYHC8ESMu5ga2yLy+oSnBcw=; b=JEEcp0gkLO5+RxgDZpU2hJ229gGULvHJX0HE7QlHL0Xf/Pn7U8dpJq7UyCO6mwpj8R LwnGaw6PHvngkOlb5pKqAiZLSLRqcpj1fWYE0GAOYPuAm+wxOrzNPR1lC2svEL9RZaNy fyxxeJviU/M+QzloNW6jcs+A7trLJQxgkAD8qzrpSoJvyCRBdnjqM6LbHwZac1y3zchz 4G05zdcu9rgzMgZ6zrZ8XLYCnIWSHNYveIO4jmeQOCX+mhp7tfMznSsOeV4x/nVGA3yS ypL7gYiszZNAO4TlGUPL09SwxPNckMVfr4diQ53KJu5LtI8YALV1iFKN54qApCbV5wqd +bTg== X-Gm-Message-State: AOAM5332j2qComIeLxAA/YKioxvI2SgvtLGRgQn/vwpTtBnlC/pV/FI4 taPxhXyYTnH8y3xjO0wNYotL8X5DG6ylemYxuGYpmDYviJGmmQ8qVpMzNYT7XDAYJo3dLZKFiol X+yQ/Xjw4c7oIcY4= X-Received: by 2002:a1c:dcd5:: with SMTP id t204mr5630259wmg.17.1594765366150; Tue, 14 Jul 2020 15:22:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZ0W1+mYV+xIoQKOnPVpPvIcjDUeg6KBenJbmH5A4pOKNAkP29TAvjcxBiE/kLmrFQ3GPijQ== X-Received: by 2002:a1c:dcd5:: with SMTP id t204mr5630246wmg.17.1594765365977; Tue, 14 Jul 2020 15:22:45 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id 68sm285232wmz.40.2020.07.14.15.22.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:22:45 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 15/19] iotests.py: use qemu.qmp type aliases Date: Wed, 15 Jul 2020 00:21:28 +0200 Message-Id: <20200714222132.10815-16-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=philmd@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 18:21:58 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , "open list:Block layer core" , John Snow , Max Reitz , Ahmed Karaman , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow iotests.py should use the type definitions from qmp.py instead of its own. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-Id: <20200710052220.3306-3-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- tests/qemu-iotests/iotests.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 8b760405ee..3590ed78a0 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -35,13 +35,10 @@ # pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest +from qemu.qmp import QMPMessage assert sys.version_info >= (3, 6) -# Type Aliases -QMPResponse = Dict[str, Any] - - # Use this logger for logging messages directly from the iotests module logger = logging.getLogger('qemu.iotests') logger.addHandler(logging.NullHandler()) @@ -561,7 +558,7 @@ def add_incoming(self, addr): self._args.append(addr) return self - def hmp(self, command_line: str, use_log: bool = False) -> QMPResponse: + def hmp(self, command_line: str, use_log: bool = False) -> QMPMessage: cmd = 'human-monitor-command' kwargs = {'command-line': command_line} if use_log: @@ -582,7 +579,7 @@ def resume_drive(self, drive: str) -> None: self.hmp(f'qemu-io {drive} "remove_break bp_{drive}"') def hmp_qemu_io(self, drive: str, cmd: str, - use_log: bool = False) -> QMPResponse: + use_log: bool = False) -> QMPMessage: """Write to a given drive using an HMP command""" return self.hmp(f'qemu-io {drive} "{cmd}"', use_log=use_log) From patchwork Tue Jul 14 22:21:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E36E5C433DF for ; Tue, 14 Jul 2020 22:30:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABC472072D for ; Tue, 14 Jul 2020 22:30:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EiElOsVK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABC472072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTRj-0006cx-Vo for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:30:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50552) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTKD-0001pr-Us for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:57 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:33186 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTKC-0004wP-2Z for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:22:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=re7gKjIDIxjZ168y2pnh2vU3k4lXjSx5/GbYDUTFUY0=; b=EiElOsVK0YFoZXcpx7rZiFAfqMnKbuirxMOnHe10CFf4OlYadJlaeOqLUmPGpoLiC81FrG WAY/Z1UTcmHZH6m0IIxfFMuzXulJ3l3qIP4VsMkS0d9UB3ojHV+9hHvwqJU0WNRP4Azh+z ETWFpufeLKcAJOJ8gfbBgE4r+n7ed00= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-345-lJDu1lPVMyyinQWGs2yLDw-1; Tue, 14 Jul 2020 18:22:52 -0400 X-MC-Unique: lJDu1lPVMyyinQWGs2yLDw-1 Received: by mail-wr1-f71.google.com with SMTP id b8so18420wro.19 for ; Tue, 14 Jul 2020 15:22:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=re7gKjIDIxjZ168y2pnh2vU3k4lXjSx5/GbYDUTFUY0=; b=idLcoaqJoohO3r4AxODE7CP8+exwyt030AfDunqcr7Lh7w/5JhDbj4+khFV4OOeVL3 P6fuOuurExlvbJ/YxwGz/1xR7T1vCoYN4eDJARAj+2ajV+hO7h83JRswSa5zT41f+dUZ S6KcxXVqS8j/AhRJ4ufVTWQGcMsIKPR9HaU5XQ7lTryAY8wMZrex/EUOTNZEPomSTvvI 4+QxT+0Gh5nW0C6W/04K8XaPsC86MEc5By+Q1vpeOv0PzYJXoGneQroUF+0yzS0lLgvV L61NBRJ77jhVAf5dmjntdmfpGNGvn/iFFUOmxaQnLygTZ4ak0AH0Hv2FcCAnwZ5Lxfii Hcmg== X-Gm-Message-State: AOAM532MU7uE5hR+ekawdFkophDJldBqyC1wrknoaztt1Q7d3Y/cJse3 y7ud8rZljPjIyc01/9qcO0ymJyy/q0ruAEZTvRoclNgdTcwlsopsonZmewt2kXKzx0jbGQGBsDO zCDGBj8063AV+6UI= X-Received: by 2002:adf:f350:: with SMTP id e16mr7854963wrp.43.1594765370841; Tue, 14 Jul 2020 15:22:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzXiNIgqK8oqPlJZrnbHoy0394nTkrzba7SHXIDy15Y4LpispKvRSNUA1ToUonH0lG38oBdSA== X-Received: by 2002:adf:f350:: with SMTP id e16mr7854944wrp.43.1594765370596; Tue, 14 Jul 2020 15:22:50 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id y189sm386094wmd.27.2020.07.14.15.22.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:22:50 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 16/19] python/qmp.py: re-absorb MonitorResponseError Date: Wed, 15 Jul 2020 00:21:29 +0200 Message-Id: <20200714222132.10815-17-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=philmd@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 12:02:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , John Snow , Ahmed Karaman , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow When I initially split this out, I considered this more of a machine error than a QMP protocol error, but I think that's misguided. Move this back to qmp.py and name it QMPResponseError. Convert qmp.command() to use this exception type. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf Message-Id: <20200710052220.3306-4-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 15 +-------------- python/qemu/qmp.py | 17 +++++++++++++++-- scripts/render_block_graph.py | 7 +++++-- 3 files changed, 21 insertions(+), 18 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 69055189bd..80c4d4a8b6 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -56,19 +56,6 @@ class AbnormalShutdown(QEMUMachineError): """ -class MonitorResponseError(qmp.QMPError): - """ - Represents erroneous QMP monitor reply - """ - def __init__(self, reply): - try: - desc = reply["error"]["desc"] - except KeyError: - desc = reply - super().__init__(desc) - self.reply = reply - - class QEMUMachine: """ A QEMU VM @@ -533,7 +520,7 @@ def command(self, cmd, conv_keys=True, **args): if reply is None: raise qmp.QMPError("Monitor is closed") if "error" in reply: - raise MonitorResponseError(reply) + raise qmp.QMPResponseError(reply) return reply["return"] def get_qmp_event(self, wait=False): diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 8388c7b603..aa8a666b8a 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -61,6 +61,19 @@ class QMPTimeoutError(QMPError): """ +class QMPResponseError(QMPError): + """ + Represents erroneous QMP monitor reply + """ + def __init__(self, reply: QMPMessage): + try: + desc = reply['error']['desc'] + except KeyError: + desc = reply + super().__init__(desc) + self.reply = reply + + class QEMUMonitorProtocol: """ Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then @@ -251,8 +264,8 @@ def command(self, cmd, **kwds): Build and send a QMP command to the monitor, report errors if any """ ret = self.cmd(cmd, kwds) - if "error" in ret: - raise Exception(ret['error']['desc']) + if 'error' in ret: + raise QMPResponseError(ret) return ret['return'] def pull_event(self, wait=False): diff --git a/scripts/render_block_graph.py b/scripts/render_block_graph.py index 409b4321f2..da6acf050d 100755 --- a/scripts/render_block_graph.py +++ b/scripts/render_block_graph.py @@ -25,7 +25,10 @@ from graphviz import Digraph sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'python')) -from qemu.machine import MonitorResponseError +from qemu.qmp import ( + QEMUMonitorProtocol, + QMPResponseError, +) def perm(arr): @@ -102,7 +105,7 @@ def command(self, cmd): reply = json.loads(subprocess.check_output(ar)) if 'error' in reply: - raise MonitorResponseError(reply) + raise QMPResponseError(reply) return reply['return'] From patchwork Tue Jul 14 22:21:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45413C433E3 for ; Tue, 14 Jul 2020 22:29:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0CE5C2072D for ; Tue, 14 Jul 2020 22:29:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DuTNakhX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CE5C2072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTQC-0004Pq-A4 for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:29:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTKJ-00023P-9n for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:23:03 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:55618 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTKH-0004wy-Eb for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:23:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=34BOciJNToD087b3DmEPBjP+atCQ4f8lF8uOK0B/VQo=; b=DuTNakhX71xD8GeB9sLqIaANETRwQUsTiY18bZKFdSbqhCnh5EiamS1VYSAmzg6GcpA53r tKE04hvDDM1xAXENrJ1TNN6uQgAXjD9BznRdviLd9wflB5acGxyPI8B1OJM5N/a3V4Gssh 9LBhH2UW3rIE6gMi7OI6Qs7A21DsJWQ= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-79--Crii8NfNNOcjMzSqN41iQ-1; Tue, 14 Jul 2020 18:22:57 -0400 X-MC-Unique: -Crii8NfNNOcjMzSqN41iQ-1 Received: by mail-wm1-f71.google.com with SMTP id w25so3502521wmc.0 for ; Tue, 14 Jul 2020 15:22:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=34BOciJNToD087b3DmEPBjP+atCQ4f8lF8uOK0B/VQo=; b=GiJamUMrS+XyxFcomsCHKM2z1o1Wb/j68Wws1HE5VGzcAqXk2n1OeGuNuRxMGfqEWx 76AQkRlbcfSnl+t6o+Y+qdsU6V72gFi6up+YM3afKh46iEIqGvu2n0wlu4vKEIi/sP+8 i0B6RU6G1yKBoVxrp49RU7M0ALG4oZ81eyrHfOF64BQ52EtRButtdneLAmSkJ7fZG5iK hfq44PFqj9hlfKc3fPJS+vqHN5zGMGLYzBpgyTClFqnRBbRt0nXr7cRCkJrlSkr7aqVX YX+sprsp5AomVban5MJMR3syuaTRG9dwBAA1XMTog2g0Tv5NqevZZ1BoIpdEu64kL2EV 1tOA== X-Gm-Message-State: AOAM532ejtBAioPntoL4tMobxCvgupUflit4Vz/xNkuzWVRiHOX7dBL6 EylC03wkefh/BkUOwzbR38eHJD32JK8HRl5B6SQ+zW7VAFlJklgNEwBOY+yDPlBYcl/xcVQVHjl /imLL8FEeYS+CJVg= X-Received: by 2002:adf:e80d:: with SMTP id o13mr7963494wrm.112.1594765375651; Tue, 14 Jul 2020 15:22:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwwNHB5UaUZCpE+Fa7BIw0lDObJzVZX+kL0W1LO6KAsXyqyguMPGZdBEvEHX7rhy4WGiyA3dg== X-Received: by 2002:adf:e80d:: with SMTP id o13mr7963479wrm.112.1594765375473; Tue, 14 Jul 2020 15:22:55 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id a2sm160275wrn.68.2020.07.14.15.22.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:22:54 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 17/19] python/qmp.py: Do not return None from cmd_obj Date: Wed, 15 Jul 2020 00:21:30 +0200 Message-Id: <20200714222132.10815-18-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=philmd@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=philmd@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 12:02:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , John Snow , Ahmed Karaman , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow This makes typing the qmp library difficult, as it necessitates wrapping Optional[] around the type for every return type up the stack. At some point, it becomes difficult to discern or remember why it's None instead of the expected object. Use the python exception system to tell us exactly why we didn't get an object. Remove this special-cased return. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-Id: <20200710052220.3306-5-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/qmp.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index aa8a666b8a..ef3c919b76 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -225,22 +225,18 @@ def accept(self, timeout=15.0): self.__sockfile = self.__sock.makefile(mode='r') return self.__negotiate_capabilities() - def cmd_obj(self, qmp_cmd): + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: """ Send a QMP command to the QMP Monitor. @param qmp_cmd: QMP command to be sent as a Python dict - @return QMP response as a Python dict or None if the connection has - been closed + @return QMP response as a Python dict """ self.logger.debug(">>> %s", qmp_cmd) - try: - self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) - except OSError as err: - if err.errno == errno.EPIPE: - return None - raise err + self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) resp = self.__json_read() + if resp is None: + raise QMPConnectError("Unexpected empty reply from server") self.logger.debug("<<< %s", resp) return resp From patchwork Tue Jul 14 22:21:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= X-Patchwork-Id: 277857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53F4BC433E1 for ; Tue, 14 Jul 2020 22:32:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC8C72072D for ; Tue, 14 Jul 2020 22:32:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bppqArzs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC8C72072D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jvTTj-0000dF-5M for qemu-devel@archiver.kernel.org; Tue, 14 Jul 2020 18:32:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jvTKV-0002OW-AG for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:23:15 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:50702 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jvTKQ-0004xx-Cm for qemu-devel@nongnu.org; Tue, 14 Jul 2020 18:23:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594765389; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Akupi2/psErTNl648GD773WL1cQbHEfDPZG4YPkFXLs=; b=bppqArzsDqcBwpInWR6vq2doUZNlZagkRfUaIi45Qko7XQn0566YvKoL++iFhtM7mUKkhp QRWRZexkZeyvT2NaXex9XUSjo3yToK66xuB59ClXopQvHFEqR24lGIRFZVFXrbSP5bxdNG eAQ88EER4ZbBEOEKmUmYo73Z0tMs8zs= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-462-yyqF41_9PEWTeYW7Tzf-kQ-1; Tue, 14 Jul 2020 18:23:08 -0400 X-MC-Unique: yyqF41_9PEWTeYW7Tzf-kQ-1 Received: by mail-wm1-f71.google.com with SMTP id s134so94655wme.6 for ; Tue, 14 Jul 2020 15:23:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Akupi2/psErTNl648GD773WL1cQbHEfDPZG4YPkFXLs=; b=F9F9OuyOThagS1Ga6X/yOwNDqX6XOOQi9J0gSORy1Xys4wDdMNE9INd5eWl992AZUL GrPfEFYvHltZG24T7bRRiILEJNhRbmMYl9gBkIJ1fjb2yKpLOSUjRZSferNvnOuh3R4D GtGxM7UmTmOSgFBMfAC6iCY5NWSw+fHkSCvLYNj4bWsmzwm58re6Xew4xFg452gNHkVR 7+qUr7T1MXa36pRRVxmGVW7wUQRIl0B94b3gLtDWaneASr+EeL9FrpiGIdujD2VGvgbb ad8wZ6xfT0KfyUt89N17KJ65Trj2Uy99glg5a4NHiiP+HZS02HrsPJFnJq7dOqOG3lwO wYSA== X-Gm-Message-State: AOAM531n1v94I6kCvcYlm05Cmc+sHC+Jq3516ng+ZKqSJtPubO/Jtqrp by5XbTvAfYymiWmrXgbQsBuFut9SMHcT/cZlUeD7CTg6DLr+8OswYu3dOIl+dB7BcplXsLMgn8m p0TFF88r0xgVTo7U= X-Received: by 2002:a05:600c:313:: with SMTP id q19mr5112983wmd.9.1594765385249; Tue, 14 Jul 2020 15:23:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzl1qd4y1/guXy5rG6B2MQmlkEIjFWiigjiGR1tZNete2GxbwUixFCefgmCOIC3Bxi6hpAqSA== X-Received: by 2002:a05:600c:313:: with SMTP id q19mr5112971wmd.9.1594765385077; Tue, 14 Jul 2020 15:23:05 -0700 (PDT) Received: from localhost.localdomain (138.red-83-57-170.dynamicip.rima-tde.net. [83.57.170.138]) by smtp.gmail.com with ESMTPSA id w17sm200963wra.42.2020.07.14.15.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jul 2020 15:23:04 -0700 (PDT) From: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= To: qemu-devel@nongnu.org Subject: [PULL 19/19] python/qmp.py: add QMPProtocolError Date: Wed, 15 Jul 2020 00:21:32 +0200 Message-Id: <20200714222132.10815-20-philmd@redhat.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200714222132.10815-1-philmd@redhat.com> References: <20200714222132.10815-1-philmd@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=philmd@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/14 01:42:04 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , John Snow , Ahmed Karaman , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow In the case that we receive a reply but are unable to understand it, use this exception name to indicate that case. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf Message-Id: <20200710052220.3306-7-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé --- python/qemu/qmp.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 1ae36050a4..7935dababb 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -62,6 +62,12 @@ class QMPTimeoutError(QMPError): """ +class QMPProtocolError(QMPError): + """ + QMP protocol error; unexpected response + """ + + class QMPResponseError(QMPError): """ Represents erroneous QMP monitor reply @@ -266,6 +272,10 @@ def command(self, cmd, **kwds): ret = self.cmd(cmd, kwds) if 'error' in ret: raise QMPResponseError(ret) + if 'return' not in ret: + raise QMPProtocolError( + "'return' key not found in QMP response '{}'".format(str(ret)) + ) return cast(QMPReturnValue, ret['return']) def pull_event(self, wait=False):