From patchwork Wed Jun 24 14:04:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191651 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp624040ile; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw0p9kAs4obgpzwIIFDmIXkJWKT7RTW5J42Adq6OCEoJJ7wsx1jicnJDJtiFZ9a1KDe7LM6 X-Received: by 2002:aed:2942:: with SMTP id s60mr12052793qtd.139.1593007513148; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007513; cv=none; d=google.com; s=arc-20160816; b=xlsnGsZtZLkLOYg65AO+GnXr5/Xi/DtscFZbNNtkqUi/AdhwXtbiCbMypCNJmDxdIS YKnZeJYsFL8aeBJVmEw86rMHMKxCh52mYegry9cSD/V6JtN4/x4EcmiRAfpiPG+IMKmz fOTEEMQbjuDUbIvWTuB6dejkZDgNYt/cGc971rRqv8ZHa97w8V52EMz2KAm880vOgF9h 6vr7wXQnWzrgJWOJJ/UGmG/ombkpH5XLnnr1S9g+vrHGZw/OmUVFZqXXpLrGNEkbQ6Zs C3X1lK8J99bNWGOyu8Tl4Sy2/lsNwoBzSOwAqxrfqn7n8pWVjdqp3nquIk3SJMUFKRK8 pPSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=XtfJG88MQcoIXPmt5qB/ndVi/mYD2J+KCz2y42wgntTIUtXh6pDsBllbkiQZ1erq4F tnX3bM6OYkQyrB7QNa/UFzCtiYSgfjbVMuTmO6bP8pPtqe0ddWsiBxY7ktPwJpV4z0m+ R2eF0YFI+rvdf/TKgI6sIYVRVSwIrK7eGnMoDVBixLxXouigl5Dm04kEeRmx2uPRGtH9 t0vD/EdS64CyTNFCLFfDK1xiFV1JiCb0+1Z7rm0hW+6x1x/RUcEwRrW1wT+KbJsH76ol Vb3n69d3FHuoe0NbYp9TSpp5Ai8oxaAYjQsHSsfl30xhSIdP2NpXGMShrvn4dlY1GYey OVCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VwcLfqPb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 188si9083779qkg.28.2020.06.24.07.05.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:05:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VwcLfqPb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo61Y-0004nl-Ga for patch@linaro.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61F-0004m3-OP for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:53 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:37772) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61C-0003Tx-Hc for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:53 -0400 Received: by mail-wr1-x434.google.com with SMTP id a6so2429179wrm.4 for ; Wed, 24 Jun 2020 07:04:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=VwcLfqPbp+T4CAYLNWhwImrkNQUg/MCxrqpgGmWz5g/2C21hymKonLfsM6GwlacWol Q6vbsoWH5gYC+W1TWoRXxM3uM5kJjYeNn79or5nGcho35BNWgsTfmeoE2X7P8F6nR8Ty ts6EncUOKpud+pHpKq7ukXC+zeNjk5fZ2EL/9THYsilF+51TF6krgr7lS7pDTn7HE/mX aiAbFnWBcofwmlVz2vLYZ/+LCpGqy53SQkADa7q07rTQN4j+jn/qE2zIQ/e7kW2kyaXv wfbgf2HqRITx9sYY+aXART2mToePlOOx3Een48LcQh0lS2H9RxTA9VNyzER3pqCMCI62 Egng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KLxxBVgESg8+q++r/+fvnKLfCmcpFTynVpvVjZzQTYc=; b=R0Q3Z76fzD+xW2/j9IwReh7NgDokcF3HjiNHCuA3KjtOCDKy1oKc7NkchPqj/MfU7P m9hxc81tvggHyDwME4XL+a00irVP+NrINjBPTeqQ+py6IwWTt8FNomCyB92gJS2mQLKj Ukw1oupafCLylz63XC6H4PsxJHAlcC0jApDASgen7YFRzXhABpVMTR4g2jFJAFfd/GNj gn+ETC+qe3WqLKYH9v2SlrLywgzI6dX2zLTK1/e+d69aN3sq3jXRzJCV2ihwIB35/By5 V8ImTIlCV/NgY7EEahN3IER0WrWKhtzfE9f6neBEBn/ZT612j43dunZJjzairAq0vj3r hzeA== X-Gm-Message-State: AOAM530GvUn0jhXs3gva5ZrQJ6uf+xWDgLUb46fIFBGCH+ycmBj8774Z GklXK7mZVbjsk5ZQVyqzSuIP/Q== X-Received: by 2002:adf:f3cd:: with SMTP id g13mr16503875wrp.45.1593007489056; Wed, 24 Jun 2020 07:04:49 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 185sm8237788wmz.22.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 4C6F91FF87; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 01/25] iotests: Fix 051 output after qdev_init_nofail() removal Date: Wed, 24 Jun 2020 15:04:22 +0100 Message-Id: <20200624140446.15380-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Kevin Wolf , Thomas Huth , berrange@redhat.com, "open list:Block layer core" , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , richard.henderson@linaro.org, f4bug@amsat.org, Max Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , cota@braap.org, John Snow , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Philippe Mathieu-Daudé Commit 96927c744 replaced qdev_init_nofail() call by isa_realize_and_unref() which has a different error message. Update the test output accordingly. Gitlab CI error after merging b77b5b3dc7: https://gitlab.com/qemu-project/qemu/-/jobs/597414772#L4375 Reported-by: Thomas Huth Signed-off-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Reviewed-by: John Snow Reviewed-by: Thomas Huth Message-Id: <20200616154949.6586-1-philmd@redhat.com> --- tests/qemu-iotests/051.pc.out | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out index 0ea80d35f0e..da8ad871876 100644 --- a/tests/qemu-iotests/051.pc.out +++ b/tests/qemu-iotests/051.pc.out @@ -142,7 +142,7 @@ QEMU X.Y.Z monitor - type 'help' for more information Testing: -drive if=ide QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs media, but drive is empty +(qemu) QEMU_PROG: Device needs media, but drive is empty Testing: -drive if=virtio QEMU X.Y.Z monitor - type 'help' for more information @@ -214,7 +214,7 @@ QEMU X.Y.Z monitor - type 'help' for more information Testing: -drive file=TEST_DIR/t.qcow2,if=ide,readonly=on QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is read-only +(qemu) QEMU_PROG: Block node is read-only Testing: -drive file=TEST_DIR/t.qcow2,if=virtio,readonly=on QEMU X.Y.Z monitor - type 'help' for more information From patchwork Wed Jun 24 14:04:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191652 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp624049ile; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwDXjXpS5BPO9kcDpsNFo4ZOAJvRMRKc7BP2aY5c+aehwD0106o/kUIh9fAPUtnx2vP8L1p X-Received: by 2002:a05:620a:1274:: with SMTP id b20mr25521145qkl.191.1593007513694; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007513; cv=none; d=google.com; s=arc-20160816; b=ErLFjG4b3VeNC5WzAkJiUDySPBkYc0J+f4HyrL+empuJwAyGviH04VGCW968m94K6h mpBWq9Pq8nfi5uw27cMy+g8HRIW0TY1z5m5aaVoVsefpEtT+UD6dV9QLj7ZWTwO0cvZ3 JwL4kCTQebcrfX3SZEkSYQDfJS1mev9TP3T3yCsG4ygwZAdYdiTeIsG7KgmGxKJCH7zz qsGbPuQhLE9Tn42ZAHtTrAypBJUhyySgaAFABiKC1n4FqEmnJLR6XP2mIsSmS/2G+bWT Lc1R4ZFchJQm9Zsyo2hNFM5TkqiY5kK9TkN5LicXPndC1hGVS8z7kdpKCqp5veR+tL4P 5Z+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=oruhTuysScB/ex7jb7bvsCR/Vk8FGwXKgYBMKXsQQK61EOEdSs8n1nsFAmXp4ezeKb jxZ3Cm+Gh+hSE7YlGelQe4LfTeEEuGEzCZZWibL4t/gtXaIrFxawKF8ud9OJqdA8yg1P AMKS+SnYQ5+JS3rqGtoa/1vmmOSoxUTbDtVWb0CP7ndl6W4IVc1xLpia06kudqHVNi1R LrQkYshvJvmLLWcaSVZDlh8bszfQoThRZxtwMC2KuCKbVUi9CrY1fLFkRCukTcUeFUce RhMbsw7gKJDUthYZCp01zdfllSSGtynczC+0aDoskbUjzA0gkAni1Cp+DB8iXsIhw/3J kVrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=D9gmfRWU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id du15si1508892qvb.186.2020.06.24.07.05.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:05:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=D9gmfRWU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo61Y-0004oB-Sh for patch@linaro.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59056) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61H-0004mk-SF for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:55 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:52330) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61D-0003U8-JL for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:55 -0400 Received: by mail-wm1-x32a.google.com with SMTP id q15so2441994wmj.2 for ; Wed, 24 Jun 2020 07:04:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=D9gmfRWUKxwHohfHoYJsgG+1X0WI91GTOdJ1S8ZlUNDyhayRgm0+kCdETfntKMjro3 4GrakiJ9zoOwBVudoThcL3hGhWWdKP4yS/PpaiIc2fWn9+HWTRD4I5Z01dsMKeUW/knq dyHf1OODjwyNpoCYM6cYobUdIpKR5ARMLo6U6000U+wfbsUqVHaVNvyhSmdbBQUOnnqa caitrgDsHV0+jTl0C4N1mWPggRsKZRfNxg5/NJ5d/Rjf0LFPI1QjsgZcPFAgdlso9tQd RxAihyAghIPfq+72oygX1VmhbRx1nr1Wmz261lzZYs1QFMRkNVVQhwfes5bLyQNqY8zO L28g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pp31LDbBMynakzVJX+RF0ypg4zh/bBa29zUb++Htq94=; b=R5Xjp2HY2VX10wkEnYW0t9tThwhWtIvcBbzigmwUzvT8Mavp+zPMzjytyt5GzQuPO9 lTWJVAdF83jwj8gjBPbP4uorMMVjFEH54K0iqShJzDTTZvlancWJ2WpH484kdYlhhRWi WtCbfPwJ23Iret4jdfUVbFcwcsCLz7fXW3W1lx6BNawImUsqIWi49e9LAzcbd2tewrnX Qp+fiMfHQ0TI2+/1OsZOrzMleblmKXW+eBdLZ7bYyGqAsTXdTPSsuSRz1eHIyzJUBMnD ODAaVnwUp2vC3Sva64xCCGI97xfDAwx+EjpfnnRo4jkCeY6sgXoXOWImlmwkutnokHCz bN5w== X-Gm-Message-State: AOAM531/W9gINAOyevjwV+URmQRZCIY+PXnpWhefzqHUWmirEUGl/pbI OaQTMJmqXVq6j49S6mhMXMgomvU/bmk= X-Received: by 2002:a1c:de07:: with SMTP id v7mr6471931wmg.56.1593007490169; Wed, 24 Jun 2020 07:04:50 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id v5sm7368959wre.87.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:47 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 610051FF8C; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 02/25] crypto/linux_keyring: fix 'secret_keyring' configure test Date: Wed, 24 Jun 2020 15:04:23 +0100 Message-Id: <20200624140446.15380-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, David Edmondson , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: David Edmondson The configure test for 'secret_keyring' incorrectly checked the 'have_keyring' variable. Fixes: 54e7aac0562452e4fcab65ca5001d030eef2de15 Signed-off-by: David Edmondson Signed-off-by: Alex Bennée Message-Id: <20200618092636.71832-1-david.edmondson@oracle.com> --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.20.1 Reviewed-by: Thomas Huth diff --git a/configure b/configure index ba88fd18244..69ffd750dbc 100755 --- a/configure +++ b/configure @@ -6369,7 +6369,7 @@ EOF fi if test "$secret_keyring" != "no" then - if test "$have_keyring" == "yes" + if test "$have_keyring" = "yes" then secret_keyring=yes else From patchwork Wed Jun 24 14:04:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191655 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp625742ile; Wed, 24 Jun 2020 07:06:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwEgUqZFAWJupT8M30xfC3fBFrTWHOA1SAmW0EI9OELE3meOVY7iSUkgCcVxPtl4CrU0Lxv X-Received: by 2002:a37:796:: with SMTP id 144mr9335943qkh.84.1593007609907; Wed, 24 Jun 2020 07:06:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007609; cv=none; d=google.com; s=arc-20160816; b=C5w270faww62LgryVTgbSI/H7why9vPB/qNpmy7FqvTZ2kujeTOEA3vZIGLDw6BYzX J/9AUnK9yhpXZPwasTaZ2t8pqEUM/JpxlsZuI/A8/FU0ja8RmyqLwxC7KueuSWIMuYhN j6av7bRbE1/kzXoXKMte6cFxEVbGwvXsVPjIB0lOWd0n0uEj4tH3Y0/xvzjQHepB3CVk GrR1qqjTK39JerKxk3n7Bdmd3dRYjiG8k85ZVwwJOJ9PXBE3GND9qv3B9lrW5vnGhrne kPoa/5uxF3rujHtlbEggjliMDj+OK6Al2QW8qPgA7D3KVEBjuRvpmoVDWwD1QsR7Bkja h6LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=Q4XCnD6m8vEUsW/0GM9b6R4zk6y6Tm/4Lys5GF8x2rv2litWONeLcQyj/KrMBfb1CJ hP7KovTJFYE2p4ZY0QLeyulDkhCpS2tRdblmi7wDvwvH2pvy0woL+qI+uvh4fMNOWZ5w Up76Wl9osPk72GM4dVBRNhrczRJ3EH0HSB3JjrMAeLDB3+bopnIzdhTKoy1gDm6q7Mnd yj7cHGMsHJb6Tn4MiB9+ghelhAtsdGaHAnYEKXMhElc5poeIilBZ4sYnXjDuJabnMg2z 51OhnkOG8Oe/yaGeSjYMHpBbIfTB9Ifx89ITqH9jCqCg9Bkgt1/ETNh0lcBwCUOzjbMM GvZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kwaweC2b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g43si691943qtk.293.2020.06.24.07.06.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:06:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kwaweC2b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo637-0000Za-Bu for patch@linaro.org; Wed, 24 Jun 2020 10:06:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59104) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61J-0004oh-Gq for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:57 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:40934) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61H-0003UG-JY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:57 -0400 Received: by mail-wm1-x333.google.com with SMTP id f139so2643732wmf.5 for ; Wed, 24 Jun 2020 07:04:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=kwaweC2bZkw1W/fNIuATNlTX6TQouli102mvlVEYK7WwCuHm7tMnRNz3dxHew5Fi4y 5/hg4d7ZmkbEljEPaXS18F0/1q4R7VLoQLGdmnvJiQ6sDyjZJFSRXfk8FR2PcPoR37p1 JamsUyt/3W2nUCtfx1Kv4Srs3393dtkEJBRecsq1yzzMWEGepFxiOdUsl/V4whKBKzgV wG8+9BQ19JJtcb6aGSztQwb6mp6IEOMQ4AWuF7Hkdz8rkiJBbEaAgqrF+exxxmFqx9tV b8FTPDIQ1FF0z/hDJGd9vHBUUhtLCwvFO0KeBibyHTMb+BBfR5ouaCGKjC61YDTe/rFR ZSqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m1Iumf3YfxuO4NhxteboEtLnibs/CAVIrPHuCmkmoi4=; b=VHl4ObqiK3gwtLG2rEmwk8FuAwfo4q/7GgrC6M4dNGu+lPXkslvzrHHN1X1oVn3Omc vJkDZW4P2VGeReFvhM6+APpC7aa4kpy2NwRGkESYmCNhDPvhYHa+pl2U1pQhaTh6uXRR CfLWDnm4vpFlgpm3W18e+ba2gLDJGb5lMilKJ3O07IMJGIsfSBTvwONKdysFhfCwUu4D PXv8Z4VgvFCybq5zVFCXFOvF6VjdDEW0vNJ/4q9UUoqufH2SonDsGZHRNf3zDAZUZ9rg FM4kneWKzvOqbHG+n0O9PZiDkGe3m7O5hQvSHakX+MCWGXTGmR7PxXWNhQruvaNdchVy Uhvg== X-Gm-Message-State: AOAM530EMRZb3/PTykxN0golaCAiUkf2cG1zsZ7tJUP5hypL7IvDLKz1 0g5ZkuP5vAF3Wo0cxcN9DNBZOHEKuyU= X-Received: by 2002:a1c:1d46:: with SMTP id d67mr31968222wmd.152.1593007491419; Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id q128sm8069049wma.38.2020.06.24.07.04.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:47 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 77F151FF8F; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 03/25] tests/vm: pass args through to BaseVM's __init__ Date: Wed, 24 Jun 2020 15:04:24 +0100 Message-Id: <20200624140446.15380-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Adding the args parameter to BaseVM's __init__. We will shortly need to pass more parameters to the class so let's just pass args rather than growing the parameter list. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-2-robert.foley@linaro.org> --- tests/vm/basevm.py | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index a80b616a08d..5a58e6c3930 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -61,11 +61,10 @@ class BaseVM(object): # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 - def __init__(self, debug=False, vcpus=None, genisoimage=None, - build_path=None): + def __init__(self, args): self._guest = None - self._genisoimage = genisoimage - self._build_path = build_path + self._genisoimage = args.genisoimage + self._build_path = args.build_path self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) @@ -78,7 +77,7 @@ class BaseVM(object): self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) - self.debug = debug + self.debug = args.debug self._stderr = sys.stderr self._devnull = open(os.devnull, "w") if self.debug: @@ -92,8 +91,8 @@ class BaseVM(object): (",ipv6=no" if not self.ipv6 else ""), "-device", "virtio-net-pci,netdev=vnet", "-vnc", "127.0.0.1:0,to=20"] - if vcpus and vcpus > 1: - self._args += ["-smp", "%d" % vcpus] + if args.jobs and args.jobs > 1: + self._args += ["-smp", "%d" % args.jobs] if kvm_available(self.arch): self._args += ["-enable-kvm"] else: @@ -456,8 +455,7 @@ def main(vmcls): return 1 logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(debug=args.debug, vcpus=args.jobs, - genisoimage=args.genisoimage, build_path=args.build_path) + vm = vmcls(args) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Wed Jun 24 14:04:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191653 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp624131ile; Wed, 24 Jun 2020 07:05:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwQgTxYya3G1fhqO0CgB9cblZJSYYXQycnv+wNX5/LR+66v3YCvJVs4EydALrI4GUp3UmA0 X-Received: by 2002:a0c:fd41:: with SMTP id j1mr31890415qvs.107.1593007518728; Wed, 24 Jun 2020 07:05:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007518; cv=none; d=google.com; s=arc-20160816; b=h1oXU2pvGGcqkIS20E/Sub3Kx6+E/mg8pxHu0u2tOmYkhAaDTG42jjFxpJWkiubd8t /0BdnxQYVCc9pdN1UHX4iErr8mSKNGsXc/sjs72xur8B2kJIVYQa495uRQZWYPh1NRPT mRtTdwEBn453fx1Y0GAxEwiMpvdRpEpe+FaXDqfXyTkoemDOyGxX1hkVUr8siCuIBnqW ELD8DOwuU8v66PCJ4PQHHWaWCJv749BHrWZLuLhqR09vOg7PpQAWQXSmVL36gOcDgMjn Ah0FeI3XoyOaMopIIIhQbNvrBKy6eV1QeyYOXb1lzvXrIM++/YXoL9w1TMrvzxautOLF M5DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=iefkR9fA1GemQOczooLzIkt85KIXV0EUBtEahom/gqO9pMWr85FE2v/L7xnE2xXCV3 vmYTkgxgab/zivSjn/lAA+t3is83yAeED3s510J9kpUvikVsGAlLWaEhWioEcNiH/+o2 o+dSY44A0TZuFCZ8JjVnMhqP5sNICZ/RBpImU9PcW3RTujxBXaGu9eGWQIxJRdSidAqB +QK40Z1Opq+COT5yZG4enYTjjpSsdfoxGBb8hRIl9gpraqbGCX2MpGXW40O+RWom7iji o97jctcfhsQM9aEPdNyQNZLlEIzZ+5rtRAFUiNthOhWThztsawcDXsprMq8aXnvgQc9e VrGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kRb9x49U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id dt16si13558245qvb.118.2020.06.24.07.05.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:05:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kRb9x49U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58380 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo61d-0004vV-VL for patch@linaro.org; Wed, 24 Jun 2020 10:05:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61L-0004t0-Gi for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:32850) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61H-0003Ut-LY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: by mail-wr1-x42b.google.com with SMTP id j94so2448518wrj.0 for ; Wed, 24 Jun 2020 07:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=kRb9x49Uqtz+fvIwF5MB3aImZKjEBV3gRH6my7iMox9DI74HOCeYeV4jsF3/aLCiPx RARUxGWqqAj0LVZ+JGQfBdagYnec1H7Is34OkSx1s7rYHESQXst6pVVb45rZ0ZwAe3+i fXYlmFTwzeaewwiNlhaW01IeJxZe1nkpb9nOOkqnLEXqMBkk0IfGBYdFrBmac+8hwQmm DykpeU2TqUzujJ0maiJ2tLjprrYVvp4efKqAX9LWx+TRDzh7DjDr2pgN3WIVdbSKRAOy iO478efuPDto127kFOwMt/hJdkQ/1ZJfy8OAAdp1ZNhOhElefeQz6nN3y320xhA3zb/8 l7Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Uqu+G7KWNLFFsBFSWuF2fzigErk1o01AVduM2CJF95Y=; b=PKlMyJmZRHpW1dmqzYMvM6JM7CoC1gUt0QuJG2DbybWoyfy/xWLDBeKGtHEsJvuVOI Lud40ruSdsI+eXbTUwfid6CCzklbDFeHrrWcXWRDJX1XlYa/4eMrxANGHkGjkxz3GsCr lVQ4lZJFFDldWZqxPmZcHoEO/YcBeqWHagaIm9RPn+e/wrJhRmLfAMwaImP7yJDwGVEv g6K6jV7/nzEAIrz5JHqtVOC6O/diFHoHZdNIaclx8CrhNTya7IJUaxXWNHs2RO1SfLYk uCJPhBAOo0EAWJhudjO+lI6//BUL5BLqgWNqxl2yERFjnp6u6o1ZH0xkG03Exh2Djav2 e8Xg== X-Gm-Message-State: AOAM531t3/9sQf0XYtctzNbfb4UQgk0BhN83v81UPHxW8TDgw9eIm1ul CDaSleHZ1+G8bVg+SyBVcHR1UQ== X-Received: by 2002:a5d:6a06:: with SMTP id m6mr15411111wru.321.1593007494037; Wed, 24 Jun 2020 07:04:54 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 65sm23055973wre.6.2020.06.24.07.04.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:50 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 8F78C1FF90; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 04/25] tests/vm: Add configuration to basevm.py Date: Wed, 24 Jun 2020 15:04:25 +0100 Message-Id: <20200624140446.15380-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Added use of a configuration to tests/vm/basevm.py. The configuration provides parameters used to configure a VM. This allows for providing alternate configurations to the VM being created/launched. cpu, machine, memory, and NUMA configuration are all examples of configuration which we might want to vary on the VM being created or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-3-robert.foley@linaro.org> --- tests/vm/basevm.py | 172 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 133 insertions(+), 39 deletions(-) -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5a58e6c3930..cfe20c58f7e 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -29,16 +29,41 @@ import tempfile import shutil import multiprocessing import traceback - -SSH_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa")).read() -SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa.pub")).read() - +import shlex + +SSH_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa") +SSH_PUB_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa.pub") + +# This is the standard configuration. +# Any or all of these can be overridden by +# passing in a config argument to the VM constructor. +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : 'pc', + 'guest_user' : "qemu", + 'guest_pass' : "qemupass", + 'root_pass' : "qemupass", + 'ssh_key_file' : SSH_KEY_FILE, + 'ssh_pub_key_file': SSH_PUB_KEY_FILE, + 'memory' : "4G", + 'extra_args' : [], + 'qemu_args' : "", + 'dns' : "", + 'ssh_port' : 0, + 'install_cmds' : "", + 'boot_dev_type' : "block", + 'ssh_timeout' : 1, +} +BOOT_DEVICE = { + 'block' : "-drive file={},if=none,id=drive0,cache=writeback "\ + "-device virtio-blk,drive=drive0,bootindex=0", + 'scsi' : "-device virtio-scsi-device,id=scsi "\ + "-drive file={},format=raw,if=none,id=hd0 "\ + "-device scsi-hd,drive=hd0,bootindex=0", +} class BaseVM(object): - GUEST_USER = "qemu" - GUEST_PASS = "qemupass" - ROOT_PASS = "qemupass" envvars = [ "https_proxy", @@ -57,25 +82,38 @@ class BaseVM(object): poweroff = "poweroff" # enable IPv6 networking ipv6 = True + # This is the timeout on the wait for console bytes. + socket_timeout = 120 # Scale up some timeouts under TCG. # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 - def __init__(self, args): + def __init__(self, args, config=None): self._guest = None self._genisoimage = args.genisoimage self._build_path = args.build_path + # Allow input config to override defaults. + self._config = DEFAULT_CONFIG.copy() + if config != None: + self._config.update(config) + self.validate_ssh_keys() self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) atexit.register(shutil.rmtree, self._tmpdir) - - self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa") - open(self._ssh_key_file, "w").write(SSH_KEY) - subprocess.check_call(["chmod", "600", self._ssh_key_file]) - - self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") - open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) + # Copy the key files to a temporary directory. + # Also chmod the key file to agree with ssh requirements. + self._config['ssh_key'] = \ + open(self._config['ssh_key_file']).read().rstrip() + self._config['ssh_pub_key'] = \ + open(self._config['ssh_pub_key_file']).read().rstrip() + self._ssh_tmp_key_file = os.path.join(self._tmpdir, "id_rsa") + open(self._ssh_tmp_key_file, "w").write(self._config['ssh_key']) + subprocess.check_call(["chmod", "600", self._ssh_tmp_key_file]) + + self._ssh_tmp_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") + open(self._ssh_tmp_pub_key_file, + "w").write(self._config['ssh_pub_key']) self.debug = args.debug self._stderr = sys.stderr @@ -84,11 +122,14 @@ class BaseVM(object): self._stdout = sys.stdout else: self._stdout = self._devnull + netdev = "user,id=vnet,hostfwd=:127.0.0.1:{}-:22" self._args = [ \ - "-nodefaults", "-m", "4G", - "-cpu", "max", - "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + - (",ipv6=no" if not self.ipv6 else ""), + "-nodefaults", "-m", self._config['memory'], + "-cpu", self._config['cpu'], + "-netdev", + netdev.format(self._config['ssh_port']) + + (",ipv6=no" if not self.ipv6 else "") + + (",dns=" + self._config['dns'] if self._config['dns'] else ""), "-device", "virtio-net-pci,netdev=vnet", "-vnc", "127.0.0.1:0,to=20"] if args.jobs and args.jobs > 1: @@ -99,6 +140,55 @@ class BaseVM(object): logging.info("KVM not available, not using -enable-kvm") self._data_args = [] + if self._config['qemu_args'] != None: + qemu_args = self._config['qemu_args'] + qemu_args = qemu_args.replace('\n',' ').replace('\r','') + # shlex groups quoted arguments together + # we need this to keep the quoted args together for when + # the QEMU command is issued later. + args = shlex.split(qemu_args) + self._config['extra_args'] = [] + for arg in args: + if arg: + # Preserve quotes around arguments. + # shlex above takes them out, so add them in. + if " " in arg: + arg = '"{}"'.format(arg) + self._config['extra_args'].append(arg) + + def validate_ssh_keys(self): + """Check to see if the ssh key files exist.""" + if 'ssh_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + + def wait_boot(self, wait_string=None): + """Wait for the standard string we expect + on completion of a normal boot. + The user can also choose to override with an + alternate string to wait for.""" + if wait_string is None: + if self.login_prompt is None: + raise Exception("self.login_prompt not defined") + wait_string = self.login_prompt + # Intentionally bump up the default timeout under TCG, + # since the console wait below takes longer. + timeout = self.socket_timeout + if not kvm_available(self.arch): + timeout *= 8 + self.console_init(timeout=timeout) + self.console_wait(wait_string) + + def __getattr__(self, name): + # Support direct access to config by key. + # for example, access self._config['cpu'] by self.cpu + if name.lower() in self._config.keys(): + return self._config[name.lower()] + return object.__getattribute__(self, name) + def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -130,8 +220,9 @@ class BaseVM(object): "-t", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=" + os.devnull, - "-o", "ConnectTimeout=1", - "-p", self.ssh_port, "-i", self._ssh_key_file] + "-o", + "ConnectTimeout={}".format(self._config["ssh_timeout"]), + "-p", self.ssh_port, "-i", self._ssh_tmp_key_file] # If not in debug mode, set ssh to quiet mode to # avoid printing the results of commands. if not self.debug: @@ -180,14 +271,14 @@ class BaseVM(object): "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)] def boot(self, img, extra_args=[]): - args = self._args + [ - "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img, - "-device", "virtio-blk,drive=drive0,bootindex=0"] - args += self._data_args + extra_args + boot_dev = BOOT_DEVICE[self._config['boot_dev_type']] + boot_params = boot_dev.format(img) + args = self._args + boot_params.split(' ') + args += self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path = get_qemu_path(self.arch, self._build_path) guest = QEMUMachine(binary=qemu_path, args=args) - guest.set_machine('pc') + guest.set_machine(self._config['machine']) guest.set_console() try: guest.launch() @@ -301,7 +392,8 @@ class BaseVM(object): self.console_send(command) def console_ssh_init(self, prompt, user, pw): - sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip() + sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" \ + % self._config['ssh_pub_key'].rstrip() self.console_wait_send("login:", "%s\n" % user) self.console_wait_send("Password:", "%s\n" % pw) self.console_wait_send(prompt, "mkdir .ssh\n") @@ -360,23 +452,23 @@ class BaseVM(object): "local-hostname: {}-guest\n".format(name)]) mdata.close() udata = open(os.path.join(cidir, "user-data"), "w") - print("guest user:pw {}:{}".format(self.GUEST_USER, - self.GUEST_PASS)) + print("guest user:pw {}:{}".format(self._config['guest_user'], + self._config['guest_pass'])) udata.writelines(["#cloud-config\n", "chpasswd:\n", " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, - self.GUEST_PASS), + " root:%s\n" % self._config['root_pass'], + " %s:%s\n" % (self._config['guest_user'], + self._config['guest_pass']), " expire: False\n", "users:\n", - " - name: %s\n" % self.GUEST_USER, + " - name: %s\n" % self._config['guest_user'], " sudo: ALL=(ALL) NOPASSWD:ALL\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], " - name: root\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], "locale: en_US.UTF-8\n"]) proxy = os.environ.get("http_proxy") if not proxy is None: @@ -447,15 +539,17 @@ def parse_args(vmcls): parser.disable_interspersed_args() return parser.parse_args() -def main(vmcls): +def main(vmcls, config=None): try: + if config == None: + config = {} args, argv = parse_args(vmcls) if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(args) + vm = vmcls(args, config=config) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Wed Jun 24 14:04:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191661 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp628555ile; Wed, 24 Jun 2020 07:09:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxU9M/Bly0LOnSYlk27NNeQCqxXV0QAxiF1UgIViWWGI0Uglc2qaAUGQir9N64qs3SL0N2P X-Received: by 2002:a25:bd09:: with SMTP id f9mr46881794ybk.344.1593007781676; Wed, 24 Jun 2020 07:09:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007781; cv=none; d=google.com; s=arc-20160816; b=wY1HBZXtKmyy8Usr+GFNOc6O/+gRXbPGYYOTNRRFaJB5AXv44GKaqvqx0DGSfo9jvu XcW5ND4bTd7cdlmo7WCoYc15FeN1x1yHXIqXdgku6LzkC0n893jf/TH0e5RuTH2xTCu3 AVHzSCcF4LFjnSChgwaOFuyEbyfBAwLsHeDTpG32XBTrm/OLDodA+Wvn1a9BgAOk8ydD iIBMaoz6dmEOkyFQeJxjicg6CJQos+/0wX2dC9jn9NgXJNRjAi+rtSD9JFdkiDJiOsop T7cv9c0VJL6SHsl6dYNhwCYdLmUWQrWUTpYVvV8bZgdWG/JMTuoe+ZqN9L0urHB+ogyA ltMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=QTL5Glu8uU6pFQ8/1tOxjnu4amcv5cf3peO4npaLek2k6QM90hfZZ34PCS31jM1C86 NQPgg91FwNeWqXWJwrwk6+uxE1M3gYbES4LC9BkSB1wHy+9J2F04ZJBpL05coypr3jXK NyczszLmBgjfEoBClBSpVkY5j0us7FGGG4xX5ns5WUqda/ml3zo/xU/PQa2tBKmxfo1U OV6yQUpi3AS0e2lavgieMjsuAt+3quhSSZx34SHRfZZ2eyLr1+CPuzKJ2upa82iNFl+O PeydvGOSEnu2jb8AVSxg8wDo7l0SogADQFKgXeb9gOh83tr/eHFesOu3iPKJygzEuEgM qONQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FolOEAN8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t13si19898559ybl.25.2020.06.24.07.09.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:09:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FolOEAN8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo65t-0007AK-5p for patch@linaro.org; Wed, 24 Jun 2020 10:09:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61O-0004ya-PB for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:02 -0400 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:37580) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61K-0003VL-84 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:02 -0400 Received: by mail-wm1-x341.google.com with SMTP id o2so2656340wmh.2 for ; Wed, 24 Jun 2020 07:04:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=FolOEAN8ONAYAXa/5Tn3d0WoK8TDgC3vjSPzSiNColv5bTr9B/4WQzWDfsnPHcENne EiIVxi9eMlwu7v4uh8cRLNMmUCZ4YLzJbAEGRM0srXQwkeCPUZIKkzbOiXabtRg54/H2 baeDbVO4GBEFv7u8Mf+02fVnZIcCC1b7SkluF+aJA3a1ZWSIGnAFNIXiwafeajZlJU2R PVp9jvSRNz0ixgFj6TUK3vJMjIg4iOSwigTPmDQKJlBhpvcGun2fFrRKismQKp4/1U2d pMWiLgIIS1oraDQa5RqxVI9qfrdJTD68uz/WM07XjuUbPqNcvjkaCRmpJc+1t73yzk1A Kgsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KjVUPk/VZOkBn/Ptl1C1G/mY93IJVmlYmN70Nj+SmGE=; b=PCpoOKzGD9i+E9yAZSyjFvlP7qZw3XmTt4wb38Y4/5kfMmDWy6vuZmk3ls2Z5e/i+S Hb6f8GtKMfl+whapXe5uhUW7Y+4L6+rz/XmKdVEMwph5tTMUXvuEIh2Iwznqgec5ev/5 hfiMc+Zv4fRP36uLFDhkGze4am0Ts5kuiZ84Y5R9BPu0IDYJNUvWty2QPyTkhTb/Xgye fKgzVDSX6ohpqZTCmw0y1HqMDIfC8lMyE7xPjbv3Dx8ujMphqD8Z+Q02v8FHWuqSRHOm aUoQPqq++hLiQKfAv/NIr/4McyP2sn7xTGBHFQPpfGHdxUH9FrhfGlxzFHmtl5qhvdBt hmXw== X-Gm-Message-State: AOAM530G2qUmsvp6jH0Q3qPjhCurJCta0jQ+OW3R7HONKgL5WFPHGEnD zpkLV2u9+X8yW5XNEpajKK0QdA== X-Received: by 2002:a1c:6788:: with SMTP id b130mr30108151wmc.100.1593007496604; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id k126sm2719576wme.17.2020.06.24.07.04.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id ABFD61FF91; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 05/25] tests/vm: Added configuration file support Date: Wed, 24 Jun 2020 15:04:26 +0100 Message-Id: <20200624140446.15380-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::341; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x341.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Changes to tests/vm/basevm.py to allow accepting a configuration file as a parameter. Allows for specifying VM options such as cpu, machine, memory, and arbitrary qemu arguments for specifying options such as NUMA configuration. Also added an example conf_example_aarch64.yml and conf_example_x86.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-4-robert.foley@linaro.org> --- configure | 9 ++++++ tests/vm/Makefile.include | 6 ++++ tests/vm/basevm.py | 40 +++++++++++++++++++++++- tests/vm/conf_example_aarch64.yml | 51 +++++++++++++++++++++++++++++++ tests/vm/conf_example_x86.yml | 50 ++++++++++++++++++++++++++++++ 5 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 tests/vm/conf_example_aarch64.yml create mode 100644 tests/vm/conf_example_x86.yml -- 2.20.1 diff --git a/configure b/configure index 69ffd750dbc..c1ec271931f 100755 --- a/configure +++ b/configure @@ -958,6 +958,13 @@ do fi done +# Check for existence of python3 yaml, needed to +# import yaml config files into vm-build. +python_yaml="no" +if $(python3 -c "import yaml" 2> /dev/null); then + python_yaml="yes" +fi + : ${smbd=${SMBD-/usr/sbin/smbd}} # Default objcc to clang if available, otherwise use CC @@ -6751,6 +6758,7 @@ if test "$docs" != "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" != "no" ; then echo "smbd $smbd" @@ -7830,6 +7838,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak +echo "PYTHON_YAML=$python_yaml" >> $config_host_mak echo "CC=$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then echo "IASL=$iasl" >> $config_host_mak diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index a253aba4579..f6c3892bb28 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -44,6 +44,12 @@ endif @echo " QEMU_LOCAL=1 - Use QEMU binary local to this build." @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=/path/to/qemu-img - Change path to qemu-img tool" +ifeq ($(PYTHON_YAML),yes) + @echo " QEMU_CONFIG=/path/conf.yml - Change path to VM configuration .yml file." +else + @echo " (install python3-yaml to enable support for yaml file to configure a VM.)" +endif + @echo " See conf_example_*.yml for file format details." vm-build-all: $(addprefix vm-build-, $(IMAGES)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index cfe20c58f7e..fa56fbbb4b6 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -481,7 +481,6 @@ class BaseVM(object): cwd=cidir, stdin=self._devnull, stdout=self._stdout, stderr=self._stdout) - return os.path.join(cidir, "cloud-init.iso") def get_qemu_path(arch, build_path=None): @@ -497,6 +496,41 @@ def get_qemu_path(arch, build_path=None): qemu_path = "qemu-system-" + arch return qemu_path +def parse_config(config, args): + """ Parse yaml config and populate our config structure. + The yaml config allows the user to override the + defaults for VM parameters. In many cases these + defaults can be overridden without rebuilding the VM.""" + if args.config: + config_file = args.config + elif 'QEMU_CONFIG' in os.environ: + config_file = os.environ['QEMU_CONFIG'] + else: + return config + if not os.path.exists(config_file): + raise Exception("config file {} does not exist".format(config_file)) + # We gracefully handle importing the yaml module + # since it might not be installed. + # If we are here it means the user supplied a .yml file, + # so if the yaml module is not installed we will exit with error. + try: + import yaml + except ImportError: + print("The python3-yaml package is needed "\ + "to support config.yaml files") + # Instead of raising an exception we exit to avoid + # a raft of messy (expected) errors to stdout. + exit(1) + with open(config_file) as f: + yaml_dict = yaml.safe_load(f) + + if 'qemu-conf' in yaml_dict: + config.update(yaml_dict['qemu-conf']) + else: + raise Exception("config file {} is not valid"\ + " missing qemu-conf".format(config_file)) + return config + def parse_args(vmcls): def get_default_jobs(): @@ -536,6 +570,9 @@ def parse_args(vmcls): help="run tests with a snapshot") parser.add_option("--genisoimage", default="genisoimage", help="iso imaging tool") + parser.add_option("--config", "-c", default=None, + help="Provide config yaml for configuration. "\ + "See config_example.yaml for example.") parser.disable_interspersed_args() return parser.parse_args() @@ -547,6 +584,7 @@ def main(vmcls, config=None): if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 + config = parse_config(config, args) logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) vm = vmcls(args, config=config) diff --git a/tests/vm/conf_example_aarch64.yml b/tests/vm/conf_example_aarch64.yml new file mode 100644 index 00000000000..9d44ae356f7 --- /dev/null +++ b/tests/vm/conf_example_aarch64.yml @@ -0,0 +1,51 @@ +# +# Example yaml for use by any of the scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: qemu + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # If one key is provided, both must be provided. + #ssh_key: /complete/path/of/your/keyfile/id_rsa + #ssh_pub_key: /complete/path/of/your/keyfile/id_rsa.pub + + cpu: max + machine: virt,gic-version=max + memory: 16G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=16,sockets=2,cores=8 + -numa node,cpus=0-3,nodeid=0 -numa node,cpus=4-7,nodeid=1 + -numa node,cpus=8-11,nodeid=2 -numa node,cpus=12-15,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: 1.234.567.89 + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + #ssh_port: 5555 + + # To install a different set of packages, provide a command to issue + #install_cmds: "apt-get update ; apt-get build-dep -y qemu" + + # Or to skip the install entirely, just provide "" + #install_cmds: "" diff --git a/tests/vm/conf_example_x86.yml b/tests/vm/conf_example_x86.yml new file mode 100644 index 00000000000..78d3f5830fa --- /dev/null +++ b/tests/vm/conf_example_x86.yml @@ -0,0 +1,50 @@ +# +# Example yaml for use by any of the x86 based scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: "qemu" + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # Provide default ssh keys of current user. + # You need to edit the below for your user. + #ssh_key_file: /home//.ssh/id_rsa + #ssh_pub_key_file: /home//.ssh/id_rsa.pub + + cpu: max + machine: pc + memory: 8G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=8,sockets=2,cores=4 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node0 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node1 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node2 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node3 + -numa node,cpus=0-1,nodeid=0 -numa node,cpus=2-3,nodeid=1 + -numa node,cpus=4-5,nodeid=2 -numa node,cpus=6-7,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: "1.234.567.89" + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + ssh_port: 5555 From patchwork Wed Jun 24 14:04:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191656 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp626211ile; Wed, 24 Jun 2020 07:07:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyWExi9745mLS+Ri1AF/SzPKe4847v+jY/JMmFPaCbZPgBcoSIsw5D6C/vDPEsW57wxzNZ+ X-Received: by 2002:ac8:4e2d:: with SMTP id d13mr21515692qtw.52.1593007636382; Wed, 24 Jun 2020 07:07:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007636; cv=none; d=google.com; s=arc-20160816; b=LeeYIzndRpZEwskuwd6X4hP/MGmagsfgjJVbv3oFC70qt22Md3fMv4TRuyrgaGKJJD UZAkquC4DHzFCGZuDw6n+1r2SpEy7BSEc0SfBSLdlrBArbgelBTBY+BWy6tf3swaVoiT rUdny2sMKd3PquzCoXEm33FhpU6MJvAf5oHXtLWUoODZvloSARuhRDL698uywGcS75/A 2GTdjN9PyCs7vneXCpoWtw/OkLtxItfJj+AYvJnJ0LRtpSrLMbWNBOdsoaxDslEnFNLv 2oVCZsS3vucOLGmRaXBiSHLoR8Z/v+PhcSyEW45HKvj/J2bS+bME+hHYiK8vZ6FcjoGO gXJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=dQFtLzkyACPffgRV7AQGHXsrX0uR76C3Gy17xSjomJgnLkk8wOjZ8+kdsohZdPnp3e HG+GRYsgpg1KQ63TH6OuVa/qUxnQaW24OET80H7UdsQlaEL7xzdEM0THYHVTpfCUVwWc NZF5PNUxyS7To7EXTiSj2FpKUyAmwlRNDMtW1faPICnJZxNaBSnsxdrRIQwZ7onoQnAZ DRC8p35cdf+V5RUGAp0ZuvvGCiQHXWwaekHh99mw3fEjqvRE5lPyv8fCuEjxpH/z2vC0 ziNez0teQu3HMBR8EsgCUhBxRme9erM3TJfTJHfp2MGkAUQ7O4TCaBYE0/kJTiiOhVxZ qxKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hydnMo+0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v132si7018493qkb.64.2020.06.24.07.07.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:07:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hydnMo+0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63X-0000B0-Oh for patch@linaro.org; Wed, 24 Jun 2020 10:07:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61M-0004uO-6D for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:00 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:44986) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61I-0003V5-QY for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:04:59 -0400 Received: by mail-wr1-x434.google.com with SMTP id b6so2392890wrs.11 for ; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=hydnMo+0y0ZC++K80GMn9ZR6OJ1W8xU5OuMxzDiD6fkNE4XFTiP+DhjJhjPi/L3vzn tr0Da1AbSDCTb2QGgrhxRtrZEgyn2GLBPf9oBommnvINOYKrZ9czErJhLXCqHxI/KT+L 6rIKpXsLC1u5kb7p+LmojWs01r2ACvKYVY03CgFE+kqJ+b1eoVatEumrjVrrGMNxSCn0 8ECEllraXFAatAy7UlcOyQCJ42l4jdC8/r9SfGlWhQNq775m3HIAB97DGLTRqfAObTE5 gdCST3z1AihIUzxucqDZW8IGYu3cLawcNpz0EGxda3CStDjJjtfahlOsE40b1rtsVQcx nrqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yfaWvkBHiSQ15nCnLOaiCfwwNxNyBgCuiCJ/EYYdU+8=; b=jBIuDX2TJEBkOrNxqi8vYr+VuVrwVioNkjfhvO3zsN0w9s7BwPZNZ1APRIrYj8Ubhz Nt+SsHMDivs3CEwzMwEMXITXgZdW+2ZoOXTQ25EynINbN+1f+0McQZtv0sZRsaRLUybU GQ6OoDOL+sI0W9LXHXetOV5tk5I/Z12llt5ryMWIRtU2vt3x3/JO92qa2bYAOOMkhjZR Novaa3PWRENwAgif/rn496wp3arFBaoPn972KFzrrQTAb2SN5F0bhu/Y2nS9Q56zY2C9 6LsqDXiSJ1baYJIAVFuP109syzaBNIGdhHEoehSXObSaIvwhCHsnRKsPCzZuICDGky0j s66A== X-Gm-Message-State: AOAM531XZWIPmzpjZ5ZTREINPgXzZPaRLNeM+Nvlh1kawXnp91g9ihhP W77xtJuyC3kKrWKAPAa5COJF5g== X-Received: by 2002:adf:8067:: with SMTP id 94mr29864433wrk.427.1593007495439; Wed, 24 Jun 2020 07:04:55 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id y25sm8232106wma.19.2020.06.24.07.04.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:51 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C30CE1FF92; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 06/25] tests/vm: Add common Ubuntu python module Date: Wed, 24 Jun 2020 15:04:27 +0100 Message-Id: <20200624140446.15380-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Philippe_Mathieu-Da?= =?utf-8?q?ud=C3=A9?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Alex_Benn?= =?utf-8?b?w6ll?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Add a common Ubuntu python module and make use of it with the ubuntu.i386 script. This is preparation for adding an Ubuntu script ubuntu.aarch64. Splitting out the common logic such as build_image() will reduce duplication. Signed-off-by: Robert Foley Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-5-robert.foley@linaro.org> --- tests/vm/ubuntu.i386 | 46 +++++++++------------------------ tests/vm/ubuntuvm.py | 60 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+), 34 deletions(-) create mode 100644 tests/vm/ubuntuvm.py -- 2.20.1 diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 24527cc78c7..5ce72610a6c 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -11,15 +11,22 @@ # the COPYING file in the top-level directory. # -import os import sys -import subprocess import basevm -import time +import ubuntuvm -class UbuntuX86VM(basevm.BaseVM): +DEFAULT_CONFIG = { + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y qemu,"\ + "apt-get install -y libfdt-dev language-pack-en", +} + +class UbuntuX86VM(ubuntuvm.UbuntuVM): name = "ubuntu.i386" arch = "i386" + image_link="https://cloud-images.ubuntu.com/releases/bionic/"\ + "release-20191114/ubuntu-18.04-server-cloudimg-i386.img" + image_sha256="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef" BUILD_SCRIPT = """ set -e; cd $(mktemp -d); @@ -29,34 +36,5 @@ class UbuntuX86VM(basevm.BaseVM): make --output-sync {target} -j{jobs} {verbose}; """ - def build_image(self, img): - cimg = self._download_with_cache( - "https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img", - sha256sum="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef") - img_tmp = img + ".tmp" - subprocess.check_call(["cp", "-f", cimg, img_tmp]) - self.exec_qemu_img("resize", img_tmp, "50G") - self.boot(img_tmp, extra_args = [ - "-device", "VGA", - "-cdrom", self.gen_cloud_init_iso() - ]) - self.wait_ssh() - self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get install -y cloud-initramfs-growroot") - # Don't check the status in case the guest hang up too quickly - self.ssh_root("sync && reboot") - time.sleep(5) - self.wait_ssh() - # The previous update sometimes doesn't survive a reboot, so do it again - self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get build-dep -y qemu") - self.ssh_root_check("apt-get install -y libfdt-dev language-pack-en") - self.ssh_root("poweroff") - self.wait() - os.rename(img_tmp, img) - return 0 - if __name__ == "__main__": - sys.exit(basevm.main(UbuntuX86VM)) + sys.exit(basevm.main(UbuntuX86VM, DEFAULT_CONFIG)) diff --git a/tests/vm/ubuntuvm.py b/tests/vm/ubuntuvm.py new file mode 100644 index 00000000000..6689ad87aa8 --- /dev/null +++ b/tests/vm/ubuntuvm.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# +# Ubuntu VM testing library +# +# Copyright 2017 Red Hat Inc. +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. + +import os +import subprocess +import basevm + +class UbuntuVM(basevm.BaseVM): + + def __init__(self, args, config=None): + self.login_prompt = "ubuntu-{}-guest login:".format(self.arch) + basevm.BaseVM.__init__(self, args, config) + + def build_image(self, img): + """Build an Ubuntu VM image. The child class will + define the install_cmds to init the VM.""" + os_img = self._download_with_cache(self.image_link, + sha256sum=self.image_sha256) + img_tmp = img + ".tmp" + subprocess.check_call(["cp", "-f", os_img, img_tmp]) + self.exec_qemu_img("resize", img_tmp, "+50G") + ci_img = self.gen_cloud_init_iso() + + self.boot(img_tmp, extra_args = [ "-device", "VGA", "-cdrom", ci_img, ]) + + # First command we issue is fix for slow ssh login. + self.wait_ssh(wait_root=True, + cmd="chmod -x /etc/update-motd.d/*") + # Wait for cloud init to finish + self.wait_ssh(wait_root=True, + cmd="ls /var/lib/cloud/instance/boot-finished") + self.ssh_root("touch /etc/cloud/cloud-init.disabled") + # Disable auto upgrades. + # We want to keep the VM system state stable. + self.ssh_root('sed -ie \'s/"1"/"0"/g\' '\ + '/etc/apt/apt.conf.d/20auto-upgrades') + self.ssh_root("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list") + + # If the user chooses not to do the install phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + # Issue the install commands. + # This can be overriden by the user in the config .yml. + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.graceful_shutdown() + os.rename(img_tmp, img) + return 0 From patchwork Wed Jun 24 14:04:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191662 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp630267ile; Wed, 24 Jun 2020 07:11:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhNeXz6WDvmarqSsFPG6OzahRSS618aXRgX7RJF1zkwq3gtMP4qIwQuFSa4tXJDI0h4TVx X-Received: by 2002:a25:3784:: with SMTP id e126mr48238824yba.267.1593007894187; Wed, 24 Jun 2020 07:11:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007894; cv=none; d=google.com; s=arc-20160816; b=0idIHRbee/SU5IzOBUnI57BTAz5D5fx62gswOkXT2wn+UAt7BRrkU2ZSf1mkqmoT9t oY7StAQQQhJF4HdyfsKAo/Pkc/wPUBtq/X4MsAVBCLa9DSadGnoQ4b0jKnMllPJUHGoF nmvlUpV4nFFbiqjLQ4+2gfUhBtiFN3id0sfu3YYe57eVvkAxxl9Y3vVk/Sl/ou6XYZ+7 SjsWbFO/4HwkXo8PGSC7WZMfUzxPzf/j/4lng1CyoVa41jsGXfFIggJ6Z6qyQ6AYJ0gZ TXITf+h5bRP45+kt4ktX7sKOr/0l2ZLAqs5H9PoA8VNdQB9OfPLqKel8b5gwXBcvwVCv x2+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=v4lNYM3+kyWgUg6vpesR/YV0UKMrkSwL5rQK2OhM7yADSJKKVNdFsWaOANCwunMpN5 /Lq4D1XewHQYDmUSVCdicsarOh00LnjkqwRe0bNHCKabn2Pp5Mslpe0282GFVxnA5imN xhAIwKDCPUOwYFC9vleYfpsaiP7PqKopFMI2rSVbkCiHlZ8wXFuqfpzrioQs5nvTINdL JUt7tYO+ZvsCfgD/bkY/N8B7Kvwka99va2296ObBfh7Z2/zNIX1aH24PasuxuHP3Jqfw bIuLiiMmmkRhYdVSoI7VE+PeWVEx2PQfb6t1G4+sykgC3NoRYiQQWBIpAl6U5kwypPsS 7YtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="dcgPp4/F"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e30si22645532ybi.449.2020.06.24.07.11.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:11:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="dcgPp4/F"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo67h-00022r-Ky for patch@linaro.org; Wed, 24 Jun 2020 10:11:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59234) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61P-00050B-Km for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:03 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:35993) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61L-0003Vd-SQ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:03 -0400 Received: by mail-wm1-x333.google.com with SMTP id 17so2664461wmo.1 for ; Wed, 24 Jun 2020 07:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=dcgPp4/FWU4YXejrJvdHvSf8UQehwYLLrHLPg19rXnm8UnS0tdGrF3NWhENQiqqI9S 1nuwS5fDUPIMUgsMzjmo67aB3HrQsDiuCYBw4CPY8wyxambMV6H+hP+QsdYQ6/cqD/Qh DpfpMvY1rSRJISRXcZEp1wpkvefqnW71owdtYnXnAmhNJh2tdL4yEiCARp9mIFTeK9t7 XNmZ/Gc0BxK1l5e12xOuHRzKHv56jjk+VsBSlGg7TQmDYXS7kJce0N9Oq/SRdfjjgIto Yr3/hSXbIvtbVmb+BO5REL8gXKkpOCAmgxeg+VVs9QrBiFTiC+Tey+MBSoV7KSJ/KQzF 9QtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eHE1ZGUQed0VLFEhjLciTrVecX1BhNTXwaA8+xC9Diw=; b=VURgE9vc+VWoZb9onRGlJmfbeDgyIFkuWJKfbx12k6GFAeZPTQIF9D6o4HZE5bttTV 4dBp1WdAssF6EyRSKfz/qK9b3aQtH7MfnRJ2kmtTCwXSb1+00f3SqfCPwpfLibrJH4e5 +28e5j9lqIY3T7qPeWkX3jJcMWQPacW7rO/Lb4flqDYduWlOxsr5i8qhE/0wdQbt6BrC t5FIyodwZyiafPwIsM9IW041drSXTiGfPsqQGDQVXeUqhTbQCc1P0Yhm9/rwrei9SieJ JA/nTu964qUPfP4S5P8D19bbFrRQjDcUUYR9dfu28g6aTiCaB+cVJO1H3afMsiB2mdUN 4mMA== X-Gm-Message-State: AOAM532Fd8jNgHSp1AX9Z1g3usD2TWYrHmAetLm/HYbsfIKQ9igbEmv/ xrUqzXvhDqkelrUzu66xiBZB7Q== X-Received: by 2002:a7b:cb4c:: with SMTP id v12mr29659257wmj.43.1593007498287; Wed, 24 Jun 2020 07:04:58 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n189sm8178049wmb.43.2020.06.24.07.04.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:53 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id E03021FF93; Wed, 24 Jun 2020 15:04:46 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 07/25] tests/vm: Added a new script for ubuntu.aarch64. Date: Wed, 24 Jun 2020 15:04:28 +0100 Message-Id: <20200624140446.15380-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-6-robert.foley@linaro.org> --- configure | 20 +++++++ tests/vm/Makefile.include | 11 ++++ tests/vm/aarch64vm.py | 106 ++++++++++++++++++++++++++++++++++++++ tests/vm/basevm.py | 12 +++++ tests/vm/ubuntu.aarch64 | 68 ++++++++++++++++++++++++ 5 files changed, 217 insertions(+) create mode 100644 tests/vm/aarch64vm.py create mode 100755 tests/vm/ubuntu.aarch64 -- 2.20.1 diff --git a/configure b/configure index c1ec271931f..980a323ac7f 100755 --- a/configure +++ b/configure @@ -417,6 +417,7 @@ prefix="/usr/local" mandir="\${prefix}/share/man" datadir="\${prefix}/share" firmwarepath="\${prefix}/share/qemu-firmware" +efi_aarch64="" qemu_docdir="\${prefix}/share/doc/qemu" bindir="\${prefix}/bin" libdir="\${prefix}/lib" @@ -1107,6 +1108,8 @@ for opt do ;; --firmwarepath=*) firmwarepath="$optarg" ;; + --efi-aarch64=*) efi_aarch64="$optarg" + ;; --host=*|--build=*|\ --disable-dependency-tracking|\ --sbindir=*|--sharedstatedir=*|\ @@ -1777,6 +1780,7 @@ Advanced options (experts only): --sysconfdir=PATH install config in PATH$confsuffix --localstatedir=PATH install local state in PATH (set at runtime on win32) --firmwarepath=PATH search PATH for firmware files + --efi-aarch64=PATH PATH of efi file to use for aarch64 VMs. --with-confsuffix=SUFFIX suffix for QEMU data inside datadir/libdir/sysconfdir [$confsuffix] --with-pkgversion=VERS use specified string as sub-version of the package --enable-debug enable common debug build options @@ -3593,6 +3597,20 @@ EOF fi fi +############################################ +# efi-aarch64 probe +# Check for efi files needed by aarch64 VMs. +# By default we will use the efi included with QEMU. +# Allow user to override the path for efi also. +if ! test -f "$efi_aarch64"; then + if test -f $source_path/pc-bios/edk2-aarch64-code.fd.bz2; then + # valid after build + efi_aarch64=$PWD/pc-bios/edk2-aarch64-code.fd + else + efi_aarch64="" + fi +fi + ########################################## # libcap-ng library probe if test "$cap_ng" != "no" ; then @@ -6758,6 +6776,7 @@ if test "$docs" != "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "efi_aarch64 $efi_aarch64" echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" != "no" ; then @@ -7838,6 +7857,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak +echo "EFI_AARCH64=$efi_aarch64" >> $config_host_mak echo "PYTHON_YAML=$python_yaml" >> $config_host_mak echo "CC=$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index f6c3892bb28..4fa292765d5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -5,6 +5,9 @@ IMAGES := freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES += ubuntu.i386 centos +ifneq ($(EFI_AARCH64),) +IMAGES += ubuntu.aarch64 +endif endif IMAGES_DIR := $(HOME)/.cache/qemu-vm/images @@ -23,6 +26,11 @@ vm-help vm-test: ifneq ($(GENISOIMAGE),) @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker" @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" +ifneq ($(EFI_AARCH64),) + @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" +else + @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch64)" +endif else @echo " (install genisoimage to build centos/ubuntu images)" endif @@ -65,6 +73,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(V)$(DEBUG), --debug) \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$@" \ --force \ --build-image $@, \ @@ -80,6 +89,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -102,6 +112,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py new file mode 100644 index 00000000000..bb04cb19c91 --- /dev/null +++ b/tests/vm/aarch64vm.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# +# VM testing aarch64 library +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available + +# This is the config needed for current version of QEMU. +# This works for both kvm and tcg. +CURRENT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", +} + +# The minimum minor version of QEMU we will support with aarch64 VMs is 3. +# QEMU versions less than 3 have various issues running these VMs. +QEMU_AARCH64_MIN_VERSION = 3 + +# The DEFAULT_CONFIG will default to a version of +# parameters that works for backwards compatibility. +DEFAULT_CONFIG = {'kvm' : {'cpu' : "host", + 'machine' : "virt,gic-version=host"}, + 'tcg' : {'cpu' : "cortex-a57", + 'machine' : "virt"}, +} + +def get_config_defaults(vmcls, default_config): + """Fetch the configuration defaults for this VM, + taking into consideration the defaults for + aarch64 first, followed by the defaults for this VM.""" + config = default_config + config.update(aarch_get_config_defaults(vmcls)) + return config + +def aarch_get_config_defaults(vmcls): + """Set the defaults for current version of QEMU.""" + config = CURRENT_CONFIG + args, argv = basevm.parse_args(vmcls) + qemu_path = basevm.get_qemu_path(vmcls.arch, args.build_path) + qemu_version = basevm.get_qemu_version(qemu_path) + if qemu_version < QEMU_AARCH64_MIN_VERSION: + error = "\nThis major version of QEMU {} is to old for aarch64 VMs.\n"\ + "The major version must be at least {}.\n"\ + "To continue with the current build of QEMU, "\ + "please restart with QEMU_LOCAL=1 .\n" + print(error.format(qemu_version, QEMU_AARCH64_MIN_VERSION)) + exit(1) + if qemu_version == QEMU_AARCH64_MIN_VERSION: + # We have an older version of QEMU, + # set the config values for backwards compatibility. + if kvm_available('aarch64'): + config.update(DEFAULT_CONFIG['kvm']) + else: + config.update(DEFAULT_CONFIG['tcg']) + return config + +def create_flash_images(flash_dir="./", efi_img=""): + """Creates the appropriate pflash files + for an aarch64 VM.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + fd_null = open(os.devnull, 'w') + subprocess.check_call(["dd", "if=/dev/zero", "of={}".format(flash0_path), + "bs=1M", "count=64"], + stdout=fd_null, stderr=subprocess.STDOUT) + # A reliable way to get the QEMU EFI image is via an installed package or + # via the bios included with qemu. + if not os.path.exists(efi_img): + sys.stderr.write("*** efi argument is invalid ({})\n".format(efi_img)) + sys.stderr.write("*** please check --efi-aarch64 argument or "\ + "install qemu-efi-aarch64 package\n") + exit(3) + subprocess.check_call(["dd", "if={}".format(efi_img), + "of={}".format(flash0_path), + "conv=notrunc"], + stdout=fd_null, stderr=subprocess.STDOUT) + subprocess.check_call(["dd", "if=/dev/zero", + "of={}".format(flash1_path), + "bs=1M", "count=64"], + stdout=fd_null, stderr=subprocess.STDOUT) + fd_null.close() + +def get_pflash_args(flash_dir="./"): + """Returns a string that can be used to + boot qemu using the appropriate pflash files + for aarch64.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + pflash_args_str = "-drive file={},format=raw,if=pflash "\ + "-drive file={},format=raw,if=pflash" + pflash_args = pflash_args_str.format(flash0_path, flash1_path) + return pflash_args.split(" ") + +def get_flash_path(flash_dir, name): + return os.path.join(flash_dir, "{}.img".format(name)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index fa56fbbb4b6..5fd66f6b26a 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -92,6 +92,7 @@ class BaseVM(object): self._guest = None self._genisoimage = args.genisoimage self._build_path = args.build_path + self._efi_aarch64 = args.efi_aarch64 # Allow input config to override defaults. self._config = DEFAULT_CONFIG.copy() if config != None: @@ -496,6 +497,14 @@ def get_qemu_path(arch, build_path=None): qemu_path = "qemu-system-" + arch return qemu_path +def get_qemu_version(qemu_path): + """Get the version number from the current QEMU, + and return the major number.""" + output = subprocess.check_output([qemu_path, '--version']) + version_line = output.decode("utf-8") + version_num = re.split(' |\(', version_line)[3].split('.')[0] + return int(version_num) + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the @@ -573,6 +582,9 @@ def parse_args(vmcls): parser.add_option("--config", "-c", default=None, help="Provide config yaml for configuration. "\ "See config_example.yaml for example.") + parser.add_option("--efi-aarch64", + default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help="Path to efi image for aarch64 VMs.") parser.disable_interspersed_args() return parser.parse_args() diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 new file mode 100755 index 00000000000..21d454c27f1 --- /dev/null +++ b/tests/vm/ubuntu.aarch64 @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +# +# Ubuntu aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import sys +import basevm +import aarch64vm +import ubuntuvm + +DEFAULT_CONFIG = { + 'cpu' : "cortex-a57", + 'machine' : "virt,gic-version=3", + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y --arch-only qemu,"\ + "apt-get install -y libfdt-dev pkg-config language-pack-en", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM + # especially using softmmu. + 'ssh_timeout' : 60, +} + +class UbuntuAarch64VM(ubuntuvm.UbuntuVM): + name = "ubuntu.aarch64" + arch = "aarch64" + image_name = "ubuntu-18.04-server-cloudimg-arm64.img" + image_link = "https://cloud-images.ubuntu.com/releases/18.04/release/" + image_name + image_sha256="0fdcba761965735a8a903d8b88df8e47f156f48715c00508e4315c506d7d3cb1" + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(UbuntuAarch64VM, self).boot(img, extra_args=extra_args) + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(UbuntuAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(UbuntuAarch64VM, defaults)) From patchwork Wed Jun 24 14:04:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191654 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp625612ile; Wed, 24 Jun 2020 07:06:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzUj0rUcRT9Fc8XSgmQkDAnY+z6OZxOaL+kG/Aanlwsyh9ufa+xNsJuCNx2SrfoJwGjhmFc X-Received: by 2002:a37:7747:: with SMTP id s68mr11804541qkc.42.1593007602163; Wed, 24 Jun 2020 07:06:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007602; cv=none; d=google.com; s=arc-20160816; b=pRhkc/ER2VyAY9HpfwSkGceIZJl+gtPbhVGMeRkWDZqWYUxC3zlGomft0mOzPoK1s2 Y4Sykvn/5H0kBuB6ozXxxTi41uRc1VADIMXYSWt7q7Ab7MsdpEmMA9oEtdorwoJN1hds Iocp73fOGWuRG33VJEYjuB9y18km/8GAkUkBl4bYU1cC5vRNBisHVMfvH0VvhmvtXAHP tTuvON0MSZOILeGNk9A4fm6xT44ro3jRvIIpV0oPPvMNb9QtA+HKBt1Bsef8mjl+zVnT ltwbgKBrakkGXu+Gvioikx4SEwBT1ckBlKAxuA7Q9e4yqez3VW/B0I9c8febgc0tqJWe NOZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=pM3yv/pTdNrzp4Ps2ZH7kEHEqn9MWtJXruU8+1kfen55q2ovd2hbWbjuohk/B782Ld EzWj5faRWEaU6I/8qbvSSJGZkCtix6gMOLjs5X7iAHARePbHoOcVM0grUEmgPWENQdPo mFsTRL9C8k8AtAy1zvkj4QSiY5ePxqhhRwsxotgnoV7ar4XGsp75AlVabWbu5n0R+8hT btBPNmCQ9zlaXnbmvA+gmfd+YsnCB/antaIct+3V0B8v6S/Icw8NaXy6k7t/I2hSh/M/ 9uZt3YdCzWthH1N+kVyjBAF2BskacbqHXBlGP17c7CZfDrNUWMYLpU0Wauis5iRyKe6A vkIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=MUJxn1sx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a7si9187827qtb.368.2020.06.24.07.06.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:06:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=MUJxn1sx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo62z-00006T-DC for patch@linaro.org; Wed, 24 Jun 2020 10:06:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61Q-00051J-Hz for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:04 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:47049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61N-0003Vu-9C for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:04 -0400 Received: by mail-wr1-x42c.google.com with SMTP id r12so2376922wrj.13 for ; Wed, 24 Jun 2020 07:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=MUJxn1sxXRAp4DhnvYyDZocNypsJUbI6Pp0bBQ4PYprWSSCQiUpWnpUclj80cUKLwp TjzCO0nh88sMGCxQ4+QHsgXMT+xkABjJkf+cVaqKpQEOw4H0KsjE+Bt5upJ7DpCkuD/G IwQb7u4TthDjDOPygwgyYE8zoWmhHHEQtG1HWrp5lH0pcMkU/Ek2aBf66lvc4eiGcN+4 ChOnEfv1wSe0Hrs8/awuS8KCaO7B0mzqwpAlkh3XyfgUOPq2ERrbAe5AIlqG89HRQieX 2E55R2GFsvuhOXOzUPGb0IIsDQsMkzTIMtZvQq4IID6MzQPcKQQ4ZvHyZQs/yfmX3TKG 8ljA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kdMIZtPe+aZda1/mXQW7aQb8KseTM0tT+oUaOxEz8I4=; b=QhPKVTclEaObuCw+O9Q+m4DwxZycINSKfDbtiyGKhTKKZZTV6IijxkDmXwmUt8mSAh ULir88rJHPmntm/ut2BV5ygP7imHlUQqPR3UBtKEQcbycJUM2GDqVrPghujoW6Pzw7YJ gsjzleO82gMC6ZsXwgAOL9x64XMGSTEBJOrts3wis9ExTHhWC/kCoHsudN43FyD5VmLR bkEbOCzI2lNl+Qb48bH1nVdlVvcoun7NEX7oVYsaZNMOwQB5Ceiv+NMNIhEqC9EXzRpD uAqVuFPbJRXw34GUR0VCjtSHqgjBdTN+BsmbzwXvLlf8lXx5+Juf93ypbcraj2GYM29V NIow== X-Gm-Message-State: AOAM533D54isK9lasU6Kg8o1dYilCTzx2RUUC1NZfJ8uUA2fDFWe9Zzn HTfRkgod2F5PuvsxSbTlvCYzdQ== X-Received: by 2002:a5d:4603:: with SMTP id t3mr4807959wrq.38.1593007499536; Wed, 24 Jun 2020 07:04:59 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z1sm13677635wrh.14.2020.06.24.07.04.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:53 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 04B301FF96; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 08/25] tests/vm: Added a new script for centos.aarch64. Date: Wed, 24 Jun 2020 15:04:29 +0100 Message-Id: <20200624140446.15380-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-7-robert.foley@linaro.org> --- tests/vm/Makefile.include | 3 +- tests/vm/centos-8-aarch64.ks | 51 ++++++++ tests/vm/centos.aarch64 | 227 +++++++++++++++++++++++++++++++++++ 3 files changed, 280 insertions(+), 1 deletion(-) create mode 100644 tests/vm/centos-8-aarch64.ks create mode 100755 tests/vm/centos.aarch64 -- 2.20.1 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 4fa292765d5..39f918a430a 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -6,7 +6,7 @@ IMAGES := freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES += ubuntu.i386 centos ifneq ($(EFI_AARCH64),) -IMAGES += ubuntu.aarch64 +IMAGES += ubuntu.aarch64 centos.aarch64 endif endif @@ -28,6 +28,7 @@ ifneq ($(GENISOIMAGE),) @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" ifneq ($(EFI_AARCH64),) @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" + @echo " vm-build-centos.aarch64 - Build QEMU in CentOS aarch64 VM" else @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch64)" endif diff --git a/tests/vm/centos-8-aarch64.ks b/tests/vm/centos-8-aarch64.ks new file mode 100644 index 00000000000..fd6ebe4d49e --- /dev/null +++ b/tests/vm/centos-8-aarch64.ks @@ -0,0 +1,51 @@ +# CentOS aarch64 image kickstart file. +# This file is used by the CentOS installer to +# script the generation of the image. +# +# Copyright 2020 Linaro +# +ignoredisk --only-use=vda +# System bootloader configuration +bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda +autopart --type=plain +# Partition clearing information +clearpart --linux --initlabel --drives=vda +# Use text mode install +text +repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream +# Use CDROM installation media +cdrom +# Keyboard layouts +keyboard --vckeymap=us --xlayouts='' +# System language +lang en_US.UTF-8 + +# Network information +network --bootproto=dhcp --device=enp0s1 --onboot=off --ipv6=auto --no-activate +network --hostname=localhost.localdomain +# Run the Setup Agent on first boot +firstboot --enable +# Do not configure the X Window System +skipx +# System services +services --enabled="chronyd" +# System timezone +timezone America/New_York --isUtc + +# Shutdown after installation is complete. +shutdown + +%packages +@^server-product-environment +kexec-tools + +%end + +%addon com_redhat_kdump --enable --reserve-mb='auto' + +%end +%anaconda +pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty +pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok +pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty +%end diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 new file mode 100755 index 00000000000..d5232ecdb8c --- /dev/null +++ b/tests/vm/centos.aarch64 @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +# +# Centos aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.aarch64 +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import os +import sys +import subprocess +import basevm +import time +import traceback +import aarch64vm + +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", + 'install_cmds' : "yum install -y make git python3 gcc gcc-c++ flex bison, "\ + "yum install -y glib2-devel pixman-devel zlib-devel, "\ + "yum install -y perl-Test-Harness, "\ + "alternatives --set python /usr/bin/python3, "\ + "sudo dnf config-manager "\ + "--add-repo=https://download.docker.com/linux/centos/docker-ce.repo,"\ + "sudo dnf install --nobest -y docker-ce.aarch64,"\ + "systemctl enable docker", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM. + 'ssh_timeout' : 60, +} + +class CentosAarch64VM(basevm.BaseVM): + name = "centos.aarch64" + arch = "aarch64" + login_prompt = "localhost login:" + prompt = '[root@localhost ~]#' + image_name = "CentOS-8-aarch64-1905-dvd1.iso" + image_link = "http://mirrors.usc.edu/pub/linux/distributions/centos/8.0.1905/isos/aarch64/" + image_link += image_name + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def set_key_perm(self): + """Set permissions properly on certain files to allow + ssh access.""" + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v /root/.ssh\n") + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v "\ + "/home/{}/.ssh\n".format(self._config["guest_user"])) + + def create_kickstart(self): + """Generate the kickstart file used to generate the centos image.""" + # Start with the template for the kickstart. + ks_file = "../tests/vm/centos-8-aarch64.ks" + subprocess.check_call("cp {} ./ks.cfg".format(ks_file), shell=True) + # Append the ssh keys to the kickstart file + # as the post processing phase of installation. + with open("ks.cfg", "a") as f: + # Add in the root pw and guest user. + rootpw = "rootpw --plaintext {}\n" + f.write(rootpw.format(self._config["root_pass"])) + add_user = "user --groups=wheel --name={} "\ + "--password={} --plaintext\n" + f.write(add_user.format(self._config["guest_user"], + self._config["guest_pass"])) + # Add the ssh keys. + f.write("%post --log=/root/ks-post.log\n") + f.write("mkdir -p /root/.ssh\n") + addkey = 'echo "{}" >> /root/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"]) + f.write(addkey_cmd) + f.write('mkdir -p /home/{}/.ssh\n'.format(self._config["guest_user"])) + addkey = 'echo "{}" >> /home/{}/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"], + self._config["guest_user"]) + f.write(addkey_cmd) + f.write("%end\n") + # Take our kickstart file and create an .iso from it. + # The .iso will be provided to qemu as we boot + # from the install dvd. + # Anaconda will recognize the label "OEMDRV" and will + # start the automated installation. + gen_iso_img = 'genisoimage -output ks.iso -volid "OEMDRV" ks.cfg' + subprocess.check_call(gen_iso_img, shell=True) + + def wait_for_shutdown(self): + """We wait for qemu to shutdown the VM and exit. + While this happens we display the console view + for easier debugging.""" + # The image creation is essentially done, + # so whether or not the wait is successful we want to + # wait for qemu to exit (the self.wait()) before we return. + try: + self.console_wait("reboot: Power down") + except Exception as e: + sys.stderr.write("Exception hit\n") + if isinstance(e, SystemExit) and e.code == 0: + return 0 + traceback.print_exc() + finally: + self.wait() + + def build_base_image(self, dest_img): + """Run through the centos installer to create + a base image with name dest_img.""" + # We create the temp image, and only rename + # to destination when we are done. + img = dest_img + ".tmp" + # Create an empty image. + # We will provide this as the install destination. + qemu_img_create = "qemu-img create {} 50G".format(img) + subprocess.check_call(qemu_img_create, shell=True) + + # Create our kickstart file to be fed to the installer. + self.create_kickstart() + # Boot the install dvd with the params as our ks.iso + os_img = self._download_with_cache(self.image_link) + dvd_iso = "centos-8-dvd.iso" + subprocess.check_call(["cp", "-f", os_img, dvd_iso]) + extra_args = "-cdrom ks.iso" + extra_args += " -drive file={},if=none,id=drive1,cache=writeback" + extra_args += " -device virtio-blk,drive=drive1,bootindex=1" + extra_args = extra_args.format(dvd_iso).split(" ") + self.boot(img, extra_args=extra_args) + self.console_wait_send("change the selection", "\n") + # We seem to need to hit esc (chr(27)) twice to abort the + # media check, which takes a long time. + # Waiting a bit seems to be more reliable before hitting esc. + self.console_wait("Checking") + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + print("Found Checking") + # Give sufficient time for the installer to create the image. + self.console_init(timeout=7200) + self.wait_for_shutdown() + os.rename(img, dest_img) + print("Done with base image build: {}".format(dest_img)) + + def check_create_base_img(self, img_base, img_dest): + """Create a base image using the installer. + We will use the base image if it exists. + This helps cut down on install time in case we + need to restart image creation, + since the base image creation can take a long time.""" + if not os.path.exists(img_base): + print("Generate new base image: {}".format(img_base)) + self.build_base_image(img_base); + else: + print("Use existing base image: {}".format(img_base)) + # Save a copy of the base image and copy it to dest. + # which we will use going forward. + subprocess.check_call(["cp", img_base, img_dest]) + + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(CentosAarch64VM, self).boot(img, extra_args=extra_args) + + def build_image(self, img): + img_tmp = img + ".tmp" + self.check_create_base_img(img + ".base", img_tmp) + + # Boot the new image for the first time to finish installation. + self.boot(img_tmp) + self.console_init() + self.console_wait_send(self.login_prompt, "root\n") + self.console_wait_send("Password:", + "{}\n".format(self._config["root_pass"])) + + self.set_key_perm() + self.console_wait_send(self.prompt, "rpm -q centos-release\n") + enable_adapter = "sed -i 's/ONBOOT=no/ONBOOT=yes/g'" \ + " /etc/sysconfig/network-scripts/ifcfg-enp0s1\n" + self.console_wait_send(self.prompt, enable_adapter) + self.console_wait_send(self.prompt, "ifup enp0s1\n") + self.console_wait_send(self.prompt, + 'echo "qemu ALL=(ALL) NOPASSWD:ALL" | '\ + 'sudo tee /etc/sudoers.d/qemu\n') + self.console_wait(self.prompt) + + # Rest of the commands we issue through ssh. + self.wait_ssh(wait_root=True) + + # If the user chooses *not* to do the second phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.ssh_root("poweroff") + self.wait_for_shutdown() + os.rename(img_tmp, img) + print("image creation complete: {}".format(img)) + return 0 + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(CentosAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(CentosAarch64VM, defaults)) From patchwork Wed Jun 24 14:04:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191663 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp631174ile; Wed, 24 Jun 2020 07:12:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyyemQ5ndzKGf+Kjqyg737Fd31qkdhbvuK5iDMg2QmC051nXFhW0S4Sq9hDiiAoOPA43UdP X-Received: by 2002:a25:a226:: with SMTP id b35mr44684605ybi.356.1593007954179; Wed, 24 Jun 2020 07:12:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007954; cv=none; d=google.com; s=arc-20160816; b=rEdmY0OVAkODzZkB8++Vewy4cinAilp5aSGOKYJR9AfCDkijIBSyD54OvsGSUEGLqE NfENvAqya6kyS0yn94stO0EVU4qfW8ZvwK2xnL5FKYM4E+UwxUKE3DEN2dLTjZK4p5XM JhuMALuZLyrclZmCpiHb3G/TwFJY13FNmONHSElBzOfWooP8ryEsGREscgKC5Xig2mHE AqZQgVsXFwSWLrkqgnqL95HkuOMnGkDPJRCPKmyUoyVuwvGOUq2WUMClm8836dlEcaUC hrDFmHZt+MwB+1nrNtRuvRVSOvKY2zh/a2u4LEylMbQHpRZNv0JJNIdYAxDa1zG5KGYx 2NdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=Xc7GdEfBo7+E7MBDQUQ92Wy/kl7FzCtRm8bY7aKoa4yZIP1GXSHic+rgnyoURBu2Kl E9Ej5BPx8iTeu6GKP1D7vmFqKTXM58ak71IUV4E4ryxIzWhSHz3RYcoXp3TMojDveaiU ZGuY71AIxqIEjMAUDWL/ndHjvbv/229enD83Gh0paWN6xi9gh+DsKT8txatTt0X98Hw7 onh3oE4rjebJCsGogjz8AXZBlPJr1o7Vs72Q0o3KA94zjWUaNh2MiDgly3c90v/miEKG Cxm9T4fF5Jy57pY8IgPZfoWySDF4ME/ALpDxSvZtPPpwphx78tqOb0sbDlOJpJP7dma2 jVXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GaytSj8f; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u9si19404749ybu.422.2020.06.24.07.12.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:12:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GaytSj8f; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34390 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo68f-0003lO-ME for patch@linaro.org; Wed, 24 Jun 2020 10:12:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61S-00054f-Dd for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:35518) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61O-0003W7-Gr for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: by mail-wm1-x344.google.com with SMTP id l2so1173633wmf.0 for ; Wed, 24 Jun 2020 07:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=GaytSj8f3EocCFCEFEosAnZX2HysoeIThJ3E1560NouuX2QkT/59j+q2g/YsfNlZ5b nzyJ7yJiao3+8aNNqVOKmP6aKIp9Ni8aVPLWKzkDRATfZUxwilKtNjFMX0cJNDIDsvw2 njzv7z8N/f8rCNqlnupKX/WyAheC2Qw9TRxrqpvdMw7UTOLxo3UO/Uw40cTomTwXvbKE vCMOx6crgo73BKPxj19mJ1cQ32YQagGn33JwXPVsYgPpbaj5OqMDmfkx0W9rCrn9X0rN HSWAgbc8qZoNBQq8ofSbzx4EaDhIK3YIOElbbuzNrRfUWLIN6BNyJiJQh5zTAUioUhql ug+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qDw5tmNjtaEA2MrxzXTWX9WRb/d3hDPxln94PjTXX2k=; b=kPAkK9XL2ZgZ4As3UoRAW4aaXmyoxVxp2MWbdahc1H+jVrwi8oSxuFhLxHI+bC3KQV 3UnoQt6CVl1RGYRNdnvWUTn9RI+7clCy/3X0f4+BggEc1Bq/NBm+Ms6Ng4e9tophOA+V r0BT0PkGhKLzDmKzk1nb21v+5FX+n9HT0W5CW2aiGaRL2D+EEUOuxTaBUTX9+zsvT8sl GbaTeP6nGuFQkvXGxEU6k9VtiPCGUn5qsOOBMfwegXVUkn1YqMO0sp2q9UyF8Niggq3R banWJt6kx0aAq6miEleBXFBrht7fJWA0JpUS4I/hx0VSrl5d2dUhcUOajh77u/UzPSBY ureA== X-Gm-Message-State: AOAM531i4Ja/qPBokHU25V/3B0So5wPVfFUugxMg56aEOdUv8N3RVDXH aCQ43n3KKFNTq3CjYzXqDhTgQA== X-Received: by 2002:a1c:2146:: with SMTP id h67mr27970796wmh.46.1593007500867; Wed, 24 Jun 2020 07:05:00 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id v7sm27540685wrp.45.2020.06.24.07.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:54 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 23DC41FF98; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 09/25] tests/vm: change scripts to use self._config Date: Wed, 24 Jun 2020 15:04:30 +0100 Message-Id: <20200624140446.15380-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::344; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x344.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-8-robert.foley@linaro.org> --- tests/vm/basevm.py | 11 ++--------- tests/vm/fedora | 17 +++++++++-------- tests/vm/freebsd | 16 ++++++++-------- tests/vm/netbsd | 19 ++++++++++--------- tests/vm/openbsd | 17 +++++++++-------- 5 files changed, 38 insertions(+), 42 deletions(-) -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5fd66f6b26a..f716798b405 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -183,13 +183,6 @@ class BaseVM(object): self.console_init(timeout=timeout) self.console_wait(wait_string) - def __getattr__(self, name): - # Support direct access to config by key. - # for example, access self._config['cpu'] by self.cpu - if name.lower() in self._config.keys(): - return self._config[name.lower()] - return object.__getattribute__(self, name) - def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -239,13 +232,13 @@ class BaseVM(object): return r def ssh(self, *cmd): - return self._ssh_do(self.GUEST_USER, cmd, False) + return self._ssh_do(self._config["guest_user"], cmd, False) def ssh_root(self, *cmd): return self._ssh_do("root", cmd, False) def ssh_check(self, *cmd): - self._ssh_do(self.GUEST_USER, cmd, True) + self._ssh_do(self._config["guest_user"], cmd, True) def ssh_root_check(self, *cmd): self._ssh_do("root", cmd, True) diff --git a/tests/vm/fedora b/tests/vm/fedora index a9195670f4b..b2b478fdbca 100755 --- a/tests/vm/fedora +++ b/tests/vm/fedora @@ -108,20 +108,20 @@ class FedoraVM(basevm.BaseVM): self.console_wait_send("7) [!] Root password", "7\n") self.console_wait("Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("8) [ ] User creation", "8\n") self.console_wait_send("1) [ ] Create user", "1\n") self.console_wait_send("3) User name", "3\n") - self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER) + self.console_wait_send("ENTER:", "%s\n" % self._config["guest_user"]) self.console_wait_send("4) [ ] Use password", "4\n") self.console_wait_send("5) Password", "5\n") self.console_wait("Password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("7) Groups", "c\n") while True: @@ -139,7 +139,7 @@ class FedoraVM(basevm.BaseVM): if good: break time.sleep(10) - self.console_send("r\n" % self.GUEST_PASS) + self.console_send("r\n" % self._config["guest_pass"]) self.console_wait_send("'b' to begin install", "b\n") @@ -150,12 +150,13 @@ class FedoraVM(basevm.BaseVM): # setup qemu user prompt = " ~]$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = " ~]#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/freebsd b/tests/vm/freebsd index f87db2b126e..29252fa4a64 100755 --- a/tests/vm/freebsd +++ b/tests/vm/freebsd @@ -113,9 +113,9 @@ class FreeBSDVM(basevm.BaseVM): # post-install configuration self.console_wait("New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Network Configuration", "\n") self.console_wait_send("IPv4", "y") @@ -134,9 +134,9 @@ class FreeBSDVM(basevm.BaseVM): # qemu user self.console_wait_send("Add User Accounts", "y") self.console_wait("Username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait_send("Uid", "\n") self.console_wait_send("Login group", "\n") self.console_wait_send("Login group", "\n") @@ -148,9 +148,9 @@ class FreeBSDVM(basevm.BaseVM): self.console_wait_send("Use an empty password", "\n") self.console_wait_send("Use a random password", "\n") self.console_wait("Enter password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Enter password again:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Lock out", "\n") self.console_wait_send("OK", "yes\n") self.console_wait_send("Add another user", "no\n") @@ -164,12 +164,12 @@ class FreeBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "root@freebsd:~ #" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup serial console diff --git a/tests/vm/netbsd b/tests/vm/netbsd index cdac502dad8..dc0e45c1d45 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -120,24 +120,24 @@ class NetBSDVM(basevm.BaseVM): self.console_wait_send("d: Change root password", "d\n") self.console_wait_send("a: Yes", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("o: Add a user", "o\n") self.console_wait("username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("to group wheel") self.console_wait_send("a: Yes", "a\n") self.console_wait_send("a: /bin/sh", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("a: Configure network", "a\n") self.console_wait_send("a: vioif0", "a\n") @@ -170,12 +170,13 @@ class NetBSDVM(basevm.BaseVM): # setup qemu user prompt = "localhost$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "localhost#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/openbsd b/tests/vm/openbsd index 13e7f9a6d56..dfe633e4532 100755 --- a/tests/vm/openbsd +++ b/tests/vm/openbsd @@ -98,9 +98,9 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which network interface", "done\n") self.console_wait_send("DNS domain name", "localnet\n") self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Start sshd(8)", "yes\n") self.console_wait_send("X Window System", "\n") self.console_wait_send("xenodm", "\n") @@ -108,13 +108,13 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which speed", "\n") self.console_wait("Setup a user") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Allow root ssh login", "yes\n") self.console_wait_send("timezone", "UTC\n") @@ -135,12 +135,13 @@ class OpenBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "openbsd#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) From patchwork Wed Jun 24 14:04:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191658 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp627183ile; Wed, 24 Jun 2020 07:08:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyy0Ybv2KlPT/v8rMf6mvG6PsP6vceu/XFxLLNoxXDKkZ/7t9hu+Cu4pSpYdlM6WoMrMhTj X-Received: by 2002:a25:1941:: with SMTP id 62mr49583507ybz.16.1593007693928; Wed, 24 Jun 2020 07:08:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007693; cv=none; d=google.com; s=arc-20160816; b=qj3M3rMYzEhMPOAS1i3ZnjOYn/A2DqrHk5Dbvuy8r8q+uhFRIliyFxEjWKLopyK/eF y8+n6ObAkY6f8gYTOQ4+Cl4YyZj0Tf3nLftxEwVEHr9PXTK5aUYIdTkB2tduGAXvONhg sKdqLy2PdFQhWoTLraVoDIlt/ndH9V6wfKxtKF1cSMsBWj7/b/I2bbgacaSeXoUKEg6v gTApg7S1EUCKq2qJdzQVTpVXg9HQgRsU2dcfaq0p3H+RIs7QlA98IxNQAaMXR5mHoM3y 9J+1ACgnNb6fYvfqWEMBeFMuqdTUFuqU+Nnl9FL6FLPTjz+iDtVsYy3jIsJurtHXi0TM +KZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=v2gSlYFdXDr2WqakgrxBrLIzWF10emFmsy4YATKgpMxHawahvlNuKwogRpxuRh2UZd Sm9DqllmmfCgO5bsC1kcKSbloIXRtVAGImErXGh4Ic9GRhAkzY1eqyIB208s8kjAblh4 pjKEKQ8SZodSiF29ZQB7oR0Ks3fHfaTzBGrxGCQEH9ASvLHoM1VJ62KbuAC6Uulqo03P iuEnYscAsnQ9ieu0V/o2Cu1m99biqHQU1zN2kHMUULRcgM3VXsohzJl3IoQap/i2Mq9k GR0Qvbl80jhGFrY7oMrcQ0CV57ytp5bAZNzMHUAXriqBm1Blo6fYCyxHHyh8+GvaDvGS rnHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UM4qUG57; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o77si19080454ybg.332.2020.06.24.07.08.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:08:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UM4qUG57; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo64T-0003kh-BE for patch@linaro.org; Wed, 24 Jun 2020 10:08:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61S-00053x-5y for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:06 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:37766) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61P-0003WQ-Nd for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:05 -0400 Received: by mail-wr1-x42b.google.com with SMTP id a6so2429962wrm.4 for ; Wed, 24 Jun 2020 07:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=UM4qUG57JiusQsCdTNT8H0O6h2xjNdxLbCahDXyz5GfqlW4t7ReyWdji7EEWjMDo2l JR6S0lsFcuD6zwzY/wSFXTZd6IAHZ+EXSNLHz/KVeF+D+fEWuuuv86dCB3BA0p0tNA91 42uNASYp6e9XvHeX2Pm+TI6o6/Tln3l0jvL+hxb8fAvfI8qG+fUXrjhUq/BsCpoYUomW dcVKJDfQ5lmPDXy80Xxa6CRTVDcINa+TTrHlYMImtAJjWzH6yIq4c2TsdyW3iom+PFFk 17bgQWv2UTDyIOXuH+EBzZWQY9C+heeAqfEWh6ViiauyMhe7Hv4Ol5QGufxabB4v/EqH +78w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SfCKKjXRpSzktD2zHuCdVYnzBzWod3odSIiX8dUA3RA=; b=JMrU5Az78C33kU6jkvSKtgEy8ciRBzvfHQ0YXEmM4MRtLfIy7aQumwKOhuRw0KPq0D uAQE2N+lStRH5hCliWOGpz0qrgPl21dGlDMJMCr4kJWipb/Xjw6phFMFTxhZPSL4bcPd 37HQ66VMM3DKe1e1CDsjiXEu5VW+bumhUxnUhJ7UmjkBdDTywTLAifZ83iFRSh8b1GB4 rpE89ygZvJC3QZ4oMNvmUFkznNJBTT1ee9fPXZT3ONvEgcfKyBQnkrQkRoEAcbB+MtIp iuEmZDNQ2iiOdduZ118gpLDiadUOny4nAhFvn26GJQkLWIXERLU4T12Sw7lNn5VpTKl4 OVdw== X-Gm-Message-State: AOAM531CHdJqY2olSqMwGofnLMVVCcf3Ta7bG2Eo/fR6QlRO7lybsmdg IDNpmQkcghRuGzjl8vgp6fqkxw== X-Received: by 2002:a5d:6052:: with SMTP id j18mr30206248wrt.387.1593007502201; Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id l132sm7688044wmf.6.2020.06.24.07.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:54 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 3DE791FF99; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 10/25] python/qemu: Add ConsoleSocket for optional use in QEMUMachine Date: Wed, 24 Jun 2020 15:04:31 +0100 Message-Id: <20200624140446.15380-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Cleber Rosa , Peter Puhov , aurelien@aurel32.net, Eduardo Habkost Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley We add the ConsoleSocket object, which has a socket interface and which will consume all arriving characters on the socket, placing them into an in memory buffer. This will also provide those chars via recv() as would a regular socket. ConsoleSocket also has the option of dumping the console bytes to a log file. We also give QEMUMachine the option of using ConsoleSocket to drain and to use for logging console to a file. By default QEMUMachine does not use ConsoleSocket. This is added in preparation for use by basevm.py in a later commit. This is a workaround we found was needed for basevm.py since there is a known issue where QEMU will hang waiting for console characters to be consumed. Cc: Eduardo Habkost Cc: Cleber Rosa Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-9-robert.foley@linaro.org> --- python/qemu/console_socket.py | 110 ++++++++++++++++++++++++++++++++++ python/qemu/machine.py | 23 +++++-- 2 files changed, 129 insertions(+), 4 deletions(-) create mode 100644 python/qemu/console_socket.py -- 2.20.1 diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py new file mode 100644 index 00000000000..830cb7c6282 --- /dev/null +++ b/python/qemu/console_socket.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +# +# This python module implements a ConsoleSocket object which is +# designed always drain the socket itself, and place +# the bytes into a in memory buffer for later processing. +# +# Optionally a file path can be passed in and we will also +# dump the characters to this file for debug. +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import asyncore +import socket +import threading +import io +import os +import sys +from collections import deque +import time +import traceback + +class ConsoleSocket(asyncore.dispatcher): + + def __init__(self, address, file=None): + self._recv_timeout_sec = 300 + self._buffer = deque() + self._asyncore_thread = None + self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self._sock.connect(address) + self._logfile = None + if file: + self._logfile = open(file, "w") + asyncore.dispatcher.__init__(self, sock=self._sock) + self._open = True + self._thread_start() + + def _thread_start(self): + """Kick off a thread to wait on the asyncore.loop""" + if self._asyncore_thread is not None: + return + self._asyncore_thread = threading.Thread(target=asyncore.loop, + kwargs={'timeout':1}) + self._asyncore_thread.daemon = True + self._asyncore_thread.start() + + def handle_close(self): + """redirect close to base class""" + # Call the base class close, but not self.close() since + # handle_close() occurs in the context of the thread which + # self.close() attempts to join. + asyncore.dispatcher.close(self) + + def close(self): + """Close the base object and wait for the thread to terminate""" + if self._open: + self._open = False + asyncore.dispatcher.close(self) + if self._asyncore_thread is not None: + thread, self._asyncore_thread = self._asyncore_thread, None + thread.join() + if self._logfile: + self._logfile.close() + self._logfile = None + + def handle_read(self): + """process arriving characters into in memory _buffer""" + try: + data = asyncore.dispatcher.recv(self, 1) + # latin1 is needed since there are some chars + # we are receiving that cannot be encoded to utf-8 + # such as 0xe2, 0x80, 0xA6. + string = data.decode("latin1") + except: + print("Exception seen.") + traceback.print_exc() + return + if self._logfile: + self._logfile.write("{}".format(string)) + self._logfile.flush() + for c in string: + self._buffer.extend(c) + + def recv(self, n=1, sleep_delay_s=0.1): + """Return chars from in memory buffer""" + start_time = time.time() + while len(self._buffer) < n: + time.sleep(sleep_delay_s) + elapsed_sec = time.time() - start_time + if elapsed_sec > self._recv_timeout_sec: + raise socket.timeout + chars = ''.join([self._buffer.popleft() for i in range(n)]) + # We choose to use latin1 to remain consistent with + # handle_read() and give back the same data as the user would + # receive if they were reading directly from the + # socket w/o our intervention. + return chars.encode("latin1") + + def set_blocking(self): + """Maintain compatibility with socket API""" + pass + + def settimeout(self, seconds): + """Set current timeout on recv""" + self._recv_timeout_sec = seconds diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 041c615052e..c25f0b42cf6 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,7 @@ import socket import tempfile from typing import Optional, Type from types import TracebackType +from qemu.console_socket import ConsoleSocket from . import qmp @@ -75,7 +76,8 @@ class QEMUMachine: def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None, sock_dir=None): + socket_scm_helper=None, sock_dir=None, + drain_console=False, console_log=None): ''' Initialize a QEMUMachine @@ -86,6 +88,9 @@ class QEMUMachine: @param test_dir: where to create socket and log file @param monitor_address: address for QMP monitor @param socket_scm_helper: helper program, required for send_fd_scm() + @param sock_dir: where to create socket (overrides test_dir for sock) + @param console_log: (optional) path to console log file + @param drain_console: (optional) True to drain console socket to buffer @note: Qemu process is not started until launch() is used. ''' if args is None: @@ -122,6 +127,12 @@ class QEMUMachine: self._console_address = None self._console_socket = None self._remove_files = [] + self._console_log_path = console_log + if self._console_log_path: + # In order to log the console, buffering needs to be enabled. + self._drain_console = True + else: + self._drain_console = drain_console def __enter__(self): return self @@ -580,7 +591,11 @@ class QEMUMachine: Returns a socket connected to the console """ if self._console_socket is None: - self._console_socket = socket.socket(socket.AF_UNIX, - socket.SOCK_STREAM) - self._console_socket.connect(self._console_address) + if self._drain_console: + self._console_socket = ConsoleSocket(self._console_address, + file=self._console_log_path) + else: + self._console_socket = socket.socket(socket.AF_UNIX, + socket.SOCK_STREAM) + self._console_socket.connect(self._console_address) return self._console_socket From patchwork Wed Jun 24 14:04:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191660 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp627269ile; Wed, 24 Jun 2020 07:08:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9qBOcEkLBBDvVesLGONVw8GUTMVuYY80Sebcjm45YjicxrGM6lMG0w0QwVbCHP8gwmIZz X-Received: by 2002:a5b:b49:: with SMTP id b9mr40608553ybr.343.1593007699539; Wed, 24 Jun 2020 07:08:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007699; cv=none; d=google.com; s=arc-20160816; b=RGVD55yM7z+p965GVzXagLHQEiv+3fisvCdfMNII/VWcFJ/xedi3/DOa/n5qO86Ckc qZBBDY4O4skOLH70RUcxMpzlkL3nOHdNf6JDFNIFnFf6kfhyQNmTZU6ECcRDQwS+S0qO 9+L6ELEmQQ4qyx9UkjRkKtAu/NmY98MgTJFj4KwQ8Ahegk5ZtFfkY4Q9QWyLIt7JAQkk SvhG0qrqel6rKFTZn9YMZ1TofMObOwkInbY0gp2xXwoVqdH/bIFfip7VduojRjRCV0YE d5hDR5E+hcu414cxVCKDh9LXeqi3S8JzpplDas2Pplkhif00CsfZJbiHAKLxgM5whvRF vo6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=IrE/iIL+9eieVv9wfV+hC+M9ATaVnxfpa0YjmFlsxm35u5jbG06FqYCd+ws5VVmxNm Eu206NoCozXuGmbWU9tEQnmoLJnOA65nGKrvczv7AOvTkQTp6xK2hRyA4fFJdNXmYloz hg2tVxrfP6nno7PYXYhg93RLfIbHzq6SXgl+RR+xuQq46rcMNVqNomXsQiIOzZ9Azivf yWJ7F6czTrCyuRXe0RWuhrdf2hXfiwsFqb1utVLVMfixWaL8G5v9hzYWERDb06Rb1AAN GH3j9BRUA/J6l1sSWotps/R2GJYODM01UTQnnEqQgeZUNZFBdruHSGhT9KX5eUPvytBG D9qA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="stLkwgi/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e70si19609381yba.125.2020.06.24.07.08.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:08:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="stLkwgi/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo64Y-0003yN-UW for patch@linaro.org; Wed, 24 Jun 2020 10:08:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61N-0004vS-8D for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:01 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:55384) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61K-0003VM-9j for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:00 -0400 Received: by mail-wm1-x343.google.com with SMTP id g75so2435444wme.5 for ; Wed, 24 Jun 2020 07:04:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=stLkwgi/ueBYCoZNMDFasbFrfEB3RKeBudIM3YKlcDrgHLPWVWK5ouOMmNfFSV48ks BtxR1AZMtI+btE5adAcEOuOi0b8DTaTc77DS1QCuLlPU4Wd2Zs82XGRynnbNdC5pnK5w XXLmc/W+8VhXcPlsGKu+l6cyNVDFqe/py6JwzEkrVQKo8IUup2p4/0mACMFtZldUnj1r 9uCKIXIe8WldmyPrQ84vipeC5PIOf1HusiRaC7r5ZZSfdbhj9KGbJ5DY8RgV9fRK3va9 /fmx3iQRRHevyNzzkkixTIyKqwgAUOkhzMjmzwu7VL5llfOu7VW7Y9NHvXJ6sRHYUVyk OKlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5xSn2L6kzBKcKjTR8zPwxbT757SBhdGuGZ/3k/jBdz8=; b=GgXD8M8fALajjj2Z7Jh1UMqy0ceRVlqb6Akqr5R5+EoxGlFmlIXMa9X2fHljYFkpCn 132ezZodOEzFqgYdGg18azhBtitv02UzggXQ6mHnYqY9S9BLFqfq5DXRkSfm6JT3dhUA nhuysq+6WBF/i46JcPwJfsJHmLJnEBlLNO+6N1OJ4S25jN1LauziHXfHgOM+HF7+3hTo bRA35SN5D3CThV9zuQYT//fnFuXafUsQkxqTB6lsMFvSG8exURjPZqiD8ZRnAKNgGdC5 Y9x3RjrjaF8b6pCR0c/+OXoVIDu7kWPIWDv5tiF6WpEbisRK547vtM7bbFqT9VOQZZd1 X4zA== X-Gm-Message-State: AOAM532wUJLQ0kaKDh1+Ib/FVu5+gpH9dVwOKmMh+Jn+jit0ohIimGAa 7Xt1GWO8DOWptOPi+khJyqAhcw== X-Received: by 2002:a1c:e209:: with SMTP id z9mr5757339wmg.153.1593007496890; Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n5sm3827122wmi.34.2020.06.24.07.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 568931FF9A; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 11/25] tests/vm: Add workaround to consume console Date: Wed, 24 Jun 2020 15:04:32 +0100 Message-Id: <20200624140446.15380-12-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::343; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x343.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, Peter Puhov , aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley This adds support to basevm.py so that we always drain the console chars. This makes use of support added in an earlier commit that allows QEMUMachine to use the ConsoleSocket. This is a workaround we found was needed since there is a known issue where QEMU will hang waiting for console characters to be consumed. We also added the option of logging the console to a file. LOG_CONSOLE=1 will now log the output to a file. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-10-robert.foley@linaro.org> --- tests/vm/Makefile.include | 4 ++++ tests/vm/basevm.py | 21 +++++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 39f918a430a..f21948c46a5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -49,6 +49,7 @@ endif @echo ' EXTRA_CONFIGURE_OPTS="..."' @echo " J=[0..9]* - Override the -jN parameter for make commands" @echo " DEBUG=1 - Enable verbose output on host and interactive debugging" + @echo " LOG_CONSOLE=1 - Log console to file in: ~/.cache/qemu-vm " @echo " V=1 - Enable verbose ouput on host and guest commands" @echo " QEMU_LOCAL=1 - Use QEMU binary local to this build." @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @@ -75,6 +76,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$@" \ --force \ --build-image $@, \ @@ -91,6 +93,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -114,6 +117,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index f716798b405..04d083409a5 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -117,6 +117,11 @@ class BaseVM(object): "w").write(self._config['ssh_pub_key']) self.debug = args.debug + self._console_log_path = None + if args.log_console: + self._console_log_path = \ + os.path.join(os.path.expanduser("~/.cache/qemu-vm"), + "{}.install.log".format(self.name)) self._stderr = sys.stderr self._devnull = open(os.devnull, "w") if self.debug: @@ -271,7 +276,13 @@ class BaseVM(object): args += self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path = get_qemu_path(self.arch, self._build_path) - guest = QEMUMachine(binary=qemu_path, args=args) + + # Since console_log_path is only set when the user provides the + # log_console option, we will set drain_console=True so the + # console is always drained. + guest = QEMUMachine(binary=qemu_path, args=args, + console_log=self._console_log_path, + drain_console=True) guest.set_machine(self._config['machine']) guest.set_console() try: @@ -285,6 +296,8 @@ class BaseVM(object): raise atexit.register(self.shutdown) self._guest = guest + # Init console so we can start consuming the chars. + self.console_init() usernet_info = guest.qmp("human-monitor-command", command_line="info usernet") self.ssh_port = None @@ -296,7 +309,9 @@ class BaseVM(object): raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \ usernet_info) - def console_init(self, timeout = 120): + def console_init(self, timeout = None): + if timeout == None: + timeout = self.socket_timeout vm = self._guest vm.console_socket.settimeout(timeout) self.console_raw_path = os.path.join(vm._temp_dir, @@ -578,6 +593,8 @@ def parse_args(vmcls): parser.add_option("--efi-aarch64", default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", help="Path to efi image for aarch64 VMs.") + parser.add_option("--log-console", action="store_true", + help="Log console to file.") parser.disable_interspersed_args() return parser.parse_args() From patchwork Wed Jun 24 14:04:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191657 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp626319ile; Wed, 24 Jun 2020 07:07:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxPqxqkl9OI1xHOVtdB6/0jQRRJHNHj3otJNaXEFp38jm4lc3yq3qT+zvxWoIeq4gm/i8Uq X-Received: by 2002:ab0:26c5:: with SMTP id b5mr2374023uap.51.1593007643083; Wed, 24 Jun 2020 07:07:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007643; cv=none; d=google.com; s=arc-20160816; b=k9tXAjFNaCGAdZg9rnOghBaxa3IgvHKW1/yHQ1HrXmKa3YJDQHh5SxfLyDL9hX+lhy byyoicu08Unpi9Bmb8hK7jNCAFQDXnva6crX4AxcoYUA6sXkKRGhe8ZxOWHdhHNA0AYy FJueO00uBqIV4cH0y8J61x7SIe7fNGHcYKHWir2lZU2bjNplHTY/grI5PRvbgJJy6Mza HH59Dv8ZOVIsz1jFQJJPl+oh4XMvh+jahTI/d+VCSt1tGst7JMKtB/HKJAjS3FbuIxoS OdC1i5Iz/y1AgLr7WvXImYDrJTcc9IHxRalCr+In9Bza84OBiLGdr+jopk58pwQoRHrB 7IpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=j/rflEmqRJN7WGfuneOrxHdfepOnh/k6dfOKVG8ho+IpDF5gcbSHwECuMr+TEtD6eS 37ZpNMJvRX6QujHlxOLg/Cqq7Zw+1zvVhQujIeYeOWbchi/9k8gVlGaoWwQMz6rkOpn1 HUEPE3ZA8boTT0jpC7nFS7TfpzpjRxo+u4Nyt8nDC9KF0lVpD3KNQzGcgJFt0cbSK1FQ R3mPgEHLYVukP2D1y+x3+R6Xhgxopkjjf4w+IpOV+LwJZJBKP2p1yjRIoHB9ioUgFDti u8caC8tnYEDGw6TxKs2+UrNA0npxNH5uAYD5DYYkghY0jyOLPlL72VjARVXqI1sftKTy u31w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=amljOvBK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n9si2522973vsq.410.2020.06.24.07.07.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:07:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=amljOvBK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo63e-0000Wa-CO for patch@linaro.org; Wed, 24 Jun 2020 10:07:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59290) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61U-000582-2o for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:08 -0400 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:46114) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61R-0003Wo-RO for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:07 -0400 Received: by mail-wr1-x443.google.com with SMTP id r12so2377264wrj.13 for ; Wed, 24 Jun 2020 07:05:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=amljOvBKYb19XKL33ec5BrEZxXJlWJo9bRCzIKec77yGf0buKkslUQThM9fO91t4iF 43aL3IhhX9QgVp+Y4RBLFKpfTGeTD04zBiKemPeRlD4YFBMkHx7wVEjDuLG35La+Lip1 74DKs6TNAbU61X2AffvS2oDQHuFw7Va3u5533t1s9osOJa+lCHopeKbenxba0swJAOjj piv6LSGE4O8UrMucTkr8X98WljFugesqdV0q5YoNAShRdO9QH3H2qk4ytoFLTX3e4mNE alZoz6+Rc36sOCllsLQCh1e01vLkJ1U3wLn5/ygxL/oV3gjLSQh4rH0niuiVaa0P+WSJ qB9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MIJDY0wCVkf83j0A2folG6JUn+sBiPxS+WIpYPexzXA=; b=uR4AwNJDauVS9sMd6Vd9XkTrJENi3j+Amu/aEIgNiiP6Vuvpnbp9icyN9rG+aXyFgj RZH74if1COyJEXQmYAWvX/3UN6LuHU2qmL3fYA7JSnpGysakPOt75oHiM8yGEjnjR8Fr p1si31o/jEps9rH93XYOu+g9qGnpkHgsysJmRGq7ZetLuDdLpX0ltyvTzPD+bJ1Pi6xT pQ/qq6ybVpLDZI/0vjf17XKqKs70Ls5ShKuN75GbQ3SfUOrjqvCeHthcrnEQI0BBT7c6 EpZeVZb4ZQiDI//IJHdu+4ejarukrru6rcu/+leK/ey56GYAoER7y4ZKqnopT932PYaR 897g== X-Gm-Message-State: AOAM5335SdUC7Lhyc8eImoD8lG1SFrQlHkf+ER2QzrVQIEMjcOAy9+Vd aGE5IZOgcaRFKOwtW7c3AAn4bw== X-Received: by 2002:adf:9404:: with SMTP id 4mr30595963wrq.367.1593007504234; Wed, 24 Jun 2020 07:05:04 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n5sm3827123wmi.34.2020.06.24.07.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:04:56 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 6D3061FF9B; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 12/25] tests/vm: switch from optsparse to argparse Date: Wed, 24 Jun 2020 15:04:33 +0100 Message-Id: <20200624140446.15380-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::443; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" optparse has been deprecated since version 3.2 and argparse is the blessed replacement. Take the opportunity to enhance our help output showing defaults when called. Signed-off-by: Alex Bennée Reviewed-by: Daniel P. Berrangé Reviewed-by: Robert Foley --- tests/vm/basevm.py | 84 +++++++++++++++++++++++----------------------- 1 file changed, 42 insertions(+), 42 deletions(-) -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 04d083409a5..93859362606 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -23,7 +23,7 @@ from qemu.accel import kvm_available from qemu.machine import QEMUMachine import subprocess import hashlib -import optparse +import argparse import atexit import tempfile import shutil @@ -556,47 +556,47 @@ def parse_args(vmcls): else: return 1 - parser = optparse.OptionParser( - description="VM test utility. Exit codes: " - "0 = success, " - "1 = command line error, " - "2 = environment initialization failed, " - "3 = test command failed") - parser.add_option("--debug", "-D", action="store_true", - help="enable debug output") - parser.add_option("--image", "-i", default="%s.img" % vmcls.name, - help="image file name") - parser.add_option("--force", "-f", action="store_true", - help="force build image even if image exists") - parser.add_option("--jobs", type=int, default=get_default_jobs(), - help="number of virtual CPUs") - parser.add_option("--verbose", "-V", action="store_true", - help="Pass V=1 to builds within the guest") - parser.add_option("--build-image", "-b", action="store_true", - help="build image") - parser.add_option("--build-qemu", - help="build QEMU from source in guest") - parser.add_option("--build-target", - help="QEMU build target", default="check") - parser.add_option("--build-path", default=None, - help="Path of build directory, "\ - "for using build tree QEMU binary. ") - parser.add_option("--interactive", "-I", action="store_true", - help="Interactively run command") - parser.add_option("--snapshot", "-s", action="store_true", - help="run tests with a snapshot") - parser.add_option("--genisoimage", default="genisoimage", - help="iso imaging tool") - parser.add_option("--config", "-c", default=None, - help="Provide config yaml for configuration. "\ - "See config_example.yaml for example.") - parser.add_option("--efi-aarch64", - default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", - help="Path to efi image for aarch64 VMs.") - parser.add_option("--log-console", action="store_true", - help="Log console to file.") - parser.disable_interspersed_args() - return parser.parse_args() + parser = argparse.ArgumentParser( + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + description="Utility for provisioning VMs and running builds", + epilog="""Remaining arguments are passed to the command. + Exit codes: 0 = success, 1 = command line error, + 2 = environment initialization failed, + 3 = test command failed""") + parser.add_argument("--debug", "-D", action="store_true", + help="enable debug output") + parser.add_argument("--image", "-i", default="%s.img" % vmcls.name, + help="image file name") + parser.add_argument("--force", "-f", action="store_true", + help="force build image even if image exists") + parser.add_argument("--jobs", type=int, default=get_default_jobs(), + help="number of virtual CPUs") + parser.add_argument("--verbose", "-V", action="store_true", + help="Pass V=1 to builds within the guest") + parser.add_argument("--build-image", "-b", action="store_true", + help="build image") + parser.add_argument("--build-qemu", + help="build QEMU from source in guest") + parser.add_argument("--build-target", + help="QEMU build target", default="check") + parser.add_argument("--build-path", default=None, + help="Path of build directory, "\ + "for using build tree QEMU binary. ") + parser.add_argument("--interactive", "-I", action="store_true", + help="Interactively run command") + parser.add_argument("--snapshot", "-s", action="store_true", + help="run tests with a snapshot") + parser.add_argument("--genisoimage", default="genisoimage", + help="iso imaging tool") + parser.add_argument("--config", "-c", default=None, + help="Provide config yaml for configuration. "\ + "See config_example.yaml for example.") + parser.add_argument("--efi-aarch64", + default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help="Path to efi image for aarch64 VMs.") + parser.add_argument("--log-console", action="store_true", + help="Log console to file.") + return parser.parse_known_args() def main(vmcls, config=None): try: From patchwork Wed Jun 24 14:04:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191675 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp637589ile; Wed, 24 Jun 2020 07:20:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwm6Ww5ZcouI/bbCIIHcHJM0dlRDa+CsQ7/wTh0viWx5W+eAJIBsu/3tQc061Kn+SsH7JVv X-Received: by 2002:a25:f20d:: with SMTP id i13mr28814696ybe.366.1593008414123; Wed, 24 Jun 2020 07:20:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008414; cv=none; d=google.com; s=arc-20160816; b=SzsySi62mdU6ptntJFgR4zgOmfqDCslZrA3Hvy2LszJBX1a1kDT/iX1DERf2I8t2QA FWdDUWxuTJ1duuuRyByiIhYIhbPBlvukJmRosJXXEoap/4UdDh1tiqq4ORYBhN1ex3DK K51IA4B5sHjkjD5y8/WpFkN+dQh27mDGr8EIVpY+Btha6TQ57hz4OwQscooKAR1FhzKf Ftofva56p0+ND9IUjBs8nEz+U10KSm1bZ/vuqCMipa/cI1Og4i26hb/j7880FVF0rrXP 0j8NnyhtBbMAkRlBb1ydu81cXyjfRso4PIfgHN1gxqa0DXOBm7drVqv3zZIQ8krVTpIJ 0saA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=YZxxKD54wDI9eURktWofae6m+ZpFAeEQo9ktV5/x5sEhaGsl19RQwTXFpZ7VF3khnx IFYVk8fP+utt1TRy1s/vFuF0onNLW3nzyL9c6xNp7j18ql3XdRFwe+Rsq1yT24rAwB+J HN/JZgNCokBDHpUM58SKstphR+KRN1iZCrNGHyR8hbYdejB1/gBj3v4a8dD3KDL3+FNf MZKc1bJiybT3f5B9R+vYIyH/fgm+slkpTfQxLky++3egHk3liMC17WUMHtEv+ZmcAq1Q 594RAK1yfpmhrzBKyKf/i4eXFMNDchGoT/Ibc3HwvScKhIeeAdWjl+l0YvQdMidL4WvV Z5FQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=H6iBPqZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d38si22003209ybe.81.2020.06.24.07.20.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:20:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=H6iBPqZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6G5-00016z-Kt for patch@linaro.org; Wed, 24 Jun 2020 10:20:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61d-0005Pq-FO for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:17 -0400 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]:35785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61X-0003Xl-Dv for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:17 -0400 Received: by mail-wr1-x442.google.com with SMTP id g18so2434093wrm.2 for ; Wed, 24 Jun 2020 07:05:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=H6iBPqZtj/NDjb+zP/vFXoXSJkBu4Ypbg9W0WZqrHWAZjn7mE8c56oQmkQTwU6zRw8 uK21bRZtmUv4ncvsHCBz7n2kc3ZzrhUQDgT/E58G6kEjQ/BBZqdrlewn8Ay+HAw6Ta3S 4HMyOH4PTJ7jfGm2MaJIweyAFbiNWOHb+IQEDmFpoGvvmlq4BgFSk64wOQZi2hfxdyns A7p9+8CklWqF63sne/90FO+KL4GDAZKjCrjaebYkEUuKnswOB9pdW8E3SZNdJBUlfosS UZruaHU7FrSj5k1ygj9Dc1VBEZSde8l7D7Z8rppaUOBmy0by9YEhtyRLO/5CjL2FblS2 c+NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xcC1dEPRPSwS0guxyTrS7h54huqSdjkxAZKSe+0TYJU=; b=IRKjF6ginI794f8/DVQt/Y/GSF47VTt9E1+h7Gu0I2vXwRzSB9Lcp23kvmlPnV6+Mu HejsA3bXyKukdilAXZqmV6EuF4Vy9NeW0mjq/6UanLUkOmiLeTsJ5SGisP6GTOmKTzb+ f0n2evAMid3UUZ7bV1NWTrGksLRWsTDUqGdGx8iK0wIjb6cK9gaIFDJd1ky9Ax/34/86 3v0o9Tu7gm7mLD8wetccMOTmR6tcsHYs2YqaaE/fJHA6IPgreY32T94HPSEs2z4zdYyB JS7TYwfCDs+RrCK3HsbCm9EUBKPqFt62H98uYcAhBM5xBvyRfrUPsGRzfvglMsYhL5Ct LVIw== X-Gm-Message-State: AOAM532m+erVib93DP6U7WCeNIZhGWYiDvdpYtlUuS5WeumDsFGLc0My R6cpUiRcsvuC16y0/WmzMPLmkA== X-Received: by 2002:adf:f608:: with SMTP id t8mr16419707wrp.308.1593007509905; Wed, 24 Jun 2020 07:05:09 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id o29sm19603059wra.5.2020.06.24.07.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 82D111FF9C; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 13/25] tests/vm: allow us to take advantage of MTTCG Date: Wed, 24 Jun 2020 15:04:34 +0100 Message-Id: <20200624140446.15380-14-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::442; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We currently limit TCG guests to -smp 1 but now we have added some aarch64 guests we can do better when running on x86_64 hardware. Raise the limit for TCG guests when it is safe to do so. Signed-off-by: Alex Bennée Reviewed-by: Robert Foley --- tests/vm/basevm.py | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 93859362606..dd96a6d4af6 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -553,6 +553,12 @@ def parse_args(vmcls): def get_default_jobs(): if kvm_available(vmcls.arch): return multiprocessing.cpu_count() // 2 + elif os.uname().machine == "x86_64" and \ + vmcls.arch in ["aarch64", "x86_64", "i386"]: + # MTTCG is available on these arches and we can allow more cores. + # But only up to a reasonable limit. User can always override + # these limits with --jobs. + return min(multiprocessing.cpu_count() // 2, 8) else: return 1 From patchwork Wed Jun 24 14:04:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191659 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp627258ile; Wed, 24 Jun 2020 07:08:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyVrpvk59jHLQex3r9A5KPWoLYKxdYG5HNgdarn21ph1+Ahg4f9BH7JwapEKMRUwmo+tM0k X-Received: by 2002:a25:7490:: with SMTP id p138mr25604785ybc.74.1593007698833; Wed, 24 Jun 2020 07:08:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007698; cv=none; d=google.com; s=arc-20160816; b=NFUH9bvPBR0aj6wKDRiH4AqNGBdQCBYuW/OSX4eRmJ0ryomfTp6NGzBBnbYs2IJHI2 181ZOqlRbQgvzVEqXcSi4rMmwIwCPYYxBDdENq6E+JsBfI9e5jlDMbIEpzBsY+TRXVnp xBqCuwALSMo5+lGiN0iXCsCytaWtktJa6C1vJlvn46DvMHFlvytWK1zvASTDwY1jh6Fr ckQV9lzec53+IFT0q9zt/cCeW3weuHg0w4vMOwvuggkXnOyc1RLesKhYqFVt1l+ZimJ6 PfUWtt3jx4TL4MipMiIEmR+x5tbcftllct6zda6w24GsHFs6a4hdUvB49AAo2VNWhEFj F8og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=XZDmr89WKbKWsAL/XOPh4RkkUAsj0VPyyT1/XDx01DQkbLQprDyC4ZFSjPrLPAuOcp EQQ3RwkpI6o9W4TeqXxNCdMUDtfA7T03JiJgw/2FzcxP6zYO1fElvqhaY9uPrY8k6TAg qyhfJji4qWbKA1+NbulrdWOWNy+UwGSLTVu0xvpLM+vNY+/CV8gxKsWgxKbalSROWH8/ fSACU2CCyontr8dVwKcFK+d+PamOkK42IqvYASkc2meRLCIZWiSQO269rpC2UgkTXsGU BqZVknVmRmpWWsHX/QuB/xyraNEc0Vn7H+5MWk3tveoT2VTDomdLBcgpZAFUVZt3ElIh MQkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mhtkAArB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h1si19780502ybo.141.2020.06.24.07.08.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:08:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mhtkAArB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47236 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo64Y-0003wy-8l for patch@linaro.org; Wed, 24 Jun 2020 10:08:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59340) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61Y-0005Gw-G6 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]:36748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61W-0003Xa-K4 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:12 -0400 Received: by mail-wm1-x342.google.com with SMTP id 17so2665057wmo.1 for ; Wed, 24 Jun 2020 07:05:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=mhtkAArBeJ4g/pLN2chI6H/E73MmkTKkwmTatlo/mA/guux3+weGLovZ++icakMoyU j+2Khfas98ct2bAFbqzfLPTkGalXvcq9qPkbSiB/i9oeD9bH5IMUkYMQbnmt4IMYoGgF 4vZ+NpQck/C54duEym+T2Xja+7gLN3Y8ReT7GJo0Ckuj3HxY4Z24B3mSVrY6cN0OEGSF aFAAG8UBU9NyshKmDHc3UZ6rE2KQzv1shF6Sxlfq5j06q+7GzM+WD/PMxftv3FFcr2NF LglPvuO4Vc9zd4ppau6bLo0IOvcT19oDv2xJmxWhalXyGxpgLs63uXUSiQGXLoHZpR26 O6aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UsvCJ1FtgsYHjIIgU0bFzUSnvljWkIkb9Subsb2kO6E=; b=OOd8Ec7NwXT+CSZO96YS9gVpHu3cwVfpBPfo+PNFujJGzeCh+FM5IAjS+USYj8R9HO 6NAPJY3vBFvq3+/3Zz39VD4zNSIUA9ScQtGgMDJRELuM9fTAnXdtRkaIPxWZt6O/R/8t r7MEB+9rkPd+ZFpRoCm+sFXqeSTzt9QGuo1UX5VUDtsp0eLUxSKr0VuAnFx1Py8QAouP 1hoJzlLQqr4HBPeSQm7WbDZBLtNgmuqt5FXF/DfYuwBJAESQ0GH5wyXGNDz6dfMZl6NW zfEkP5FDDfvQpFRkjynxRwpAcg6ZPxDcyQhDryyr9VaJDvmvqDiH5jq0zA09HY0ZORD4 xPGQ== X-Gm-Message-State: AOAM532L3SVSnOOQblvY6f/08JRJSSB2F9rVYpUpwvIuqwUHsbmnVATN RcbwPWxcHELgw+vjI5YOLoaM9g== X-Received: by 2002:a1c:1946:: with SMTP id 67mr31731088wmz.59.1593007509054; Wed, 24 Jun 2020 07:05:09 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id l17sm8465200wmh.14.2020.06.24.07.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 991891FF9D; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 14/25] tests/docker: check for an parameters not empty string Date: Wed, 24 Jun 2020 15:04:35 +0100 Message-Id: <20200624140446.15380-15-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::342; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x342.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Peter Maydell , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reported-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Suggested-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé --- tests/docker/common.rc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.20.1 diff --git a/tests/docker/common.rc b/tests/docker/common.rc index 02cd67a8c5e..b27ce17e333 100755 --- a/tests/docker/common.rc +++ b/tests/docker/common.rc @@ -47,7 +47,7 @@ build_qemu() check_qemu() { # default to make check unless the caller specifies - if test -z "$@"; then + if [ $# == 0 ]; then INVOCATION="check" else INVOCATION="$@" From patchwork Wed Jun 24 14:04:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191671 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp634389ile; Wed, 24 Jun 2020 07:16:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzT2JrvUJQ5wNr6HKhPac3XaBQaJppyt+Ec+/zE/Syc4yxK2e22iLI2My8jW+mxNxzDrbKg X-Received: by 2002:a25:ce4b:: with SMTP id x72mr43956505ybe.78.1593008176550; Wed, 24 Jun 2020 07:16:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008176; cv=none; d=google.com; s=arc-20160816; b=pmjyCBKuGDl2af4L+RpQWDv0MBITC4tVYTV4bbo84Qs2l1vu2GIasCkPUuM5ZwDbhX 3t3M89JldRr02a9BZD7Moc5PzKdAAAQ+bxKHZ1aCHvjFVJlYlwwz7SgorO7OIRUyp/97 XDSmhQ3+whGnZY8x63pEbbs7COGyxx24jpTVXHcpKTdYZR1HDFkewnfSVFjCCSMBFYSL kbeQlUwdbrv/o4rLN3tB5eqkXzdHTFAzJzi64KJLU0eOIXyxShJx/Bl3CYa3QZKlkE/M hkQ3MIQQGNEzvJAhnU8pppxHa3xUND6Z2aTZiudgQCXoXBIo0XKrSeAvvbEc1JQg/pbe qX7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=k40c9VjxSWPNbhywxer/6zC54AD8vKJsue/6k/M6bTwBbcy+4CEoF3LPWlk9xvk6Xl epefpVP/YlXS5k2vH67enINH4mjdwvQFqujzWDRZ+LHqk7Ta77i5o+XjD26Hx7z5mVfF zj5lxIn0sWAAIOfTFe6ZVVFheEoB6gYdX/457KwSKxJ8QrQ3oTdfTtvOL6R3JTyGcfqO CZlqIYivUePs/IXOxGtnWjkOJWOZQPn37HwJhlJoh7z7zDY6D6qtnuz8brgiZGTto7kS A6KkYf8ynDd7jG8W1td0l+JVNKwCFYABQ2R2lutlJHpgYfyg7eqB/3KU4s+9/mdV9PTN 2/QA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mdjXnnwr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o6si19552575yba.198.2020.06.24.07.16.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:16:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mdjXnnwr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6CF-0002XY-SE for patch@linaro.org; Wed, 24 Jun 2020 10:16:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61a-0005LK-VR for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:15 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:35006) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61U-0003XG-M8 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:14 -0400 Received: by mail-wm1-x32a.google.com with SMTP id l2so1174056wmf.0 for ; Wed, 24 Jun 2020 07:05:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=mdjXnnwr4AEoSGvM85xMNKZwqFsizhFDD96walh/C77iUjymYsig5jydQ5K0mZKJO1 fGPZfCYB1hQi7QyeD/BBBmZJgua+z7pXPCkYSzSxWFwDza5Tw7KaXSxx2/8Y4sKbnhgh G4oee14IJVDe2dVFCaFyxc2lywyVxB+FmmIl3ToveN/+x4nkMLFVRmH+SY9Hd63mauUR 4SacVMtg3S+EJb3+lT94cjRTz7pSAj2y4z7Wx1WxjlWWiETju6OcYSMEI46OdiUfmSyL 9upTi6QtHzW+IV8XB2rV2BJpZ79xkQrWIwwYeWmUt4DtqKX0XMGfrNr60730nUyklQm1 OiWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OeJNj8ZC3zV8CkJ1XiE1kv6/qIn2cm2cQAQo+Livlkg=; b=Eq3vMt7LtxJxlJuj5LwQjjySbUoDy6eaR7xDLv7wfMDf2enz+LhYYnZ5DLtRZ1qPGt cMGvekTvbt+sxu/FP8jc7PvKARakwDaQ/gL0/NYKBa2o1Y6dMCfVffv350mKnCQC7GBB JKiauMam/enuNBzWXqYc6VWQYw2R4cY4z/LOHNxG+rXVQH83nr1ayfMJSFNW9vjK6oUn qoByWBLFa9MmuUBpLzIfImsn/AqaCB+gZ5E7bFiqsUn5giv1rpTvTezaRPXvr5DCxczs zJDdGwvqvqDukbZQX13yDPUlH1gghuN56fB4AI6cjOfzHTptOsfRrlZ0h4qQs2TTABGv wsLg== X-Gm-Message-State: AOAM533QtWy6h05e4gPjFcaInaHy8UmQhup67psj5pd1/ZBwCy8EIBJU Rtcjwnk0PR9VXJxYytUISmzrgA== X-Received: by 2002:a05:600c:218f:: with SMTP id e15mr23451277wme.187.1593007507061; Wed, 24 Jun 2020 07:05:07 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id h14sm11339731wrt.36.2020.06.24.07.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D9A231FF9E; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 15/25] tests/docker: change tag naming scheme of our images Date: Wed, 24 Jun 2020 15:04:36 +0100 Message-Id: <20200624140446.15380-16-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We've been misusing the tag naming scheme for some time by overloading the post : section with the image type. Really it should be saved for the revision of that particular build. Move the details to the other side so we have: qemu/image-name with the implied :latest version added by the tooling. Suggested-by: Daniel P. Berrangé Signed-off-by: Alex Bennée --- tests/docker/Makefile.include | 2 +- tests/docker/docker.py | 2 +- tests/docker/dockerfiles/debian-alpha-cross.docker | 2 +- tests/docker/dockerfiles/debian-amd64-cross.docker | 2 +- tests/docker/dockerfiles/debian-amd64.docker | 2 +- tests/docker/dockerfiles/debian-arm64-cross.docker | 2 +- tests/docker/dockerfiles/debian-arm64-test-cross.docker | 2 +- tests/docker/dockerfiles/debian-armel-cross.docker | 2 +- tests/docker/dockerfiles/debian-armhf-cross.docker | 2 +- tests/docker/dockerfiles/debian-hppa-cross.docker | 2 +- tests/docker/dockerfiles/debian-m68k-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips64-cross.docker | 2 +- tests/docker/dockerfiles/debian-mips64el-cross.docker | 2 +- tests/docker/dockerfiles/debian-mipsel-cross.docker | 2 +- tests/docker/dockerfiles/debian-powerpc-cross.docker | 2 +- tests/docker/dockerfiles/debian-ppc64-cross.docker | 2 +- tests/docker/dockerfiles/debian-ppc64el-cross.docker | 2 +- tests/docker/dockerfiles/debian-riscv64-cross.docker | 2 +- tests/docker/dockerfiles/debian-s390x-cross.docker | 2 +- tests/docker/dockerfiles/debian-sh4-cross.docker | 2 +- tests/docker/dockerfiles/debian-sparc64-cross.docker | 2 +- tests/docker/dockerfiles/debian-tricore-cross.docker | 2 +- tests/docker/dockerfiles/debian-win32-cross.docker | 2 +- tests/docker/dockerfiles/debian-win64-cross.docker | 2 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 26 files changed, 26 insertions(+), 26 deletions(-) -- 2.20.1 diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 3e3617816ec..7bac1a67e3e 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -55,7 +55,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker else docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ - $(DOCKER_SCRIPT) build -t qemu:$* -f $< \ + $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e630aae1089..cc6f76caa60 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -204,7 +204,7 @@ def _dockerfile_preprocess(df): for l in df.splitlines(): if len(l.strip()) == 0 or l.startswith("#"): continue - from_pref = "FROM qemu:" + from_pref = "FROM qemu/" if l.startswith(from_pref): # TODO: Alternatively we could replace this line with "FROM $ID" # where $ID is the image's hex id obtained with diff --git a/tests/docker/dockerfiles/debian-alpha-cross.docker b/tests/docker/dockerfiles/debian-alpha-cross.docker index 74bcabfdb12..10fe30df0dd 100644 --- a/tests/docker/dockerfiles/debian-alpha-cross.docker +++ b/tests/docker/dockerfiles/debian-alpha-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker b/tests/docker/dockerfiles/debian-amd64-cross.docker index 5d890419251..870109ef6af 100644 --- a/tests/docker/dockerfiles/debian-amd64-cross.docker +++ b/tests/docker/dockerfiles/debian-amd64-cross.docker @@ -4,7 +4,7 @@ # This docker target is used on non-x86_64 machines which need the # x86_64 cross compilers installed. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Alex Bennée # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker index 957f0bc2e79..8fdfd6a6b04 100644 --- a/tests/docker/dockerfiles/debian-amd64.docker +++ b/tests/docker/dockerfiles/debian-amd64.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. Further # libraries which are not widely available are installed by hand. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé RUN apt update && \ diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker index 09ca0a1ba79..b1a4613da5f 100644 --- a/tests/docker/dockerfiles/debian-arm64-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10:latest # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-arm64-test-cross.docker b/tests/docker/dockerfiles/debian-arm64-test-cross.docker index a44e76d9421..53a9012beb8 100644 --- a/tests/docker/dockerfiles/debian-arm64-test-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-test-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Bullseye base image. # -FROM qemu:debian11 +FROM qemu/debian11 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker index e3794a61c95..b7b1a3585f8 100644 --- a/tests/docker/dockerfiles/debian-armel-cross.docker +++ b/tests/docker/dockerfiles/debian-armel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker index e163b8b9569..25d76188337 100644 --- a/tests/docker/dockerfiles/debian-armhf-cross.docker +++ b/tests/docker/dockerfiles/debian-armhf-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture armhf diff --git a/tests/docker/dockerfiles/debian-hppa-cross.docker b/tests/docker/dockerfiles/debian-hppa-cross.docker index 5c68b2d3304..3d6c65a3efc 100644 --- a/tests/docker/dockerfiles/debian-hppa-cross.docker +++ b/tests/docker/dockerfiles/debian-hppa-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-m68k-cross.docker b/tests/docker/dockerfiles/debian-m68k-cross.docker index 25edc80e9a3..fcb10e35347 100644 --- a/tests/docker/dockerfiles/debian-m68k-cross.docker +++ b/tests/docker/dockerfiles/debian-m68k-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker b/tests/docker/dockerfiles/debian-mips-cross.docker index 08a8e1c29cc..26c154014db 100644 --- a/tests/docker/dockerfiles/debian-mips-cross.docker +++ b/tests/docker/dockerfiles/debian-mips-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-mips64-cross.docker b/tests/docker/dockerfiles/debian-mips64-cross.docker index 1a79505d696..09c2ba584e5 100644 --- a/tests/docker/dockerfiles/debian-mips64-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker index 453b53ef72c..c990b683b7a 100644 --- a/tests/docker/dockerfiles/debian-mips64el-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker index 3b6e975c686..0e5dd42d3c4 100644 --- a/tests/docker/dockerfiles/debian-mipsel-cross.docker +++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-powerpc-cross.docker b/tests/docker/dockerfiles/debian-powerpc-cross.docker index 89dd4fbf870..07e1789650b 100644 --- a/tests/docker/dockerfiles/debian-powerpc-cross.docker +++ b/tests/docker/dockerfiles/debian-powerpc-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64-cross.docker b/tests/docker/dockerfiles/debian-ppc64-cross.docker index 4bf88ab02dd..8efe68874ed 100644 --- a/tests/docker/dockerfiles/debian-ppc64-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64-cross.docker @@ -2,7 +2,7 @@ # Docker ppc64 cross-compiler target # # This docker target builds on the debian Buster base image. -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker index cd386f01d93..1146a06be62 100644 --- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture ppc64el && \ diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker index 5e2d6ddb600..2bbff19772a 100644 --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/docker/dockerfiles/debian-s390x-cross.docker index 43fe59836f2..9f2ab51eb09 100644 --- a/tests/docker/dockerfiles/debian-s390x-cross.docker +++ b/tests/docker/dockerfiles/debian-s390x-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the s390x architecture RUN dpkg --add-architecture s390x diff --git a/tests/docker/dockerfiles/debian-sh4-cross.docker b/tests/docker/dockerfiles/debian-sh4-cross.docker index 9d7663764e4..fd3af895759 100644 --- a/tests/docker/dockerfiles/debian-sh4-cross.docker +++ b/tests/docker/dockerfiles/debian-sh4-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-sparc64-cross.docker b/tests/docker/dockerfiles/debian-sparc64-cross.docker index 31fd34f120b..f4bb9b561cf 100644 --- a/tests/docker/dockerfiles/debian-sparc64-cross.docker +++ b/tests/docker/dockerfiles/debian-sparc64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/docker/dockerfiles/debian-tricore-cross.docker index 4a0f7706a39..769d95c77bb 100644 --- a/tests/docker/dockerfiles/debian-tricore-cross.docker +++ b/tests/docker/dockerfiles/debian-tricore-cross.docker @@ -7,7 +7,7 @@ # # SPDX-License-Identifier: GPL-2.0-or-later # -FROM qemu:debian9 +FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker index d16d6431bc8..b045e821b91 100644 --- a/tests/docker/dockerfiles/debian-win32-cross.docker +++ b/tests/docker/dockerfiles/debian-win32-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker index b0bc960445b..2fc9cfcbc66 100644 --- a/tests/docker/dockerfiles/debian-win64-cross.docker +++ b/tests/docker/dockerfiles/debian-win64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian9-mxe.docker b/tests/docker/dockerfiles/debian9-mxe.docker index 62ff1cecf2d..ae2c222a6f0 100644 --- a/tests/docker/dockerfiles/debian9-mxe.docker +++ b/tests/docker/dockerfiles/debian9-mxe.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian9 +FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé From patchwork Wed Jun 24 14:04:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191667 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp632724ile; Wed, 24 Jun 2020 07:14:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw/8mWwDXgcGztFN+Ln+WAyIvQDIRzYUGJOl8sNYIVfBfBT7wzVqPlA389DqCZ1R4po8UVg X-Received: by 2002:ac8:c7:: with SMTP id d7mr12469845qtg.235.1593008067644; Wed, 24 Jun 2020 07:14:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008067; cv=none; d=google.com; s=arc-20160816; b=xiPRnDi5Cn+tIlMuWCOKA9c5SmnSgHPseGTSaB6mr3IYz6CgnMDF7oK+8GIGRGXoBD bpYdE4ewc8/AXzpyjyYHyJOcN6pn+xLEkkR5JW6b+nS6LVnT5emlI9FkMNKGqs7yCRzw /IjwSIfQS3yW9DjChvPpvr8/cR0DhxWuqAEuyqoPsDwnBwLVDDtOFJVk05nVVJJyP79S bc/OrEfOTV3j8AAVVP1FwQAjTZe5+BuTvtsXppM86uwGJnwEVpcYzf6NoB895b5spOkN lCCB1kvjzJuz61LA3p1CEggaEse2/NLw/0kXmogimPngfnmYaUa0rRc5VwtBo/fsw0wV rYNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=nzFMAMTe82HnWVEZ18HLlF+w7GGnbFzxXGG48JQWFqVKxsjOwWKwQ3xKWFS9azKvW7 I+tbWkCiX1NF5FlBl/a9uhay/W/A27SfYJ7J9qcfLfUaBL5nrj1tXDTPvSO9RfCChLed pirIrFhtRFfVw6eR2gqnxhGxs/zOS7pQLvxMUmN3bbO5sw5vA0ACx6FeEJUwiGn5tUuS 7xVdcrRKKO7Ivfcb7y/dUgNDT9picFortVuS2c6WIb0D8726c6EyZWot6W6Qzivkq3yU Gw/cJKrvbui5n/oNTYRYrS1ZS8W9CgPbxFFGlRWsGW9Cqn9rN1Y8yR4AH0SVYLsVWHkZ VBUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qoVgHKhl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x12si7837861qtx.338.2020.06.24.07.14.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:14:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qoVgHKhl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6AU-0006fm-Vh for patch@linaro.org; Wed, 24 Jun 2020 10:14:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61X-0005EL-0J for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:11 -0400 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:40721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61V-0003XK-8A for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:10 -0400 Received: by mail-wr1-x443.google.com with SMTP id h5so2416053wrc.7 for ; Wed, 24 Jun 2020 07:05:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=qoVgHKhlnZMVtwgby0vh1A08Lu4m3eq7kDoxQxMuTRN05QuT8ASII+v2D83VXiJ2KH uItqonBdKB++2jl2DsS8U2/G1eO8HV9YKbimwSBr0H4uCuDuCg021y2GKHJ+opXRS4mK XXWox5tELn4s4QMvE+VpH2+J0vmuDJH/06s4+gnbvwkafQ5wCZp+9XPNLqylzc5sQWLN d9X7sV3Ax/DamLuedPt70f9NxiMRbXDl0kncC4rh+jnn0UPBBQUnWPWtkvwirL4lhaMS k0NDc9NzACSh1HVo+RwtltZAWF5DGKf/65xabbQnSOiNYEAOB7Xv8VIRZG53JK2G3kuB 1avQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XJD1GL90UWNycRg4BT4TsZXTpQO4+SzPpVo+RlIQlho=; b=af8FCoZ/bRaIPtOIriogm/7AGIu07TzpT9U768kW7E6FrWw98MigQcfswLJmx11j4h yf/aRBl93Z9Mv52qyfdphbNhMi48G5eIfGLtQLsQ+S7BWlmc/2OViGvgXnEYe+Q8dwSE dEtf8tT361FnM/cGpZezdLsqRUzt/DnIbQ6JMeip8Nfn0/yKVJazCjIi/Ny7c0NU5ssD +zhnFerLy09OxFfgd+GwXbxTtsVuVA9t9ieuL3at65JNTBPRdtA32+himTbN6uquSokn XsLWhuX68iFSTn8L/9jx1tNZCpMSaRE7vddcHhdOnOVhMJUHiVWld1/gcnsdN2sm9I90 +tAw== X-Gm-Message-State: AOAM532ZdhFAGm9fvoZH73aPOsB6tg25ffGJ7Olw8NEdRsY5u99N+2Nt zilzGii2wMSllkq+G+eug9/WlQ== X-Received: by 2002:adf:e38d:: with SMTP id e13mr10562160wrm.304.1593007507864; Wed, 24 Jun 2020 07:05:07 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id i15sm27161336wre.93.2020.06.24.07.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id EE1891FF9F; Wed, 24 Jun 2020 15:04:47 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 16/25] .gitignore: un-ignore .gitlab-ci.d Date: Wed, 24 Jun 2020 15:04:37 +0100 Message-Id: <20200624140446.15380-17-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::443; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The sooner we deprecate in-tree builds the sooner this mess of regexes can be thrown away. Signed-off-by: Alex Bennée --- v2 - just use explicit !/.gitlab-ci.d --- .gitignore | 1 + 1 file changed, 1 insertion(+) -- 2.20.1 diff --git a/.gitignore b/.gitignore index 90acb4347d4..2992d15931a 100644 --- a/.gitignore +++ b/.gitignore @@ -93,6 +93,7 @@ *.tp *.vr *.d +!/.gitlab-ci.d !/scripts/qemu-guest-agent/fsfreeze-hook.d *.o .sdk From patchwork Wed Jun 24 14:04:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191669 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp633369ile; Wed, 24 Jun 2020 07:15:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzq6eYbRaCp59rGMpZQ6KmFpmwiG5JljVAJeX80xE1uw1VNHuxMGL4xQs+5+ZmqMQyOZP/v X-Received: by 2002:a25:ca45:: with SMTP id a66mr45457879ybg.164.1593008107454; Wed, 24 Jun 2020 07:15:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008107; cv=none; d=google.com; s=arc-20160816; b=Ec+ngQgJ42qXN1wSmJP482lVFNH36/JIIo5Wl/sSGEecayo11fWrFY5yTYqnEgvuHW 4J5a9F6S7IQYCLZMIwpw32tPxOaFm/ARS49dot46dv3WuQjmYc7VYom/5ls3sLRVmZD9 L7FbNcBuK4pmOqXrobHv7HjyJES3xIT4V1K9DMQgkkGi/qIJCHGZ/XhqMWVFONNq8y/5 vt4iE0wi+h1YyDhpmty2h48JAFxSvNhuoW91Fxb5tokawChqskpWJOngvn+gYVJ3cwJ0 HzELGzJLibbk3a3JNVfkFcyro7+8R0+KoaVIzBztDVMZ4OW2nWViRFfan0SHXpK1jvIn Fbww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=oNxfLFFk6fN4Plk14+aSaWuJXYr3RTJIsGPAq5F58o7pmVvtbfaMN43zsnxtDZFXn8 c+mJmmkUQK2HqbTnm+bmxsEfAV2wuS+mWcvxWG2//wZbfsFuh/wx0yzQy0i8wifWNyaN ia+SnOJP0dRSgPbroXsIga0ojsYJEsBMRZ1O/Vguq5XDxdv8kiyyyQk6V/VDhIk20DyY TuNuKknha2QprTt5/j8aWDQmEJ4wl6f5WMEpcPdeunY1MIoBYAO8O9GB/H+6d4XG4MBb ukd4g11ZBkXLzHCbpaibX9CD0rMkwVJLUuqeglwevrdzXsoDKmvI6n/51sr/ko6rb73B VEdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xRVAKtGz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r9si14458004ybm.175.2020.06.24.07.15.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:15:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xRVAKtGz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6B8-0008NA-SO for patch@linaro.org; Wed, 24 Jun 2020 10:15:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68v-0004AC-Cr for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:49 -0400 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:35448) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68t-0005vz-If for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:49 -0400 Received: by mail-wr1-x42b.google.com with SMTP id g18so2461547wrm.2 for ; Wed, 24 Jun 2020 07:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=xRVAKtGzctVjzoQCyfyZRkdeeUi2NkP5Issuk3qYnS3FbpxRyGuXNBvBmHHYenzAvH fC+Dn+5oQ+oCb+ipGsAcPTRNOTKLwcwAXpIeipS3+hvyoRvVMaEftrdv1nb8d/t6Pnp8 v1gHA41TJeN344UqvquKepl+L/ONSiXMAz8GPLoiC2IbEbqrf8XfmmfqGNkX/FrtXwKx q7o392Wk+RGfKyTDxRxu61GXZU+PWy4AwOQl6dUcYKd8QzpVtPFTeXKwukT7YjivDyG6 V6lFpasCGfJIoxBljMVMi6hzjdN/OVLORWSKBwEk75DCAFBvp949mPtFYL3k6VERthg0 xZPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c7AoJ4KTaY68imxRMzTzv8EVx07Om13RK/P3Xy39tRA=; b=OvlLxZQxDdOX66q83dyrIclxCdn3PAcb/ECT8aJ83Qv5IYjjlNP7TZRe1kzvOO0lFZ IrY0z5ZQx/Gx+oW3IP9CFEEzgKORk982aVi38K5Vh5f5J8XvTpD8l4Gh8Hu+226y5h+i 4E83DR/vqNMhX/2n+tonybz1704t+vexVKVNmmvAii7f9ITc2elziBzoN0CLzE1ltkhi P8Yw5FTujGa6RBpchqj7kHtf63JEXmiiuvmP7FQTNw8Npo12j+V6ngM3vhEB8GzCyZi5 ZBiqbHrcuRGkf+mk6TSFjzGJ06VGp8EaNq7mqXmA3BPLrk4zSdBKNDDTaYZhXQ0PoZiA GCwg== X-Gm-Message-State: AOAM532H1nI3LL/rp/JPQ4qNkV3fAuvJX+bfWjAqWCAU+W55XvyNxZti 7xw1BeRoyp1YdKCtNOUTUfLbWA== X-Received: by 2002:adf:f70e:: with SMTP id r14mr33237282wrp.150.1593007966044; Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id c25sm8081800wml.46.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 1500B1FFA5; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 17/25] gitlab: introduce explicit "container" and "build" stages Date: Wed, 24 Jun 2020 15:04:38 +0100 Message-Id: <20200624140446.15380-18-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42b; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, Laszlo Ersek , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé If no stage is listed, jobs get put in an implicit "test" stage. Some jobs which create container images to be used by later stages are currently listed as in a "build" stages. Signed-off-by: Daniel P. Berrangé Acked-by: Laszlo Ersek Signed-off-by: Alex Bennée Message-Id: <20200622153318.751107-2-berrange@redhat.com> --- .gitlab-ci.d/edk2.yml | 3 ++- .gitlab-ci.d/opensbi.yml | 3 ++- .gitlab-ci.yml | 11 +++++++++++ 3 files changed, 15 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index 088ba4b43a3..d4e7dfcba65 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -1,5 +1,5 @@ docker-edk2: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-edk2.yml @@ -24,6 +24,7 @@ docker-edk2: - docker push $IMAGE_TAG build-edk2: + stage: build rules: # Only run this job when ... - changes: # ... roms/edk2/ is modified (submodule updated) - roms/edk2/* diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index dd051c01245..ec1c1f4cab7 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -1,5 +1,5 @@ docker-opensbi: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-opensbi.yml @@ -24,6 +24,7 @@ docker-opensbi: - docker push $IMAGE_TAG build-opensbi: + stage: build rules: # Only run this job when ... - changes: # ... roms/opensbi/ is modified (submodule updated) - roms/opensbi/* diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 349c77aa580..9fdc752ea63 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,3 +1,7 @@ +stages: + - containers + - build + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' @@ -17,6 +21,7 @@ include: - JOBS=$(expr $(nproc) + 1) build-system1: + stage: build image: ubuntu:19.10 <<: *before_script_apt script: @@ -31,6 +36,7 @@ build-system1: - make -j"$JOBS" check build-system2: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -46,6 +52,7 @@ build-system2: - make -j"$JOBS" check build-disabled: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -62,6 +69,7 @@ build-disabled: - make -j"$JOBS" check-qtest SPEED=slow build-tcg-disabled: + stage: build image: centos:8 <<: *before_script_dnf script: @@ -82,6 +90,7 @@ build-tcg-disabled: 260 261 262 263 264 270 272 273 277 279 build-user: + stage: build <<: *before_script_apt script: - mkdir build @@ -92,6 +101,7 @@ build-user: - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user build-clang: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -106,6 +116,7 @@ build-clang: - make -j"$JOBS" check build-tci: + stage: build image: centos:8 <<: *before_script_dnf script: From patchwork Wed Jun 24 14:04:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191672 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp635637ile; Wed, 24 Jun 2020 07:17:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxKWvoGW3Bk9XAT0wwEGSKUiBxWcBjnMTy2MMBDIDxPDNsFmNgJR9Yw9fqjAafGWPKnbdmo X-Received: by 2002:a25:60c5:: with SMTP id u188mr43901977ybb.268.1593008269268; Wed, 24 Jun 2020 07:17:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008269; cv=none; d=google.com; s=arc-20160816; b=0EqhHGxHSpEM2kFn5BZfn60b5O5CbUJLVQs2vzVo6EWE2MEGKt0Gf4KWfcwCC9F2s8 UaBZL8DLMaeU/s13iN0W7p4PwJyEOAdOu0B9VS0rDbkPQOw43GlsjbdAcod31PXOixhk qu1UTC4vpRNfKfBh60laIAf0HE/xibuuz2yjsbC8Ys/oKJSYcsRNrwaE1FT+4YIZsUv7 aUvSi8+VilxJdOMuQhcJMtf/7HUbx9EvVIXPbg2frfmQghFNfjBvQFix+hGMXWXJ9d3/ JweMYLVaiSnzybkve3Zg/h5fmHrpJRTtyyhhlh/qkNRbkgayeg9X4stCRt+0zxguZCHQ TENg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=lg7yVStaFiCphxFtdOAJanK/D+52X1Xo/oiJ0lxSebYQXIPESoZB/hRblqM2idCVVl go0op9B0FRHxX02GBixRimi9wzZuamft80ZDsqt5kZNdOFSrtLsgi6MVQm2NdFv/O+n0 ywUCnq0wUnd9VMJRM9I9yWBUYTJKs/qstgd+7R743Q/pxwJCzH1gKQYiyBXPK6srJLhe mOe4rHg07qI9fgRUU9G4AE5WSm2GpU/9CMly36IDHsp1v/bkKFXZ6Q+PNASYC7OFhtoO aTvI1BDBklnXxtUDCUa9RUUbJLvXXzOEI+pysu8ggYOCW71q8Zm7JeSZP14FSNGh/jX1 R4Pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=k4Tsq37p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a5si18798021ybg.97.2020.06.24.07.17.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:17:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=k4Tsq37p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Dk-0004wk-Om for patch@linaro.org; Wed, 24 Jun 2020 10:17:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo690-0004MG-7c for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:54 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:46103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68y-0005yx-2O for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:53 -0400 Received: by mail-wr1-x433.google.com with SMTP id r12so2404322wrj.13 for ; Wed, 24 Jun 2020 07:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=k4Tsq37pNmIh4FQGQktJmIeM+22h8iZzyUuLGzPNnPZSff+zV+vOm2yBPRSk+yj8qq djBgFexMBGJVdnkuiemFkYZjSm9oUsUNexWwdDVeb+s61XxWjQpdw0A+hcZOuF7r50vR Cp0kMz0bgJWGG3CHw1MwadXGfzLi7Rm3tRz1Y/NkeJt/ykShFkBFBceQcAerq1hifHWB jPpzWRq/lP2ffM9LZaeBaOCClymjZEOIz8Bty7LV39FhQ/JNCQtheD/CObAs2HUk0Hfs MldG9SbtxVJKq6Ggcxq+N8tDanl0SEBVzOyrOu8LgIEco8EC8QmCosjviKiNSlIWWTd2 O6yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=WZt/lLDU+ikOaxTaQ5/S+0otsBP37tyt6H6gRLoS+IIoCCS4D3a4FNtU21b+tCCaen 1VChqSal+ZwW2OqB16vYgydKy5yki0lgoj/W7BbOokyjoMOsRKExQDu+BomKYGStuxuv EdQruDy40rQXAO3c7/kI9g4esIebp1YQGRm+AtWFVmhvkmGGLKeOFtuCZa4F3MhW2dxs ZwDQkCtHoCtFXL1uKFlFEx7CZRBqQo+eQ37zdgkpcdYEfdAxhwXmO4tAf0586u2PPbok +6IUQ9ouz5RLDoP8oVuAB54eJLmzCZ5g1yT86qgULvSkoA+WJWX5QJjsvqGsp/GwKRlY boMQ== X-Gm-Message-State: AOAM532VasMVUlo71VOSMx5oAfp36km/7TFt5jPZucxG3nX9SszCXzJH Nft/+v98TxojcUj8wpXBm7SuCw== X-Received: by 2002:adf:e50a:: with SMTP id j10mr33329426wrm.71.1593007970435; Wed, 24 Jun 2020 07:12:50 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z1sm18759179wru.30.2020.06.24.07.12.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2FCFD1FFA6; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 18/25] gitlab: build all container images during CI Date: Wed, 24 Jun 2020 15:04:39 +0100 Message-Id: <20200624140446.15380-19-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée --- .gitlab-ci.d/containers.yml | 248 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 251 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 00000000000..580439647ef --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,248 @@ + + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu + diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea63..72d688875fd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' .update_apt_template: &before_script_apt before_script: From patchwork Wed Jun 24 14:04:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191666 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp632202ile; Wed, 24 Jun 2020 07:13:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzg9Dp6O0HUJ6NY9bqJ/N+pbDjvfy03qslRJFAlMhksvZdnjQ9Nz55oFCktl7TDssa+v8FF X-Received: by 2002:a25:e8b:: with SMTP id 133mr44826459ybo.102.1593008030237; Wed, 24 Jun 2020 07:13:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008030; cv=none; d=google.com; s=arc-20160816; b=HmJG64pCfwGMaQKMrceslhUFaukSBaV0untcMZRVZsGkI/Bg9jt3Ma8OxfNB7cdOfk IKjTFyR4E4AJBlEj5hzx941GKweHZ3TJb+XIBEgOU7EMLf4efe2OYm1rwEXQe2YQGwzk D8iyULsfe5GYXL3F31jRehc/uxNuoulCMZuvQacUTQTEF4LwGOUnDrLp8jpop1qdZ0LP R8QDrDt5CFbq/FTSguqy7qSrNry8HDrxhNkSHkCmDWjcorI+sVl2Qr7TM3JZoRvjKReE Vb/z5pPJ0Spud9Um19Galfc+z57PIBQ1SOkTlGi+MRPbt6R25qLJJMFPxYKTE/u2kPxg SfGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=CaimTliZGDtA1/MCMNM2TIGjB7QFCXYq/Q0MzrTEVt+pCKazbEAeKbtgyE+O2SXGr2 GWhAHrurD1J8Up1+0TyArksVt8/QYKwaHplV4afgd8PL6hwzq+E2VL6A5Y7dUO+BwzjW NoJyZzJjpJalzgmYSrWMeoahjttb5/3dpOtwQjwnop+8nFnc0YDsuu3c3mChqc12KXmI MQdq7GfTN0cpKfx9dZ6vFo/H+ofJYdlM6cW6eCSz4sIS/CuH3y1GqweNzPt+z3qVHw5t oxQh1qatCfdUiFcSMxWa0RBVzNYuLdSKfTp6fYl+baU1/QYnQW2gXdL3I2lQlfhgXktC s1kg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Gd5Vpp7L; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p16si19091723yba.199.2020.06.24.07.13.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:13:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Gd5Vpp7L; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo69t-00049T-LA for patch@linaro.org; Wed, 24 Jun 2020 10:13:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68t-00047D-Ot for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]:38193) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68r-0005v0-KQ for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: by mail-wm1-x333.google.com with SMTP id f18so2682240wml.3 for ; Wed, 24 Jun 2020 07:12:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=Gd5Vpp7L+pxJ1UyhJa7Iayqcx7za3wgVZYEPEFRIvIP183AjkWg19BllD8Fbw3n02A YAaT+g4ocyAqkcGOWbpfmrqTzRb1ndTDrhI1yk0J0kKuvsHvSv41GuOE3U4v5lLCdVea AlK1n50e6+wSS7Fg5F61TsfvFVeF7QidmI33/ab8laR3rHhBZFkxi8stGY45Czs1Uc7c LoV2hdW5R/T1DHD2h5TX1HEkAce6ZvnoM0atl0T0MhVcsFIx52E2TjAL5tk048TzdQr3 ksZVpIwA5AK30jVtR7GXiHH1sYKIq2+7k213WtF6CQFnoDYgAv1JfTfmpYJPzDcr9Q3q U4BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gYny8RQMLGvnYX7pxE0cYv3y41J+K9oraa4mW6ajsxI=; b=D1ldMzNmibpH5Slpf2RYx9pNtkwb05YbVKG1L3MMaXa4lL4Kfb17oJeGKPBFoMKM0o VAIuWH0sgrDHDyIKlTgCR4Ra1H27xmeCZSS6gUj/fJhFNyhUoWV+pBCi7XVo4C9g4R6i n1B+31detmRxrtD+PmjPcqJTC4tjTTjHAvJyFER5MVJ7oI3/+r+QNR+kPL0ba3i6pvr2 75738uGk6V9s8lUzUR7y/VzPyLvGxeYz5Z5FDtz0+fgS+MfP4830TsKSNnNTdShrkzhq jXs82uO1igZeG84GTaIDRH8V+/muknMVt6sDrQxSIDy31FLk4j0V/Pi1pjHS59kekZ8I 2Upg== X-Gm-Message-State: AOAM531LFQjWIt8cHnKy0MW0SZodvEks1hvz8lD1ndJnNuWlVXYH5MWx L4K4cPr3I557Rp8KoUdb5nXj+w== X-Received: by 2002:a7b:c1cc:: with SMTP id a12mr21586291wmj.112.1593007963596; Wed, 24 Jun 2020 07:12:43 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id k11sm24741091wrd.23.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 457EE1FF7E; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 19/25] gitlab: convert jobs to use custom built containers Date: Wed, 24 Jun 2020 15:04:40 +0100 Message-Id: <20200624140446.15380-20-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x333.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé Now that we're building standard container images from dockerfiles in tests/docker/dockerfiles, we can convert the build jobs to use them. The key benefit of this is that a contributor can now more easily replicate the CI environment on their local machine. The container images are cached too, so we are not spending time waiting for the apt-get/dnf package installs to complete. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-4-berrange@redhat.com> [AJB: tweak naming convention] Signed-off-by: Alex Bennée --- .gitlab-ci.yml | 187 +++++++++++++++++++++---------------------------- 1 file changed, 81 insertions(+), 106 deletions(-) -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 72d688875fd..a7abc55a5c6 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,133 +9,108 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' -.update_apt_template: &before_script_apt - before_script: - - apt-get update -qq - - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make - genisoimage - - JOBS=$(expr $(nproc) + 1) - -.update_dnf_template: &before_script_dnf - before_script: - - dnf update -y - - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-devel - make python3 perl-podlators perl-Test-Harness pixman-devel zlib-devel - - JOBS=$(expr $(nproc) + 1) +.native_build_job_template: &native_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + before_script: + - JOBS=$(expr $(nproc) + 1) + script: + - mkdir build + - cd build + - if test -n "$TARGETS"; + then + ../configure --enable-werror $CONFIGURE_ARGS --target-list="$TARGETS" ; + else + ../configure --enable-werror $CONFIGURE_ARGS ; + fi + - make -j"$JOBS" + - make -j"$JOBS" $MAKE_CHECK_ARGS build-system1: - stage: build - image: ubuntu:19.10 - <<: *before_script_apt - script: - - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev - libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev libvdeplug-dev - - mkdir build - - cd build - - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu - cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu - mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu + moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu + riscv64-softmmu sparc-softmmu + MAKE_CHECK_ARGS: check build-system2: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel - libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-devel - libzstd-devel - - mkdir build - - cd build - - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu - microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu - sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu + riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu + xtensa-softmmu nios2-softmmu or1k-softmmu + MAKE_CHECK_ARGS: check build-disabled: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-rdma --disable-slirp --disable-curl + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --disable-rdma --disable-slirp --disable-curl --disable-capstone --disable-live-block-migration --disable-glusterfs --disable-replication --disable-coroutine-pool --disable-smartcard --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user - --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user" - - make -j"$JOBS" - - make -j"$JOBS" check-qtest SPEED=slow + TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + MAKE_CHECK_ARGS: check-qtest SPEED=slow build-tcg-disabled: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel - - mkdir build - - cd build - - ../configure --cc=clang --enable-werror --disable-tcg --audio-drv-list="" - - make -j"$JOBS" - - make check-unit - - make check-qapi-schema - - cd tests/qemu-iotests/ - - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 + <<: *native_build_job_definition + variables: + IMAGE: centos8 + script: + - mkdir build + - cd build + - ../configure --disable-tcg --audio-drv-list="" + - make -j"$JOBS" + - make check-unit + - make check-qapi-schema + - cd tests/qemu-iotests/ + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163 170 171 183 184 192 194 197 208 215 221 222 226 227 236 253 277 - - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 122 + - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 122 124 132 139 142 144 145 151 152 155 157 165 194 196 197 200 202 208 209 215 216 218 222 227 234 246 247 248 250 254 255 257 258 260 261 262 263 264 270 272 273 277 279 build-user: - stage: build - <<: *before_script_apt - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-system --disable-guest-agent - --disable-capstone --disable-slirp --disable-fdt - - make -j"$JOBS" - - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + CONFIGURE_ARGS: --disable-system --disable-guest-agent + --disable-capstone --disable-slirp --disable-fdt + MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user build-clang: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-devel - libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-devel - - mkdir build - - cd build - - ../configure --cc=clang --cxx=clang++ --enable-werror - --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --cc=clang --cxx=clang++ + TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu + ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user + MAKE_CHECK_ARGS: check build-tci: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64" - - mkdir build - - cd build - - ../configure --enable-tcg-interpreter - --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" - - make -j"$JOBS" - - make run-tcg-tests-x86_64-softmmu - - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test - - for tg in $TARGETS ; do - export QTEST_QEMU_BINARY="${tg}-softmmu/qemu-system-${tg}" ; - ./tests/qtest/boot-serial-test || exit 1 ; - ./tests/qtest/cdrom-test || exit 1 ; - done - - QTEST_QEMU_BINARY="x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/pxe-test - - QTEST_QEMU_BINARY="s390x-softmmu/qemu-system-s390x" - ./tests/qtest/pxe-test -m slow + <<: *native_build_job_definition + variables: + IMAGE: fedora + script: + - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64" + - mkdir build + - cd build + - ../configure --enable-tcg-interpreter + --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" + - make -j"$JOBS" + - make run-tcg-tests-x86_64-softmmu + - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test + - for tg in $TARGETS ; do + export QTEST_QEMU_BINARY="${tg}-softmmu/qemu-system-${tg}" ; + ./tests/qtest/boot-serial-test || exit 1 ; + ./tests/qtest/cdrom-test || exit 1 ; + done + - QTEST_QEMU_BINARY="x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/pxe-test + - QTEST_QEMU_BINARY="s390x-softmmu/qemu-system-s390x" ./tests/qtest/pxe-test -m slow From patchwork Wed Jun 24 14:04:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191668 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp633268ile; Wed, 24 Jun 2020 07:15:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwSE4OkIM2yijkkc2EMy4dZZuSQAGFaz/q0zalgM4unOTvjGi0fh7xY6PdvmbHBlJKkm8YO X-Received: by 2002:a25:8011:: with SMTP id m17mr45717278ybk.370.1593008100455; Wed, 24 Jun 2020 07:15:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008100; cv=none; d=google.com; s=arc-20160816; b=xOsyEv+6bBsAPUVgojAejoDNqvjs4X+wnUFJtAnmSog0C70UeMRlC+ElvP8eexGfwb OQZ+WzHC73HaonVfjGhxJsb8/CyakZGloMp7VwXbVvsxx11rBaTcsdTSuH0fM6QayjdP W70U6jB20xmjoMVtNe1dIEuwYtU5ioYfVqnrlolJiadjeUCWSz9ymQIs2IJlWZcHfSs/ YbfHFfQ20NCFAZEyti0pFvstpwGKNDwRTJH8AEiY16sGpldOlbAVtoHSQ6lZAVn/hZFt R4H/Y0EGFOkBoGOj3UOh0cuaMHdbonq7W9DO3GsDYhQFRf7r+Ibgu2N8iXafY76IYMyn En1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=myDz7A/gOtbYJ3Nm89jiFSHKGdy83u61TMPnUjIExEG0/0RJBZj6RDrdJXAMIp0TYs ecGLuNJgyYr/ejzZV4qTbeiRhaW7+Nfo23mcY6QtVfEGhxMwUpwZIXeHZ+9NLVJZVMLd LUe+HRjmYQV4EnP9gN4+D2fS982pffQWo2jr9m3k3DdyCcOxHSHw/TmWxoUiLdbAmqd3 y5d6cJpujZxWM82PXC7HGwvd3yt195VBogbBU6PdJujoMjKnRfHU0BzPvxky6v/Utp28 PJEfY9NMUlP4vLhh2Cg+6B4sLUP/Q+Ik+Zq4B4WrQwOrZEpnf/UWnWNMh2SvTej0TSs+ f6vQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=wskmbiq6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w126si4220346ybg.145.2020.06.24.07.15.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:15:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=wskmbiq6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43458 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6B1-00087S-Sm for patch@linaro.org; Wed, 24 Jun 2020 10:14:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33514) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68u-00047l-1q for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:48 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:40418) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68s-0005vN-BT for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:47 -0400 Received: by mail-wr1-x431.google.com with SMTP id h5so2443927wrc.7 for ; Wed, 24 Jun 2020 07:12:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=wskmbiq6B0bDmhHqaq+JKx4A3ASBEDCd4VUB0g9n4AwmKCCjXDuy285gB64yqW6Jux TWfX08nwJNpDzaoFcHIq7HoVxEmWYlmj2zCq/Qa/CBGFB3DpJKjzJCMsPWAZwH9rSzAZ lfxBm6s1mLeNg20ch7zhUIehous6cLRbG/RwTCxC6L+b5qztarw/6I3LX3oCgInWHotT x0gYhiRDcHtfeyWtNoE1upryP7mfCiksNIrw/1+mqn6xYHtH9CuWnFOiWRZQLqiatZ+h MdH7714PQ9eLFwDPpc9Cufz3KavE0SYOLgJ7dEzKF88dTCdlShG32dc5dpVQRyKlX7F1 7s0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WD6VNcX9MpLlUaAe843m6W+P0R0E6C+GHfBa0ZiWKL8=; b=cPejsaQkjroCmx/qETIlb2Ut64pXVOX6XG0UzdX6287bLJRRLxoa8L4AfIx4gZU0YO 6yq5+9owQNTadz1xp5ClLVDBGXNtvsRA35IULaBy/j7gYMIEijErNU11NIPEsSQGLiht aar4yJ2BAw5tyqrcfqd5vLJrab6TppccKt2nakFNyGorcJjdgGWLBelstNwnEExu/nNd t1v5AqqybyPL7xVlk5IIHjNJqE9n56ewoO4bkVmkUg3x2ESmDH9GQUlHsxL7XIWgQR8D WT9e5RJT0mG0byj/+oVrqb79WCDHh48hRC8kd62Xu/ewmmdS7GgmannrZtxSkMm/kgnW JXhw== X-Gm-Message-State: AOAM531TFjKjPwxTP/N2B/1pf1SWH0j/INwskqox7lRaCiSOJgsSlTDf psmJkT1CxgU3DbJg4bpxrkg4mQ== X-Received: by 2002:adf:81c8:: with SMTP id 66mr26929810wra.348.1593007964920; Wed, 24 Jun 2020 07:12:44 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id e4sm1020934wrt.97.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 59FE31FFA9; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 20/25] gitlab: build containers with buildkit and metadata Date: Wed, 24 Jun 2020 15:04:41 +0100 Message-Id: <20200624140446.15380-21-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" According to the documentation to be able to use --cache-from for remote registries you need to enable both buildkit and inline the metadata. We want to do this to support pulling from gitlab when users build their local docker images. Signed-off-by: Alex Bennée --- .gitlab-ci.d/containers.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 580439647ef..ba9c24e98f1 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -13,7 +13,9 @@ script: - docker pull "$TAG" || docker pull "$COMMON_TAG" || true - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker - - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - DOCKER_BUILDKIT=1 docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" + --build-arg BUILDKIT_INLINE_CACHE=1 + -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles - docker push "$TAG" after_script: - docker logout From patchwork Wed Jun 24 14:04:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191673 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp636020ile; Wed, 24 Jun 2020 07:18:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxjHzELBxxLACYa0gFnvA28i8H5aEuRZu03035E0mrktq+qcUOf7GlqPopZv1vaK3R5C/ha X-Received: by 2002:a25:da44:: with SMTP id n65mr47643306ybf.387.1593008297110; Wed, 24 Jun 2020 07:18:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008297; cv=none; d=google.com; s=arc-20160816; b=IPTOkpyWiXIFr5XGSLvYx4mVaXVoFGOKh1B5nOM9WhYdASyU2FNaXWv+JozmUY5xTd IyswfrdSlLWVlu+2kcP+ZVjSe11XvkWT/YmouL2W7HbQcQPR7a0xQORAMDpLZSWGQYwU IvA/JJI3VfQkL/Iastdnz6yF/NBlftaFa8uWKq3A/v+cHpJjDAqgXo1/0qFpmEKkEqTi Utc6i5PtISnu7IF9qTPJzpWRjtFVAU60HuOrJMR/F2UrpStZ051ZKcAHTyZDrzSENfRs dwcZgiXxtg6JZe+Lch2Z559VFFSgxM/lIIIMSJhaGB40a8heiedyYeym0fWc66uqOA9J R59Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=db4pT0kX4V10YiTq/mCrQCysWc/nSzXa9D21gwWRnedeko0QybON8oD09aTS7A+/uD +Buet2ji+W/obY7Cfxkiz7dnyKD0knQDopwkyoaQNCjmqVymHfhoLWOquHJrIHbesRf6 644Jo+SxQCXVd+F35GFc+K2E8APunSep//YoAhSjnv4+1DC5FZyzKv3BozbHxSHsA+Ak BnG7uqiDwq+qvQI8vf7d+FO4dBGhvqaEGogPPkd9PdYw1LC43wxXSnBGus8MXraZg4i4 jxaTCBkOUdJYgP2vK3iLO5vhoAolVwNr/y6CbvOir1rPi2vi+g2yGgO1t1jvqX/CoNLs Fs4A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hyyFrqkw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e5si21058062ybp.312.2020.06.24.07.18.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:18:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=hyyFrqkw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57726 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6EC-0006Bs-Cm for patch@linaro.org; Wed, 24 Jun 2020 10:18:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo61c-0005NH-3M for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:16 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:54120) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo61Z-0003Yc-Pu for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:05:15 -0400 Received: by mail-wm1-x343.google.com with SMTP id j18so2441419wmi.3 for ; Wed, 24 Jun 2020 07:05:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=hyyFrqkwGkLUP6+PLih+TsKY3F1hRXCmQnL3uNwg+PEUaF6E3v9AjGMEkY9lZPtp2a 8Gdw9waGFQfkVBvOMxd9ruWCt88vlCIMcUnSkcheK0weGnBzp8ZVQJvwphKInmH1eGTa 19vXG7TrgTbRcsuuETPhHYqXad+A1N+eIRnTH8h6UzD+ZYDK9TyDjuj8TjLIt2HEUHup 1l/QWIoD86xLuP9WF7cj02bXwiVzPNClhwDFFUDY3ETJ5xiwVq2G8f2XgxI0bwTsUxDk l1oiPnoEmt5t7QnhS9VijNQSdNe0RJ0DGm7pOfM4cJzmwc1Nygch87UGKZI0Z01pm9Ya vgcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2kkJsCsmjZ6u2MwSGAsWWjkvARamWfB9kpoiOtV7oLc=; b=DjGQn6wZtB2zcP9LDkiyHTJB+vsQGz2D0HcAv18a+n05iTdJkcm+h6zyip0FxSIwsV r6M+SLITKUjwnGtfzoD3Tq+kSvxQtK6ZMbP3Mplb0daRfFeyWyMykPVzfptUmjTo93r+ RU9wCbATrWlSHfWjOPjpEvUOaf0UxzDskhEgZ3J8NgtbVjR/ygICV8vkges73UHa2qT2 YmpEdX4ajIwKDLNPHxFLrNcsdELpa/3BJehjIQLFtjzKx6b7xwaCl9LUIy48cKA8FY3G R+IHlTkGoiv3SnfLJ3LCh9CAUbnzkjCVOLUIzqEkC8kTfNFANdOkm6rvvqiygBRpExIQ ryDw== X-Gm-Message-State: AOAM533ASOKrWcaiZT7sayQLylrgZO9Om/t/414/A2yDNCnjDCkcCtOs Nxa1PjSwA5yxb3CNpKpmszRqDg== X-Received: by 2002:a7b:c926:: with SMTP id h6mr27226435wml.184.1593007511369; Wed, 24 Jun 2020 07:05:11 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id r1sm18997755wrt.73.2020.06.24.07.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:05:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 707F51FFAA; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 21/25] tests/docker: add --registry support to tooling Date: Wed, 24 Jun 2020 15:04:42 +0100 Message-Id: <20200624140446.15380-22-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::343; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x343.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Philippe_Mat?= =?utf-8?q?hieu-Daud=C3=A9?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This allows us to point the tools towards a registry from which they can grab pre-built layers instead of doing everything from scratch each time. To enable this we need to be using the DOCKER_BUILDKIT engine. [AJB: note registry.gitlab.com/stsquad/qemu is for my testing, the final version will see DOCKER_REGISTRY default to registry.gitlab.com/qemu-project/qemu] Signed-off-by: Alex Bennée --- tests/docker/Makefile.include | 3 +++ tests/docker/docker.py | 25 +++++++++++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) -- 2.20.1 diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 7bac1a67e3e..4c5ebc6197f 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -13,6 +13,7 @@ DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.doc DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache +DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),registry.gitlab.com/stsquad/qemu) DOCKER_TESTS := $(notdir $(shell \ find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f)) @@ -57,6 +58,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ + $(if $(DOCKER_REGISTRY),--registry $(DOCKER_REGISTRY)) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ $(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\ @@ -213,6 +215,7 @@ endif @echo ' Include extra files in image.' @echo ' ENGINE=auto/docker/podman' @echo ' Specify which container engine to run.' + @echo ' REGISTRY=url Cache builds from registry (default:$(DOCKER_REGISTRY))' # This rule if for directly running against an arbitrary docker target. # It is called by the expanded docker targets (e.g. make diff --git a/tests/docker/docker.py b/tests/docker/docker.py index cc6f76caa60..09dedbf83f9 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -221,6 +221,8 @@ class Docker(object): """ Running Docker commands """ def __init__(self): self._command = _guess_engine_command() + if "docker" in self._command: + os.environ["DOCKER_BUILDKIT"] = "1" self._instance = None atexit.register(self._kill_instances) signal.signal(signal.SIGTERM, self._kill_instances) @@ -289,10 +291,16 @@ class Docker(object): return labels.get("com.qemu.dockerfile-checksum", "") def build_image(self, tag, docker_dir, dockerfile, - quiet=True, user=False, argv=None, extra_files_cksum=[]): + quiet=True, user=False, argv=None, registry=None, + extra_files_cksum=[]): if argv is None: argv = [] + if registry is not None: + dockerfile = dockerfile.replace("FROM qemu/", + "FROM %s/qemu/" % + (registry)) + tmp_df = tempfile.NamedTemporaryFile(mode="w+t", encoding='utf-8', dir=docker_dir, suffix=".docker") @@ -313,8 +321,14 @@ class Docker(object): tmp_df.flush() - self._do_check(["build", "-t", tag, "-f", tmp_df.name] + argv + - [docker_dir], + build_args = ["build", "-t", tag, "-f", tmp_df.name] + if registry is not None: + cache = "%s/%s" % (registry, tag) + build_args += ["--cache-from", cache] + build_args += argv + build_args += [docker_dir] + + self._do_check(build_args, quiet=quiet) def update_image(self, tag, tarball, quiet=True): @@ -403,6 +417,8 @@ class BuildCommand(SubCommand): parser.add_argument("--add-current-user", "-u", dest="user", action="store_true", help="Add the current user to image's passwd") + parser.add_argument("--registry", "-r", + help="cache from docker registry") parser.add_argument("-t", dest="tag", help="Image Tag") parser.add_argument("-f", dest="dockerfile", @@ -458,7 +474,8 @@ class BuildCommand(SubCommand): for k, v in os.environ.items() if k.lower() in FILTERED_ENV_NAMES] dkr.build_image(tag, docker_dir, dockerfile, - quiet=args.quiet, user=args.user, argv=argv, + quiet=args.quiet, user=args.user, + argv=argv, registry=args.registry, extra_files_cksum=cksum) rmtree(docker_dir) From patchwork Wed Jun 24 14:04:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191670 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp633865ile; Wed, 24 Jun 2020 07:15:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyoBU8UZHJQP+RJ8WVmEmuIt0yf8nYkmoZF+uWnDZhXHWekia7Bzjq2mn6LCUlTL1t8WDjQ X-Received: by 2002:ac8:4e4f:: with SMTP id e15mr26733866qtw.318.1593008141506; Wed, 24 Jun 2020 07:15:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008141; cv=none; d=google.com; s=arc-20160816; b=NyHM88OTz8nOojbhplYJhLNIpK3zO+Ly75SCLQLsS+11Z0bCMs6AggfrebUys0f+2m ytlGKyHBJd31Che2KD2BwSzvwnk/DVofkhH3HLkV8MIHHf4VN3b8SKYgVFHxHSdoNVof 7cMt1beyH+tC2rr2BxOd38gyV35SUjpCxCBxNOoM5ikLLibKhnikCWpeWgQoszwGGDGm bDGRQ8XnvdIrfkMDPOBsge52qJVYq0Q8vPEr0FU9TK7xKUp03ORE4nJnOhHhfoWMfXsq zzYcS53xYctAy0HfoH+MOS2xEyfw2du5ifs4dq3iV0WcHx2ann6JtY1+zrWXPb409ddz 633Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=t/FWodzDOuKSWzuBJ1HTlfkV4douXPqhAEmdykUfse2FS0xG8LpJP/XgRPIFTXGU9u UY8Dz9rpAAAF/IZ/Ef154GGMvgECOv4vFCoZhujos3cOJKxjS45DylKMrQs33cPnjHPn Q3hFTTFRaXjndIvinssr7wpOQF3PWcscKYnv1Ce/T8nJsHLq9g6LsPpSRbotfPizC7Iq cBp2r26xkcQIC/ccBvci7MNlx/IZMZ7KkhkkFx3UHibu/hrJYbfdts5/Eqeei/rjx8kx /9y3SHffbyHhN/ZpDyYUS0DMF7RwhDqf6cuMGDdN7xa8YWycaze15DOIggMmlZkCYs+E HDKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zy5uGhZb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m52si13636239qtm.10.2020.06.24.07.15.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:15:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zy5uGhZb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Bg-0001TG-Og for patch@linaro.org; Wed, 24 Jun 2020 10:15:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68y-0004I3-E7 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:52 -0400 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:51284) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68w-0005xn-C4 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:52 -0400 Received: by mail-wm1-x332.google.com with SMTP id 22so2474749wmg.1 for ; Wed, 24 Jun 2020 07:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=Zy5uGhZbVrUE8s9R5MwQljmeTArI2h6EuIR7XdyOUQB2h42SYTpt+fTHzwLIAxx9gA RazQPygD9QjKE8ReAH+2lo/+RcbihCXmbhIIzeNMP8KgzB9AcY3SHqGY+rASAmlDf8So 3trIBocAW6fjoDQIq3FlYX3YGCu+aHrJYMaK1h3RlIT2CNiEfDt9FRKwVRDlZrhODdai w6THDSkse2iaMdF9IyIpDI0YTA0GwZMpUlTk8tQvFzv+UPh/TYmFGyEjISETJ4hiwgtL vZ2OkrYWvURyseBtKoAOFjdCMRfuu1QuZFiR27/hW8uhYvzPqdilDyXCOB0gATO5dInG +8lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4JsqTbe4nx5Y4ze+RsCvycmftPXZdnpOMBPVnwlH7Pw=; b=YJBFVT6ZrGeTQ11uadxWw+q0N9PY25TB7Rxwi7w/1xXzg8+UX2T42fOQaJZT49LZpy VKB/NFUBra4gg/1haHsv31qjg3WWeLHwyCc8NVLOjTiLqZinUy+9fLZpLDUMMLO+adNN BaMEEDMoqIn/K1JTnJd8c0u74sQOBy82Q0N2D34rn5XmBvgK1oziDy72/M9WosEOTuJp vX3mOkNvvFiQELTepo5icp9NYsBp1l6GOQvfoD9JfIiKikq1bLqiIM1KgYgbThMup0vw RcaEc/w+skA5pzdXXHEAyJbPN5cjdqgEbh5ind1vwex4Yvk+c2jKiXk0sA69HaVandhi dMKA== X-Gm-Message-State: AOAM532Lho1lF3XFm9CxquzU/i4OiAfqSqQbtB1m7ZPWCQGZs8bmcyOI pvOYATF2j2666tm7JTRPJ5GIIQ== X-Received: by 2002:a1c:1d04:: with SMTP id d4mr31544499wmd.156.1593007968819; Wed, 24 Jun 2020 07:12:48 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z16sm17596847wrr.35.2020.06.24.07.12.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 8804C1FFAB; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 22/25] gitlab: add acceptance testing to system builds Date: Wed, 24 Jun 2020 15:04:43 +0100 Message-Id: <20200624140446.15380-23-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As part of migrating things from Travis to GitLab add the acceptance tests. To do this: - rename system1 to system-ubuntu-main - rename system2 to system-fedora-misc - split into build/check/acceptance - remove -j from check stages - use artifacts to save build stage - add post acceptance template and use Signed-off-by: Alex Bennée Message-Id: <20200622143204.12921-16-alex.bennee@linaro.org> --- v2 - updated with danp's docker changes - use needs instead of dependancies --- .gitlab-ci.yml | 65 +++++++++++++++++++++++++++++++++++++++++++++++--- .travis.yml | 23 ------------------ 2 files changed, 62 insertions(+), 26 deletions(-) -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a7abc55a5c6..7fcbdacd9e9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,8 +1,12 @@ +# Currently we have two build stages after our containers are built: +# - build (for traditional build and test or first stage build) +# - test (for test stages, using build artefacts from a build stage) stages: - containers - containers-layer2 - containers-layer3 - build + - test include: - local: '/.gitlab-ci.d/edk2.yml' @@ -24,26 +28,81 @@ include: ../configure --enable-werror $CONFIGURE_ARGS ; fi - make -j"$JOBS" - - make -j"$JOBS" $MAKE_CHECK_ARGS + - if test -n "$MAKE_CHECK_ARGS"; + then + make $MAKE_CHECK_ARGS ; + fi + +.native_test_job_template: &native_test_job_definition + stage: test + image: $CI_REGISTRY_IMAGE/$IMAGE:latest + script: + - cd build + - make $MAKE_CHECK_ARGS -build-system1: +.post_acceptance_template: &post_acceptance + after_script: + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat + - du -chs $HOME/avocado/data/cache + +build:system-ubuntu-main: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu + artifacts: + paths: + - build + +check:system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build:system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check -build-system2: +acceptance:system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build:system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 + MAKE_CHECK_ARGS: check-acceptance + +build:system-fedora-alt: <<: *native_build_job_definition variables: IMAGE: fedora TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu + artifacts: + paths: + - build + +check:system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-alt + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check +check:system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-alt + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + build-disabled: <<: *native_build_job_definition variables: diff --git a/.travis.yml b/.travis.yml index 74158f741b1..c24dfbe377f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -289,29 +289,6 @@ jobs: python: 3.6 - # Acceptance (Functional) tests - - name: "GCC check-acceptance" - dist: bionic - env: - - CONFIG="--enable-tools --target-list=aarch64-softmmu,alpha-softmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-softmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sh4-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu" - - TEST_CMD="make check-acceptance" - - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-acceptance" - after_script: - - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat - - du -chs $HOME/avocado/data/cache - addons: - apt: - packages: - - python3-pil - - python3-pip - - python3-numpy - - python3-opencv - - python3-venv - - rpm2cpio - - tesseract-ocr - - tesseract-ocr-eng - - # Using newer GCC with sanitizers - name: "GCC9 with sanitizers (softmmu)" addons: From patchwork Wed Jun 24 14:04:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191665 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp631562ile; Wed, 24 Jun 2020 07:13:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJylUz4YYfqsXdmGTZfOV6+3rwhqrd+GzTL97ROVRLkp/16lJuEPDVrFjEfAy4QbkvWjdt6M X-Received: by 2002:a25:1045:: with SMTP id 66mr45570070ybq.314.1593007982083; Wed, 24 Jun 2020 07:13:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007982; cv=none; d=google.com; s=arc-20160816; b=zTydQUHvgVpBZiq/gE0ckd+lm1WZHOa5ylNZaWDqOp51NDOfnpsEVh/qS0JI9rE5y5 ibjWQhEchZK7I8jckO3t+jk2y7XPcXIhtIVx7MKissZCiVu9bE1BE/r8EVLnox30oS9M /xRzMx/y0okZqet2gy5mWOCGCT2tn5ZQYBP89POP9bbj3xrp0CIO0u8xrJyF4Vwb5O55 qjwgWYdYXNWnXU4clQn88awsGeJWw/eO4kFLhsKmVuWeeMTF+XmKnQM7xkS11yWkOCph quQ3FLClxrp2/cN4nStdUePZVjv89nOEWrYXtWrNbLoEXpqBhQ265CpQILZ3exIdEgr+ FKNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=FbKyVnpTkBOlvbP/LJyqgvrB7RXPPzYwzSzwbY7+na7/ihMowgO39PLc2lDiekf/MI 4re7jR+Zjx6ajxOB7+nLp+qtU9qmZJoGkKUeQnRnOYJ5KAL+7q9W5HUtAmoPCeed8QSx DhVig3J9bMNmw6MUohrJRrhDFGRHwilfQzZcXH6/zSt34Tg69ZKFnoUYCXAsLMqAQV0K 4Ni6qw6Z/d/fnv+4OF+4IFUmyfhtya57H+vrcpiJGr0xQ0tEJo4GhzoL9rcWUPbddz/x a5MB2LPLYKW+fjO6bLbh58ytE1ptr4k9Ut4JqdOCB+NKAQmlbP+4duFy3llvNQibWshc gVVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=nW6IQeiz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o23si18342783yba.275.2020.06.24.07.13.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:13:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=nW6IQeiz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo697-00049x-HI for patch@linaro.org; Wed, 24 Jun 2020 10:13:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33478) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68s-000459-Fl for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:46 -0400 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]:50429) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68q-0005ui-P9 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:46 -0400 Received: by mail-wm1-x336.google.com with SMTP id l17so2482674wmj.0 for ; Wed, 24 Jun 2020 07:12:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=nW6IQeizoebZzBBnUvlFkI1djmJVrNzyP9KYRi1BhllJgQsA31/tFl6+Qeq44yCpli RjUN7LkGuoYegmsqR3LyEJQlp/lRkyMqp6L44TpAdymaueX/+FTxPrGUPrbr7CMCLInR 3IXGibs2Z6tOvvYRMzriOwI77c0kMHZPTqf0CSVoQDCun5PeBTq4k0L+Z6dGe9SGlffz x1yN5T+REERB1tQWSos3GrjsDBx0+JVfc6FY9gsIX1IgQcULu/YMNWyMJQBnLtCnm8uV OElPuWU7e8z/fcKxfbSVYHzm23Q7RAoj6H1ZTqYMU8TpTtOsFKXOgZus9af+Lob8yYm3 pMfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OTrZxr+gr7TRXS20xbwh/wb3Pjs8p21fdYjxEPsPxt4=; b=YOZlFoY050RetW0RCy3o8bdBuVoQ4KQoRPJwdk33cRf/do3Oo5nvD12WNiiKfnGrqU XsbWTdiEXJ6PB11YPP2CwKLEa/n5wimcNHfPHQmZvtdckO8nshCN8bnwZTq09EYpwX+K cqg/h3k1sGVUKEJD4n3lB8B7ZrLvnBSj7GpDrG2dA2ri7+HhTn+gQa8jnvKngse6uvnn rA1+Ek+eM1pbHHWQ4VlIAH7jdzrJJHDnEW5jGWn26ldFZmDM7z2qpq5RQPVld+ympfTx 8tuaW8kwOZyvtnnOFWd5SpHVct9s3emfA27SIFEGK5Yp1oPbpgOG+wggAfgA2C3B+KCh Cqlg== X-Gm-Message-State: AOAM530kbjG1BtK/oM9B8dlsXoRHGwOb/AUoh+TcBn7JMaK7+mTByBuD J/Wk/doWsQqBgkGYG3Q3/vq0kg== X-Received: by 2002:a1c:c90a:: with SMTP id f10mr17217993wmb.121.1593007962417; Wed, 24 Jun 2020 07:12:42 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id s5sm8358484wmj.18.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 9EB621FFAC; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 23/25] gitlab: enable check-tcg for linux-user tests Date: Wed, 24 Jun 2020 15:04:44 +0100 Message-Id: <20200624140446.15380-24-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::336; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x336.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We should have the containers available now. Signed-off-by: Alex Bennée --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7fcbdacd9e9..54da29dd384 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -142,7 +142,7 @@ build-user: IMAGE: ubuntu2004 CONFIGURE_ARGS: --disable-system --disable-guest-agent --disable-capstone --disable-slirp --disable-fdt - MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + MAKE_CHECK_ARGS: check-tcg build-clang: <<: *native_build_job_definition From patchwork Wed Jun 24 14:04:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191664 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp631512ile; Wed, 24 Jun 2020 07:12:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy/yFfqbrUMvMMXG7FbrL3P8+l2v9UohdOSvVXRzkP1lgcgBmHTkIDhjWLWkYeurQIX6l11 X-Received: by 2002:a25:7003:: with SMTP id l3mr43705105ybc.380.1593007978507; Wed, 24 Jun 2020 07:12:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593007978; cv=none; d=google.com; s=arc-20160816; b=hDxJjpOld/a3pqecq2gFBRNYVx3HPUFK7ivxWvHHWEYsH6I9Lc2E3oxJMsq2hHyo2+ +FBZhldMUipcO1pOIn+YE9qB4j0RAwWpU41oQZIJzsoUB/irKyGYLfuRyPORdBNvQEF0 VzGddNUO+KDKlF0muRwl0tKbPytT3/K2YhiZcHw5rFXqYr6u+6ZfPXRp1RfAla/vVXdp L9kjAnNF8fYCj8HH+IM/RGD4ex7beKDnksEBMsMB/9msSvV+/hs+4GEi1M+TmgxiRIPp +Kdp+tKKLX9Way50mZQc6a4KWWUiSxPRSvrERqszTEIMdbXPTQcr/ilwGP2JdMJzSh3N 6DOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=myDxXIAy50g5v6cIa2PTvMHUFyHMB12i3slAkfRwlhuwayCt9KadDlcwtS2bEuyXy/ hguAPtvQL0ACABKVs4ZbHFgyxfVu3aB22Rxi4BIw8WoSwq508FY/bXE3Lo6KPubiqzci xKb9foqKoo2uRGHMOW5EvYrQp1lONHarzD/WJDGAl64boqqkmiKgRmp+pL6zhIJcxZdT 5PIUdnYno5LZkNdBwxWy8SvZCeIZM4+D/udELxGa3vuoQx+LPyg9CDif6IU+NmbSiftZ tLRMFcm4Zm7wVS4OwziXLnLT/56reH55FFTqiWY9QFm+0eig5qGbE+9oXMuoQ8JWMxCS aFfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=U5+yr8Ah; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 193si20130857ybd.490.2020.06.24.07.12.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:12:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=U5+yr8Ah; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo693-00045e-VO for patch@linaro.org; Wed, 24 Jun 2020 10:12:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68r-00043l-6d for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:45 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:39624) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68p-0005ta-B3 for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:44 -0400 Received: by mail-wm1-x32b.google.com with SMTP id t194so2677415wmt.4 for ; Wed, 24 Jun 2020 07:12:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=U5+yr8Ah+wdD1Tv5Yq6iMv4KxqWlivSYl+AqLVsBoTZfVMuIeoharISrhSuMY0aEXJ 0dnXh71iWsL/5A/dvwAQO1uElLHONHgcCnBtRTzavwMKuWQcZi04pT/alFvPHtRoOlDR /Q4bqaaxfhG3F9uZJ/BqYB3tFvLqTTRYQywLAV/nxs5RYG6AgREsT9sZOPuKL5pZxnhO qJFb5MSSMN5G5s4GhWN3QHzwX0XgWdgtGGnl3G5FMJhZWVQ3eJk43gGA5NR358GmP6nE LkQkNp62ARw1R8TJ/SrocCkCaLzNEtMg2JFPRrmbN46rbV10xyjyqI/1KppP5qVnz7YX MTcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkJ3TlSVKMVotvh/AON95M2rOXL6DLcXLqurDZ1ML4w=; b=HjIEsCoVKSCQZLP+7R+YPok806vxdv5fsT/782+CN3I2XBJLz+Fv2o1JMbf7PmYswM zzqryswfBbIaDmOcA8kUiRq3VG275Bb4iqBN3/XzvRJxaqn4KHbncxaeiBcbfRROjSOg IZJDFUwrAEbEcy9uYYjH8HjYyoDbKIqTuMUYv+Z1kf4U8OGyOoXrHaMICCsL1APCHRjR MNiyT7a2AFRlq5Sl0ULbUrhlowvpArHk7Cygi21kdfW1uS82SGjy6lOegIF5atuvrb7D DvnM8h/H6PKRG77E6MgTHhOxS6ToEm6X/6f3kM6T0O5uYKYZIIBlQsRHYTa8cKF0tUP8 p5tw== X-Gm-Message-State: AOAM532aJPPEoVO6eUrnsNwQV0hvOy/n5Bm7Aa+E2w7jxycNSJ0j0MM5 FTXhmBif8B9VhhmEDnjRgNuQew== X-Received: by 2002:a05:600c:2144:: with SMTP id v4mr19647966wml.128.1593007961346; Wed, 24 Jun 2020 07:12:41 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n16sm20955950wrq.39.2020.06.24.07.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:40 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id B500D1FFAE; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 24/25] gitlab: add avocado asset caching Date: Wed, 24 Jun 2020 15:04:45 +0100 Message-Id: <20200624140446.15380-25-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" These can be quite big so lets cache them. I couldn't find any nots on ccache in the gitlab docs so I've just ignored it for now. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200622143204.12921-19-alex.bennee@linaro.org> --- .gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 54da29dd384..683a1d90fe6 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -8,6 +8,12 @@ stages: - build - test +# We assume GitLab has it's own caching set up for RPM/APT repositories so we +# just take care of avocado assets here. +cache: + paths: + - $HOME/avocado/data/cache + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' From patchwork Wed Jun 24 14:04:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191674 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp636378ile; Wed, 24 Jun 2020 07:18:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyo+nurYC+2G2fmixonWwvse52FhnMJCoQiuNtZOy+dI6omYffldiafjVg1rJSkcH3hGo/0 X-Received: by 2002:a25:cfd6:: with SMTP id f205mr43295952ybg.363.1593008327832; Wed, 24 Jun 2020 07:18:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008327; cv=none; d=google.com; s=arc-20160816; b=cBOQYvRkWCwjBpK3VbBrp1BbBDV4yZPi3R1q/OX3BGWNP50O8s9cFSG2jmsc+3iQyr Aks3evCmU7YNImPqDeog1rejPpfBsU6pyLJCWjTmPzzlgnwyGIv4M0ekePomWJd/Iq+3 xTsk8syvEJgGGWy+rJBdft371PoQMC+irqksbWfbt9r6Jm3E48i4Typ7QWv75qa5K91X xY7ff525TuOI1vy60fA+e3SL7+VkvxTzrREPahY6R7R8Lcgfk984EBPYbVOf1dK4Z4lc S3MDYy3TtiiYjfYaqkJ/J+pKcxp5RfwJ/0fnxuyp0kF3+EtPcSpb8F5YogsY+M9Iheox mBGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=oZnSlt8cWGFo0Wne2z+RoYPPer0/KpS737MRD8Y8AFL7lXxmSCUALL/GQNOX1BYi3F /cOViOBGo5OyKlWePlPADGuEqj6W0ub41oLUZ/hWQlDf8ClDIUdeNGOC+CTcmKM40oXT m7juswqJAPIVNjODZlHV6bzx/5erSORNzK5uzAR2koy4GWlqV8vmi2PsvciftztStTLS HqLZB1ZgrYa9rG60SvRHnIEyWacVi47n9IvxYNKwMMYgc8Fqhd5S3cYvQpXc5I+/Rrdi as3IHKK2uk0iqi9sgC7nC10GvYrr7bcbbslS17bnuYh1evJrp5MeLWJldIBkcOqCE7dz HjfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xf4HwYVQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j1si20017009ybg.130.2020.06.24.07.18.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:18:47 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xf4HwYVQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Eh-0007HY-BC for patch@linaro.org; Wed, 24 Jun 2020 10:18:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo68x-0004Ec-1Z for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:51 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:37848) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68v-0005wr-8M for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:50 -0400 Received: by mail-wr1-x436.google.com with SMTP id a6so2457966wrm.4 for ; Wed, 24 Jun 2020 07:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=xf4HwYVQa2TwFiK7FK4SfJ7rbA0WfBzMSdNwuq9AtPmuET4wwmYI11fqOhMzGhgZbT nhn2iGl4RX6CU4bLiWV6+BJ3XacG8upCIgnVe8Z3vP1ZnLXT8mMkgFUa4ospWZw30V0v Y3rtZ1hlfIywtrzWgaKQeqE03iZmpmMho5ayD2Uuh9sKoCPgFa+pXS8xMcg51CJxsfXF cKFpop+2G5uf8VfKrK5rkOjRi7FnSaH1yJgweyH46nH9kQCiZsvU5Mgdms7xCKlrMpew cxu8Y8G473nSrJwfhodkRgUwJn2n4nHv0IyYtuc5ElAqFqsT5NrCCU2WdSfFw/e3OiBU 5LtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NSbE/oPaUPqzCfEUFHpvKz9daAgMJApN4FMtaAwBhjs=; b=O6GiJ73TINB5rR6cN/CcbAfURRZmTrppPnjCuF4dMKsYkO5yDOznUxflejpIdBEgSM aHojChJ+JMSxlOGvs6UZOLCrVmM4yNbh6cGIuevC3RgY1d9GfHUr6i98Ys7LP58XXr7l Rw+spELqeT/A/foA6R1tVpDMdI1VY5C8/VoQKqv5NfPf8pDf4O51okJlF/eU6LgRULSL kEXb8gRF1hQ6xzMkEwj42nuv1cv+Gcxggthvu2VHB5lPm6hvMYr/YYFdXE0bax0LpJSd cH/ibkaqGeGiVLx5hrh3gT1A+zxO5zSdWb0jKukF8sAIJBYQLZsSherofKLESIGhpOeZ MnIg== X-Gm-Message-State: AOAM531P5DqHwTDU4ZpcJ89UbZtlKbV4KYOHs0OxcYbSb42n3r5iTtuK jmPTDK6bElI5FY8sjt5wnxqMVA== X-Received: by 2002:adf:f14c:: with SMTP id y12mr30961671wro.30.1593007967876; Wed, 24 Jun 2020 07:12:47 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u186sm8568589wmu.10.2020.06.24.07.12.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id CAFB01FFAF; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 25/25] gitlab: split build-disabled into two phases Date: Wed, 24 Jun 2020 15:04:46 +0100 Message-Id: <20200624140446.15380-26-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As we run check-qtest in "SLOW" mode this can timeout so split into two jobs. Signed-off-by: Alex Bennée --- .gitlab-ci.yml | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 683a1d90fe6..2567c88f99d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -109,7 +109,7 @@ check:system-fedora-alt: IMAGE: fedora MAKE_CHECK_ARGS: check-acceptance -build-disabled: +build:system-fedora-disabled: <<: *native_build_job_definition variables: IMAGE: fedora @@ -120,6 +120,17 @@ build-disabled: --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + artifacts: + paths: + - build + +qtest:system-fedora-disabled: + <<: *native_test_job_definition + needs: + - job: build:system-fedora-disabled + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check-qtest SPEED=slow build-tcg-disabled: