From patchwork Tue Jul 7 07:08:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234909 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp651938ilg; Tue, 7 Jul 2020 00:09:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwhZ8B2G2M3bILdl5e4Lo15EqEkmL+KKNlX/QHiLot0NEtP+vHt/75Rhgg7kHw0nP7jgb42 X-Received: by 2002:a25:2f43:: with SMTP id v64mr74104078ybv.463.1594105786941; Tue, 07 Jul 2020 00:09:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105786; cv=none; d=google.com; s=arc-20160816; b=B361HxpwdyqeB69tYhxzTL9Dq1YWplvZiiDyHOs0lf0VM4ugyh57sVY231LD3bzQd1 676a7HUayH2HW2P1xx+gqYak4aoE22PDhh+mJDeT9aRkE+nfMVYnmqkw7bBHjmlOHjgt vCg6cqCkgKrwLALV+1U/W8VDUMSPY1rIfg1/hFDPeI2Txuw0yK87IADtaZoB5AU45LqZ AxQeNlP4HZ7TjrXYZHKAoWm9Y5smCoEUdE/44Bksw5iwicT7pdT3Du31rxlW1bB17lb1 Aw9ESIeM6dCyY5xezSDimrB0Vq4b3mD4viHF9jUNcVsMHM1WYx9qGgJPU1qz4udFnTdl l0Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=qixSpKSHjOkHks026Zc/v8eSQoIoi5iJ+5J0KAQW+wU=; b=mtjK7EkRJAX+TGE8u/IRK4ix7hKmVYcly+87clxPjssHaHEaaZq7SkeplT3asauUTc yRPXU1dAjaubLTrUdz6T3eOLH3E/jCBZoECxKPuZZdsIo3kZ8hfqCq148B9WTtiFENZH cosojs+dFhefl+RoELVIukBZswLEy8YSmMbdzY4hIKNPvMfnPA8n0iRYoKhreP1IaBoN BesF5PnvHQ1Vi3AZyj+7Z1NDWN4TtaUN/h97LnEWppovipJQD6FT5OdkLcCmOaZSSaqY II2rQND8IP+Iq3gLqZMcf1g/Q8RomPe0qSHeegERRsKy0234jYHIISw53xrO0Xn8eLP8 ltzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iFZaKWSp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r204si23655799ybc.324.2020.07.07.00.09.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:09:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iFZaKWSp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshje-0004JK-DJ for patch@linaro.org; Tue, 07 Jul 2020 03:09:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshiy-0004Fa-GD for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:04 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:45511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshiw-00042q-An for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:04 -0400 Received: by mail-wr1-x42c.google.com with SMTP id s10so43922621wrw.12 for ; Tue, 07 Jul 2020 00:09:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qixSpKSHjOkHks026Zc/v8eSQoIoi5iJ+5J0KAQW+wU=; b=iFZaKWSp972Lz/X10ZTm7kdbn6L+KSALeLcK5TEXFmI+KXHOpJ/KZFWcnOzwQ375eI mkxnOjwbkBN+U+cDn8wNslwj51xtwGM0+TPTX8Mj67k6YaPzbZUu7PXjWnTm8MZAiRwG ipqOx3H7xD4l2//WtWodSYwPuAeS7uzODTD6F2dEMr+QuXy3L3IHA4PPKJf2s+7fBOgH 87EIvtToJKn1esxvbtPOt3+3FCNQi6QLH5lwXoLCP4yCEbqOM8N1IhWyOe4frgnnouti jEw4oJ9VVrsV6LLtg2XDBVbS70dbeQ8o/sHto+EFXeLVjNSXG510EI8vHt6vDdS+4Y3P LH9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qixSpKSHjOkHks026Zc/v8eSQoIoi5iJ+5J0KAQW+wU=; b=Ve1ghbCk1sVISoWJ7z0naIsXiDRLfnds8hZIOUqttAD+1TDk5AiGHTMT40MVeIfmFD LTWw2kbp6ioXi3BvensOlzUsTcQbJ8o+Gy8fTZXvGn9PIE5zn/cLk2OdTFDsRGvsLSll qIcg6Q5xBnyjx6acWXEhAcNukWucyKpdjyyxemjl6vsg+AymA1cNwkig9DG8qR85xBeh gGH2FeoKqyT6creZmnZEOSJZOG5scudu4hBc8p2JMJQE1FRM0yfp95oIwJHm+De+yT2Q E/r1j70Ii0RaU4WI96uglJUWRkIVCUEKuJkcBR3JehrqQ4Uoz1qwYCORl2OEyIoBB1+o zbbg== X-Gm-Message-State: AOAM531rsgvj2SyI4mqtwcjs8Puv3fggiw2PW1A9HjkT5bc5QGnpTdFg F8h5YaRj+7eZzqp+ieYtXBMnXQ== X-Received: by 2002:a5d:5084:: with SMTP id a4mr54155154wrt.191.1594105740565; Tue, 07 Jul 2020 00:09:00 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n14sm27971011wro.81.2020.07.07.00.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:08:59 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 920D31FF87; Tue, 7 Jul 2020 08:08:58 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 01/41] crypto/linux_keyring: fix 'secret_keyring' configure test Date: Tue, 7 Jul 2020 08:08:18 +0100 Message-Id: <20200707070858.6622-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Edmondson , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: David Edmondson The configure test for 'secret_keyring' incorrectly checked the 'have_keyring' variable. Fixes: 54e7aac0562452e4fcab65ca5001d030eef2de15 Signed-off-by: David Edmondson Signed-off-by: Alex Bennée Reviewed-by: Thomas Huth Message-Id: <20200618092636.71832-1-david.edmondson@oracle.com> Message-Id: <20200701135652.1366-4-alex.bennee@linaro.org> -- 2.20.1 diff --git a/configure b/configure index 8a65240d4a8a..83e36c270cce 100755 --- a/configure +++ b/configure @@ -6461,7 +6461,7 @@ EOF fi if test "$secret_keyring" != "no" then - if test "$have_keyring" == "yes" + if test "$have_keyring" = "yes" then secret_keyring=yes else From patchwork Tue Jul 7 07:08:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234915 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp653094ilg; Tue, 7 Jul 2020 00:11:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVtk5tABvcY68gGXzpO88k63UvB4pbXNzMdGKjJUAYx7EpeXp2gqGIDYdWFiYJsojmFyu/ X-Received: by 2002:a25:56c3:: with SMTP id k186mr57480718ybb.183.1594105892322; Tue, 07 Jul 2020 00:11:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105892; cv=none; d=google.com; s=arc-20160816; b=UTWVPKxwSxRWklM2Pw/MMrR2KAv2PD/977JLIlVWesDp/Lzb0JE0qx2J8mwQ0nYiAs X75C/EgLVdcSCzq7JShH+tUePOLZUArgFUQyjAgV6jgHTp3VpqepKHSwOgLBtwxEIM+A cZWox/NopSCkjqyPoeWy0UzjGGUpEZQGsdLqABxGGMgQkH6t/TQaN/UKam+IbZSGk6Ds KHqH+IWjMkv+MoD7nOuRS5sYToFcgidT+FIP5ZP1HsIy9TH/kZNHSdwWDILi43Gl+ZBH THj1wCf2ti5xhNYTnAOycJyptkdPeUnS/hEnFxLSOcc5Lj+DldYAKfNC4hnifHTihl5T Rhyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0qqS4w4mRnCRDCTfWQqRgg26scMxtoAIPpRe2SvN8Gk=; b=Yqr1lXV468s06YRNUS/0TB4+4OZ3RcD+XNvKPJtLSvZqMnqvtwDM9hkqh/cEoZCdri uLxzGP5YVYm2SGf6C1UQOujIx76T9Su1f2lEw9Swk1xQbeL2aYwkBcNahfhy3icrXrSi YmUi2+Ip4Ev/KBtit5AHus1gT8/qpM7E7nXCStpeVIdHpykqr6Wg46DLuZwyGKPzhEEP NgxoPY4Y34Mze/DQjk0/R99i0UIYEqJ0Tj8iG+kUXJV1rcEUQoQQPnnImuE9zl9Vj+pR BaeTFmvskRRPhuUuUcnI4yPKzJvOppkeEiLqKLSbgufPh7TdScrZPu32sWQANmMn+TMF Y2Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="RQeNH/EW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t45si22693966ybi.478.2020.07.07.00.11.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:11:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="RQeNH/EW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43102 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshlK-0007tP-Vk for patch@linaro.org; Tue, 07 Jul 2020 03:11:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42462) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj2-0004PW-QW for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:09 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:34549) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj0-00043W-W3 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:08 -0400 Received: by mail-wr1-x42c.google.com with SMTP id f7so40959475wrw.1 for ; Tue, 07 Jul 2020 00:09:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0qqS4w4mRnCRDCTfWQqRgg26scMxtoAIPpRe2SvN8Gk=; b=RQeNH/EWZqgdFsCUzh4H97+zKDBdI28lkG6B4qicotvqO80bI/hyKfy1nMuWVrCy25 vA7ijNBi00ddkc1DzKSKhvapsBJBXXHWj/6as2U5/WQsPfJCp+S5dN0nkUarUR+Y4V2w r/5Ml5kiS2ZH/STPZnDNEkT8ZmRwR5+lMxB8W65RMsXxwPQQ1zL1ex/SUHHug+t818n1 19mnHstgArAs5Wpsp9XABoKreF1DdPRN1zp2gULPCBbYNCvQLJ+mR2P8p8RdXxBPlokP 3WUOlSntttjtd275XDqcM6iwby9i4Jn6kj00X+nmqHSKf0DmjzbYRwn1ncPrWaBy1ofi aPIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0qqS4w4mRnCRDCTfWQqRgg26scMxtoAIPpRe2SvN8Gk=; b=leolTC7Zdcjp8jUYrJNKeTMLi4DRAe4PLqLlwix3kB9/T7YuYikH50/tiHeEeAmFKP 0JiKlYHzsjnkBZwZ1S9FBo5J9QDZjBEyMh4evIE6M/1mvTgg5fEhsH6L9VDAUo9qrj8q uHBehSE+WnwDP+8I40hLpRMxeazXzTRyqzGijhFDi6iOAfsspTGDG1zZ9sFT4uA6p1c8 hgkmzxcJ09kAxasZPf59jGRelZt/1JwVMgS5bP/U40DSGqCTiZIxvtcrOkjeYTz0Jia1 Q4JFZBQwneE5JZmLOzXLvvi7rjlmQACvq6Vo68iqs7wSqT9NnpmrJojwg3TjUs1rrFl7 8fLQ== X-Gm-Message-State: AOAM530ol0msfAa/D/+1gsDn9nSQXd2NykucXYQXs8VJWAkixjBQHWXr tgeY3Ebl8PC0tp5t6+vi8dzSXw== X-Received: by 2002:a05:6000:1c8:: with SMTP id t8mr56219990wrx.73.1594105745480; Tue, 07 Jul 2020 00:09:05 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id o9sm26892564wrs.1.2020.07.07.00.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:02 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id A638F1FF8C; Tue, 7 Jul 2020 08:08:58 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 02/41] util/coroutine: Cleanup start_switch_fiber_ for TSAN. Date: Tue, 7 Jul 2020 08:08:19 +0100 Message-Id: <20200707070858.6622-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Stefan Hajnoczi , Robert Foley Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley This is a cleanup patch to follow-up the patch which introduced TSAN. This patch makes separate start_switch_fiber_ functions for TSAN and ASAN. This does two things: 1. Unrelated ASAN and TSAN code is separate and each function only has arguments that are actually needed. 2. The co->tsan_caller_fiber and co->tsan_co_fiber fields are only access from within #ifdef CONFIG_TSAN. Signed-off-by: Robert Foley Reviewed-by: Stefan Hajnoczi Signed-off-by: Alex Bennée Message-Id: <20200626170001.27017-1-robert.foley@linaro.org> Message-Id: <20200701135652.1366-5-alex.bennee@linaro.org> -- 2.20.1 diff --git a/util/coroutine-ucontext.c b/util/coroutine-ucontext.c index f0b66320e102..fff20aad80a4 100644 --- a/util/coroutine-ucontext.c +++ b/util/coroutine-ucontext.c @@ -52,8 +52,10 @@ typedef struct { #endif sigjmp_buf env; +#ifdef CONFIG_TSAN void *tsan_co_fiber; void *tsan_caller_fiber; +#endif #ifdef CONFIG_VALGRIND_H unsigned int valgrind_stack_id; @@ -77,7 +79,10 @@ union cc_arg { int i[2]; }; -/* QEMU_ALWAYS_INLINE only does so if __OPTIMIZE__, so we cannot use it. */ +/* + * QEMU_ALWAYS_INLINE only does so if __OPTIMIZE__, so we cannot use it. + * always_inline is required to avoid TSan runtime fatal errors. + */ static inline __attribute__((always_inline)) void on_new_fiber(CoroutineUContext *co) { @@ -87,6 +92,7 @@ void on_new_fiber(CoroutineUContext *co) #endif } +/* always_inline is required to avoid TSan runtime fatal errors. */ static inline __attribute__((always_inline)) void finish_switch_fiber(void *fake_stack_save) { @@ -109,18 +115,29 @@ void finish_switch_fiber(void *fake_stack_save) #endif } -static inline __attribute__((always_inline)) void start_switch_fiber( - CoroutineAction action, void **fake_stack_save, - const void *bottom, size_t size, void *new_fiber) +/* always_inline is required to avoid TSan runtime fatal errors. */ +static inline __attribute__((always_inline)) +void start_switch_fiber_asan(CoroutineAction action, void **fake_stack_save, + const void *bottom, size_t size) { #ifdef CONFIG_ASAN __sanitizer_start_switch_fiber( action == COROUTINE_TERMINATE ? NULL : fake_stack_save, bottom, size); #endif +} + +/* always_inline is required to avoid TSan runtime fatal errors. */ +static inline __attribute__((always_inline)) +void start_switch_fiber_tsan(void **fake_stack_save, + CoroutineUContext *co, + bool caller) +{ #ifdef CONFIG_TSAN - void *curr_fiber = - __tsan_get_current_fiber(); + void *new_fiber = caller ? + co->tsan_caller_fiber : + co->tsan_co_fiber; + void *curr_fiber = __tsan_get_current_fiber(); __tsan_acquire(curr_fiber); *fake_stack_save = curr_fiber; @@ -144,12 +161,9 @@ static void coroutine_trampoline(int i0, int i1) /* Initialize longjmp environment and switch back the caller */ if (!sigsetjmp(self->env, 0)) { - start_switch_fiber( - COROUTINE_YIELD, - &fake_stack_save, - leader.stack, - leader.stack_size, - self->tsan_caller_fiber); + start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save, leader.stack, + leader.stack_size); + start_switch_fiber_tsan(&fake_stack_save, self, true); /* true=caller */ siglongjmp(*(sigjmp_buf *)co->entry_arg, 1); } @@ -208,10 +222,10 @@ Coroutine *qemu_coroutine_new(void) /* swapcontext() in, siglongjmp() back out */ if (!sigsetjmp(old_env, 0)) { - start_switch_fiber( - COROUTINE_YIELD, - &fake_stack_save, - co->stack, co->stack_size, co->tsan_co_fiber); + start_switch_fiber_asan(COROUTINE_YIELD, &fake_stack_save, co->stack, + co->stack_size); + start_switch_fiber_tsan(&fake_stack_save, + co, false); /* false=not caller */ #ifdef CONFIG_SAFESTACK /* @@ -287,8 +301,10 @@ qemu_coroutine_switch(Coroutine *from_, Coroutine *to_, ret = sigsetjmp(from->env, 0); if (ret == 0) { - start_switch_fiber(action, &fake_stack_save, - to->stack, to->stack_size, to->tsan_co_fiber); + start_switch_fiber_asan(action, &fake_stack_save, to->stack, + to->stack_size); + start_switch_fiber_tsan(&fake_stack_save, + to, false); /* false=not caller */ siglongjmp(to->env, action); } From patchwork Tue Jul 7 07:08:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234910 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp651939ilg; Tue, 7 Jul 2020 00:09:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwp/Iyf2mRSiboqZ1N9M/C9jVQr+kY1BUaL/81w3vfciC7upxw77zxMbPZBInZzaJ4kcmqA X-Received: by 2002:a25:3f87:: with SMTP id m129mr89535794yba.371.1594105786948; Tue, 07 Jul 2020 00:09:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105786; cv=none; d=google.com; s=arc-20160816; b=y1U1lK1+3qyOaKvTTuTqOxIUAKV41T4847Fa9MP9AbxrV7A+o9YUGuL2BEt2GlrIjY lb8WhHuTfqGKedB8CEIbGL5V99DPEh4UB4sO/BHyMeN1EV+l1GS4f4AKIsIYfE8iC0/Q U8dQ8tUdasr1fx1ToPideSIChv003somBnNkPr4jxLCujIafm75fEjJlF5mEJajDL14H DyOYJh4Bs7B0tC4WwIM6MYikKhUp2lpa41ZCiZ0mOsKIE+tUiD0Hks8w/1a12McU8Ljo uDk0duSL6VJ1FCL9J/47aXxb/y/p0QqqpvOpWcxNlQz8ihANWdFKDk3bAm43ooEQkR7I FaRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=f+mo9JyjAOQaUqLAqsYtfO14+eY9TpKX1IQ4XQcaJvA=; b=yw4dCk4glR3bZauP9zSh5HkOf86jL0CycOGQdXZspgvG0GGo82Mum8q5CCVZ1aciMO FZw8M7VmDiZHqRXgJJ1qvE1VnRdSWeJiHdlwy6GAGL01vdToFYTSO/XFOo6uF2JWsDOc EiknMKNpHzBdCiH9WYCrbD6prrYOb7fqEMSdOx7QNP/TOZviZqr49GQe+dTn1rFYT7e6 zx9grBZB2DFJjvM/jE9aGeohSvTgt+czVwsArl7zIdyH+O92w1ymzuwiqOmzoJaeSf5q Nu1K+pYLgCP5SdBa5IzvdTCWNTPkKqeBTnY6t06+fupKOvWYG6QTSK2u6/eyaOivWPhS BiQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=pKLMru5T; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x184si7276920ybx.325.2020.07.07.00.09.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:09:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=pKLMru5T; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshje-0004Nl-Dq for patch@linaro.org; Tue, 07 Jul 2020 03:09:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshiz-0004HU-7G for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:05 -0400 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:56224) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshix-00042z-F3 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:04 -0400 Received: by mail-wm1-x331.google.com with SMTP id g75so42097641wme.5 for ; Tue, 07 Jul 2020 00:09:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f+mo9JyjAOQaUqLAqsYtfO14+eY9TpKX1IQ4XQcaJvA=; b=pKLMru5Terw3NoNSYbVQESW5nNqn9aM1pUsdTDOL5pVY6TDXOxpTonURneZfTQJ7Ds r9kkzdQGSoM7vFhZTytWyOk9RrVSoiB6uVW4FJQSsGIEbPqPyqaZB1OmfacUe/ZzIyF/ 0HPnOSoo8TB8Shtmy8xn28BABOKmw3eE4UES6FgzJyVfwO+7+0uYwJKWLyzv59bU1cIJ GTxxQYxyXdlTVSo7p6tBra1dcfuejTWVHwCkHUsljujrwsXRsW70H7+6lG2SfgRyxBRL wVmlfVi1K89CFV+VdmGL02U0HSnA/cYHdESwYb3+BDF6PmJMgroMICxZyGgC4LwdhrmE oniQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f+mo9JyjAOQaUqLAqsYtfO14+eY9TpKX1IQ4XQcaJvA=; b=A+z7SkoG5bdrcxYHdg4VEwCs+GUoOYI2fVWIs8aIelLaSb1JiW1M/WOkNZ/V1sDALm MOl4oU6F4y47IyNXwD88/wRoQT1H/gWCM+otjf7gffj5/jTf6TcDl+KXMK9Eo1S8YG6l Cyp9qP8LfnPx3RQyxGZ4ArAB34pme/KKJmKqhLpCxANL3PylSaeUb1x2oAWj+ezF9+iR EE1jZ31KavXx+6NXkHMQXO1zzKe9iOgqEZuo7c7R/YQ8vaNezF2c6s4JIGv3ZQI2g17l huN4cPZqF+UpW1op3fiZJYZp/unu4M75UpgPnEUvOoeP7hoW2qkYZYguFR2KMOiL7iXA MhOA== X-Gm-Message-State: AOAM531PvmkUEXPEZARmoQko/KfZJgWGvLx5ydqylFb3rBoE9qZhtKKr h9xPFS1FV70OqNd+AXB9TwFHzA== X-Received: by 2002:a1c:bcd4:: with SMTP id m203mr2611987wmf.124.1594105741756; Tue, 07 Jul 2020 00:09:01 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id v5sm26480240wre.87.2020.07.07.00.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:08:59 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id BADF81FF8F; Tue, 7 Jul 2020 08:08:58 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 03/41] tests/vm: pass args through to BaseVM's __init__ Date: Tue, 7 Jul 2020 08:08:20 +0100 Message-Id: <20200707070858.6622-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::331; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x331.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Robert Foley Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Adding the args parameter to BaseVM's __init__. We will shortly need to pass more parameters to the class so let's just pass args rather than growing the parameter list. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-2-robert.foley@linaro.org> Message-Id: <20200701135652.1366-6-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index a80b616a08d3..5a58e6c39300 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -61,11 +61,10 @@ class BaseVM(object): # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 - def __init__(self, debug=False, vcpus=None, genisoimage=None, - build_path=None): + def __init__(self, args): self._guest = None - self._genisoimage = genisoimage - self._build_path = build_path + self._genisoimage = args.genisoimage + self._build_path = args.build_path self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) @@ -78,7 +77,7 @@ class BaseVM(object): self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) - self.debug = debug + self.debug = args.debug self._stderr = sys.stderr self._devnull = open(os.devnull, "w") if self.debug: @@ -92,8 +91,8 @@ class BaseVM(object): (",ipv6=no" if not self.ipv6 else ""), "-device", "virtio-net-pci,netdev=vnet", "-vnc", "127.0.0.1:0,to=20"] - if vcpus and vcpus > 1: - self._args += ["-smp", "%d" % vcpus] + if args.jobs and args.jobs > 1: + self._args += ["-smp", "%d" % args.jobs] if kvm_available(self.arch): self._args += ["-enable-kvm"] else: @@ -456,8 +455,7 @@ def main(vmcls): return 1 logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(debug=args.debug, vcpus=args.jobs, - genisoimage=args.genisoimage, build_path=args.build_path) + vm = vmcls(args) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Tue Jul 7 07:08:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234914 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp653074ilg; Tue, 7 Jul 2020 00:11:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz5HeukbKwxlzZZMDoupwX3E7ikJhk4wBb2psobj4wopcHSn7c5z6AbSWgEAYwhmhO6hPHe X-Received: by 2002:a25:ad50:: with SMTP id l16mr14844292ybe.110.1594105890128; Tue, 07 Jul 2020 00:11:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105890; cv=none; d=google.com; s=arc-20160816; b=uGmW6U9McGGbdqQs+twNTlLRGF+6XSjCl322nEPXmRlIHthpAWviNu91ozBu38alrn H+/OAkNm2Z/oV7T/VgCdQdzm2jey9PERD1CndeoOW+jSagej6ugd9ZOs3lfi7rzue275 O5M3NUJeVsxBwzlHjACMO4dyvIwk6FW31PzOoJelIF6rSYbqg/7ojuSfR8BAiy3MfMmC 8G4BrgGulHfCtwHXSqyL7x+QRrtN+cFtN69l8KOBvuy0cRZ+5GKr2Dn/HJp3iN2L5Azp HDuRDk+3vp1EssDXPviPWrH+XI7bmSLF/Ym776Z380MjzPKkYXYnHxPthhNlu2HurmCy z5Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ryQZtDkkMiGG0LRwlJZrH9SXH+wC9CMjzKlJDj1nBHY=; b=bEecyYHe4bTzvU8b/yferBoVsgUHV0wl2g/7nB6Z3S87r3zmvCz8sCmG+yZ8qp9eOy fFJEZrfEB0f1O0vDCi3ESswG/7DewC2t8VXfSORpG4kBRsuKzrUkOu843AFa3jbkMRPP iuTP6S1rN+Wzci1/yhgducuQHuF3/bTg/4yCDewfDcHcqObX2tdMdpFEtXmTeBWsBaFo 6XHoqC7XgiIHsg1NPkdRL1d/vOHe9KirRlVowyAv8sxrb9tKtK/O1taOteGDcRWRukeP f3LGs8AdhcXD2Hz3YAwWelBz+U2rVxnhmKOuqNuuaR6apY1HXdO8dBdDZFzVvvSXjKn7 EopA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ojj2axe4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y7si22405187ybk.128.2020.07.07.00.11.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:11:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ojj2axe4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshlJ-0007qI-It for patch@linaro.org; Tue, 07 Jul 2020 03:11:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42458) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj1-0004No-R3 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:07 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:34547) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshiz-00043O-JX for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:07 -0400 Received: by mail-wr1-x42a.google.com with SMTP id f7so40959425wrw.1 for ; Tue, 07 Jul 2020 00:09:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ryQZtDkkMiGG0LRwlJZrH9SXH+wC9CMjzKlJDj1nBHY=; b=Ojj2axe4pV40LvHtuaFDhAe1bwnEZ4eaLJmxYZtm+646l1etH8+rGaAI8opsZTruA/ 97C1kafgAQ5xr3Ohsuf7iLa5fuPOvDiGLcXLLHUfU3GuMnUJqoqdd1lkH3WrQQv91K0U Pgn38WHO+NvGBg2xHCbxGefn6vKC5IaztkyEGJi68isRaOvVRYmVjs8mtK818xVhIEzO 5VEJJw8+hhhllFxmLK392CTZfWMso4wBk7MJNAUn8yIZ5lQ/LogZLtnTb3qn4c7vJ57L nqJbjT3hCWJaFaJDm4O6hORUlhf+l9mgHD/w2Dk4gKswXpwBIgoYj368E7z24wFvMm/5 BARQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ryQZtDkkMiGG0LRwlJZrH9SXH+wC9CMjzKlJDj1nBHY=; b=F44IYqErAGyW9wanouxS4nCq2iBYlMamDc3pcOM2UPZa3j+/fabkJKMuQ+Glr4J6nh yFGX0nwv91uDRZpqip7VColhO4fVN5sH+1CUTXSb6GZtQhrc6KpKKtYqwG2ctvCcKaYF WpT0e8JRvG03+GtL/z6ur8s7UclVYduRYhlGFGpewmUQZ9viyJCkWobjoNILeZ7FPcZg 8jeatoILH4sJdNoWy8aTk541mDq3KoNx/PFTYDYWjYfFPhFuEAfinRo51DdFtMjm8aff 3YtnQkfhD/4qbcnSqTR27kGe/75XviPFV32zN7hWCQ0JRaWYyBS1tFD6daHCB7P+dfoW oFlg== X-Gm-Message-State: AOAM531MEUHmKLiaVVshFDhfDenvVaQmI485e5i+jK1QBcYO8YyLC9Bk cB5xJ7Fm1PZfWN3v1HUpRvulDw== X-Received: by 2002:adf:f20a:: with SMTP id p10mr55555920wro.41.1594105744080; Tue, 07 Jul 2020 00:09:04 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id g145sm2637160wmg.23.2020.07.07.00.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:00 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D17C61FF90; Tue, 7 Jul 2020 08:08:58 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 04/41] tests/vm: Add configuration to basevm.py Date: Tue, 7 Jul 2020 08:08:21 +0100 Message-Id: <20200707070858.6622-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Added use of a configuration to tests/vm/basevm.py. The configuration provides parameters used to configure a VM. This allows for providing alternate configurations to the VM being created/launched. cpu, machine, memory, and NUMA configuration are all examples of configuration which we might want to vary on the VM being created or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-3-robert.foley@linaro.org> Message-Id: <20200701135652.1366-7-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5a58e6c39300..cfe20c58f7ef 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -29,16 +29,41 @@ import tempfile import shutil import multiprocessing import traceback - -SSH_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa")).read() -SSH_PUB_KEY = open(os.path.join(os.path.dirname(__file__), - "..", "keys", "id_rsa.pub")).read() - +import shlex + +SSH_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa") +SSH_PUB_KEY_FILE = os.path.join(os.path.dirname(__file__), + "..", "keys", "id_rsa.pub") + +# This is the standard configuration. +# Any or all of these can be overridden by +# passing in a config argument to the VM constructor. +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : 'pc', + 'guest_user' : "qemu", + 'guest_pass' : "qemupass", + 'root_pass' : "qemupass", + 'ssh_key_file' : SSH_KEY_FILE, + 'ssh_pub_key_file': SSH_PUB_KEY_FILE, + 'memory' : "4G", + 'extra_args' : [], + 'qemu_args' : "", + 'dns' : "", + 'ssh_port' : 0, + 'install_cmds' : "", + 'boot_dev_type' : "block", + 'ssh_timeout' : 1, +} +BOOT_DEVICE = { + 'block' : "-drive file={},if=none,id=drive0,cache=writeback "\ + "-device virtio-blk,drive=drive0,bootindex=0", + 'scsi' : "-device virtio-scsi-device,id=scsi "\ + "-drive file={},format=raw,if=none,id=hd0 "\ + "-device scsi-hd,drive=hd0,bootindex=0", +} class BaseVM(object): - GUEST_USER = "qemu" - GUEST_PASS = "qemupass" - ROOT_PASS = "qemupass" envvars = [ "https_proxy", @@ -57,25 +82,38 @@ class BaseVM(object): poweroff = "poweroff" # enable IPv6 networking ipv6 = True + # This is the timeout on the wait for console bytes. + socket_timeout = 120 # Scale up some timeouts under TCG. # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. tcg_ssh_timeout_multiplier = 4 - def __init__(self, args): + def __init__(self, args, config=None): self._guest = None self._genisoimage = args.genisoimage self._build_path = args.build_path + # Allow input config to override defaults. + self._config = DEFAULT_CONFIG.copy() + if config != None: + self._config.update(config) + self.validate_ssh_keys() self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-", suffix=".tmp", dir=".")) atexit.register(shutil.rmtree, self._tmpdir) - - self._ssh_key_file = os.path.join(self._tmpdir, "id_rsa") - open(self._ssh_key_file, "w").write(SSH_KEY) - subprocess.check_call(["chmod", "600", self._ssh_key_file]) - - self._ssh_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") - open(self._ssh_pub_key_file, "w").write(SSH_PUB_KEY) + # Copy the key files to a temporary directory. + # Also chmod the key file to agree with ssh requirements. + self._config['ssh_key'] = \ + open(self._config['ssh_key_file']).read().rstrip() + self._config['ssh_pub_key'] = \ + open(self._config['ssh_pub_key_file']).read().rstrip() + self._ssh_tmp_key_file = os.path.join(self._tmpdir, "id_rsa") + open(self._ssh_tmp_key_file, "w").write(self._config['ssh_key']) + subprocess.check_call(["chmod", "600", self._ssh_tmp_key_file]) + + self._ssh_tmp_pub_key_file = os.path.join(self._tmpdir, "id_rsa.pub") + open(self._ssh_tmp_pub_key_file, + "w").write(self._config['ssh_pub_key']) self.debug = args.debug self._stderr = sys.stderr @@ -84,11 +122,14 @@ class BaseVM(object): self._stdout = sys.stdout else: self._stdout = self._devnull + netdev = "user,id=vnet,hostfwd=:127.0.0.1:{}-:22" self._args = [ \ - "-nodefaults", "-m", "4G", - "-cpu", "max", - "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + - (",ipv6=no" if not self.ipv6 else ""), + "-nodefaults", "-m", self._config['memory'], + "-cpu", self._config['cpu'], + "-netdev", + netdev.format(self._config['ssh_port']) + + (",ipv6=no" if not self.ipv6 else "") + + (",dns=" + self._config['dns'] if self._config['dns'] else ""), "-device", "virtio-net-pci,netdev=vnet", "-vnc", "127.0.0.1:0,to=20"] if args.jobs and args.jobs > 1: @@ -99,6 +140,55 @@ class BaseVM(object): logging.info("KVM not available, not using -enable-kvm") self._data_args = [] + if self._config['qemu_args'] != None: + qemu_args = self._config['qemu_args'] + qemu_args = qemu_args.replace('\n',' ').replace('\r','') + # shlex groups quoted arguments together + # we need this to keep the quoted args together for when + # the QEMU command is issued later. + args = shlex.split(qemu_args) + self._config['extra_args'] = [] + for arg in args: + if arg: + # Preserve quotes around arguments. + # shlex above takes them out, so add them in. + if " " in arg: + arg = '"{}"'.format(arg) + self._config['extra_args'].append(arg) + + def validate_ssh_keys(self): + """Check to see if the ssh key files exist.""" + if 'ssh_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_key_file']): + raise Exception("ssh key file not found.") + if 'ssh_pub_key_file' not in self._config or\ + not os.path.exists(self._config['ssh_pub_key_file']): + raise Exception("ssh pub key file not found.") + + def wait_boot(self, wait_string=None): + """Wait for the standard string we expect + on completion of a normal boot. + The user can also choose to override with an + alternate string to wait for.""" + if wait_string is None: + if self.login_prompt is None: + raise Exception("self.login_prompt not defined") + wait_string = self.login_prompt + # Intentionally bump up the default timeout under TCG, + # since the console wait below takes longer. + timeout = self.socket_timeout + if not kvm_available(self.arch): + timeout *= 8 + self.console_init(timeout=timeout) + self.console_wait(wait_string) + + def __getattr__(self, name): + # Support direct access to config by key. + # for example, access self._config['cpu'] by self.cpu + if name.lower() in self._config.keys(): + return self._config[name.lower()] + return object.__getattribute__(self, name) + def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -130,8 +220,9 @@ class BaseVM(object): "-t", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=" + os.devnull, - "-o", "ConnectTimeout=1", - "-p", self.ssh_port, "-i", self._ssh_key_file] + "-o", + "ConnectTimeout={}".format(self._config["ssh_timeout"]), + "-p", self.ssh_port, "-i", self._ssh_tmp_key_file] # If not in debug mode, set ssh to quiet mode to # avoid printing the results of commands. if not self.debug: @@ -180,14 +271,14 @@ class BaseVM(object): "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)] def boot(self, img, extra_args=[]): - args = self._args + [ - "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img, - "-device", "virtio-blk,drive=drive0,bootindex=0"] - args += self._data_args + extra_args + boot_dev = BOOT_DEVICE[self._config['boot_dev_type']] + boot_params = boot_dev.format(img) + args = self._args + boot_params.split(' ') + args += self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path = get_qemu_path(self.arch, self._build_path) guest = QEMUMachine(binary=qemu_path, args=args) - guest.set_machine('pc') + guest.set_machine(self._config['machine']) guest.set_console() try: guest.launch() @@ -301,7 +392,8 @@ class BaseVM(object): self.console_send(command) def console_ssh_init(self, prompt, user, pw): - sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip() + sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" \ + % self._config['ssh_pub_key'].rstrip() self.console_wait_send("login:", "%s\n" % user) self.console_wait_send("Password:", "%s\n" % pw) self.console_wait_send(prompt, "mkdir .ssh\n") @@ -360,23 +452,23 @@ class BaseVM(object): "local-hostname: {}-guest\n".format(name)]) mdata.close() udata = open(os.path.join(cidir, "user-data"), "w") - print("guest user:pw {}:{}".format(self.GUEST_USER, - self.GUEST_PASS)) + print("guest user:pw {}:{}".format(self._config['guest_user'], + self._config['guest_pass'])) udata.writelines(["#cloud-config\n", "chpasswd:\n", " list: |\n", - " root:%s\n" % self.ROOT_PASS, - " %s:%s\n" % (self.GUEST_USER, - self.GUEST_PASS), + " root:%s\n" % self._config['root_pass'], + " %s:%s\n" % (self._config['guest_user'], + self._config['guest_pass']), " expire: False\n", "users:\n", - " - name: %s\n" % self.GUEST_USER, + " - name: %s\n" % self._config['guest_user'], " sudo: ALL=(ALL) NOPASSWD:ALL\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], " - name: root\n", " ssh-authorized-keys:\n", - " - %s\n" % SSH_PUB_KEY, + " - %s\n" % self._config['ssh_pub_key'], "locale: en_US.UTF-8\n"]) proxy = os.environ.get("http_proxy") if not proxy is None: @@ -447,15 +539,17 @@ def parse_args(vmcls): parser.disable_interspersed_args() return parser.parse_args() -def main(vmcls): +def main(vmcls, config=None): try: + if config == None: + config = {} args, argv = parse_args(vmcls) if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) - vm = vmcls(args) + vm = vmcls(args, config=config) if args.build_image: if os.path.exists(args.image) and not args.force: sys.stderr.writelines(["Image file exists: %s\n" % args.image, From patchwork Tue Jul 7 07:08:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234918 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp654362ilg; Tue, 7 Jul 2020 00:13:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwDG+jpD7REkJfanahmvNURphX4fta/Er8MapaQdKqRKuTNEDkyykc8g7itOEh0jOFoiHh/ X-Received: by 2002:a25:b992:: with SMTP id r18mr82338834ybg.283.1594106012786; Tue, 07 Jul 2020 00:13:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106012; cv=none; d=google.com; s=arc-20160816; b=OYeUstkXzlUZBGaOprpR+nTXy7Zk4ihDIp8X66e1vwmLWjZDdEY39ghOyum2RD//aN d4IT9pmIpyflHrl7CGaN+qE+4ZA7srwcOHJJBaXnvnVTheWlEHEMyKKIGoqfl2BcbrRB DE/DUI9vAjznyU30X3vEDot7XNL6vVXn+5vgBfTU62YKMHHWzrX1XhEjFt3th3gLqmum LYkF08FXBhJacBFAOHvL5lTSb+JLOwffeOSCqQYma8qZ8UW+MTXKSYoOWNV1xIgpiNsU ouk+ICts/t1i4ZtVErUgIlvRbhkEBTcND8laqSGhKn9W6LYrKacBrFvmMvnLIhyd4DRX uNsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ffjD3xFJ7V8ELReeSR1/1F6adXpcPs6lDfrQfmzZ+PU=; b=VASlo/PSMtRoYPPXK3fIrG/dkOzdj1dQAF+MkOKDfAi9s+T2g7d7xoJOkbJH4rXjPF QiwI9Fi51F0Uha6ipN3pOK+1GS4WV3XTyK7SQw9NGNPdlGNqoFKEP2UUwof9a0ZMvrZu VgMN43LC2thTy6nnDajIPs7w+U7QBEM2+UW9BPfqMfuvBMFLbyWxFjS170rpaz5WES/v Dk3aC18eGVMuaOeObkTP5PquCgGN1ipp/ddqcylpkyvz7FR68qk+eFgn3+MjRcYsA5ZT zfZre3poAn5IKio/PI4IZuu8kMWvUTLFFQw+8sTpYO1KuNmyBPll02UMBBq9mU21EiaD 0p7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=b10kpPAG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 132si22414266yba.45.2020.07.07.00.13.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:13:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=b10kpPAG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51532 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshnI-0002rH-5e for patch@linaro.org; Tue, 07 Jul 2020 03:13:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj5-0004X5-NR for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:11 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]:53710) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj3-00044m-7o for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:11 -0400 Received: by mail-wm1-x329.google.com with SMTP id j18so42111855wmi.3 for ; Tue, 07 Jul 2020 00:09:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ffjD3xFJ7V8ELReeSR1/1F6adXpcPs6lDfrQfmzZ+PU=; b=b10kpPAGWV6CJ7Jy6+vaT1nk2+iuYe9ZwOcFf71qZeJXM8HO9L+fEus1JXdqo400s0 OQd3epHFS8HL/QqwcOzE+EldiRZJ2hxVelxZDHuauI0wSQ1yS8X+9wztu3Tl40RMEj+S hlPZYPHolnvQPrS2oLiRdDvgcBWk+YcN2vvi+76nrNWnUpKxR1oSspjlzOEG+U8HUMyx mCKhSQKh+aE61kVwPC12HmpwvFYT/k3z3bx5t2HPhe02RynSe2raWUwxZ5AtPGwihYr1 1Gc5ufP8IjYbCo6aZBB7T/Irzfjr1+tf6zqT/RT/snGckpePQafOSe8YXcFBFi6y0hnk Enxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ffjD3xFJ7V8ELReeSR1/1F6adXpcPs6lDfrQfmzZ+PU=; b=CG2QvHbYibwo8qnozmSCaS60E1diMea5kcf7kCE6wdOH9gSvVGQUHxrRCxqAh6mDto 6RKI45/hlpdbakQCGSlTlXFT3U9xGQCtoKyfBJhpEdzltX0RYc5vHRGz7rk0+YZGntpR fRomgwwBccKtdyFFgvj/hLkguKg+NYh+eSNnQNTzl5Rq/npsOLenCOVLUpNlLIwf7gSz HW4tBeWz7iJU6cKdowIcjtaSGeDAjCdDgk17eRecIik29BepF6us1+1o2XMNB7zoQwKR 2aMPDfpSNki9RUjRKSA9WogGEpAjA03wV3dT60mAuGy4M6J7qk/lhQHNJ6YXT4/4eOLb ImtQ== X-Gm-Message-State: AOAM533bIKQ8ItJOxGzcHxcZOtzEU7dtLrX0Qb1fz8ZcKXwMja4IR7zH 5cmk0KUVnrN1xh/vHK88DatmvQ== X-Received: by 2002:a1c:2e0e:: with SMTP id u14mr2638785wmu.55.1594105746940; Tue, 07 Jul 2020 00:09:06 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id y77sm2348418wmd.36.2020.07.07.00.09.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id EEEAB1FF91; Tue, 7 Jul 2020 08:08:58 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 05/41] tests/vm: Added configuration file support Date: Tue, 7 Jul 2020 08:08:22 +0100 Message-Id: <20200707070858.6622-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x329.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Changes to tests/vm/basevm.py to allow accepting a configuration file as a parameter. Allows for specifying VM options such as cpu, machine, memory, and arbitrary qemu arguments for specifying options such as NUMA configuration. Also added an example conf_example_aarch64.yml and conf_example_x86.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-4-robert.foley@linaro.org> Message-Id: <20200701135652.1366-8-alex.bennee@linaro.org> -- 2.20.1 diff --git a/configure b/configure index 83e36c270cce..3cabd004e329 100755 --- a/configure +++ b/configure @@ -960,6 +960,13 @@ do fi done +# Check for existence of python3 yaml, needed to +# import yaml config files into vm-build. +python_yaml="no" +if $(python3 -c "import yaml" 2> /dev/null); then + python_yaml="yes" +fi + : ${smbd=${SMBD-/usr/sbin/smbd}} # Default objcc to clang if available, otherwise use CC @@ -6843,6 +6850,7 @@ if test "$docs" != "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" != "no" ; then echo "smbd $smbd" @@ -7928,6 +7936,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak +echo "PYTHON_YAML=$python_yaml" >> $config_host_mak echo "CC=$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then echo "IASL=$iasl" >> $config_host_mak diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index a253aba45796..f6c3892bb28e 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -44,6 +44,12 @@ endif @echo " QEMU_LOCAL=1 - Use QEMU binary local to this build." @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @echo " QEMU_IMG=/path/to/qemu-img - Change path to qemu-img tool" +ifeq ($(PYTHON_YAML),yes) + @echo " QEMU_CONFIG=/path/conf.yml - Change path to VM configuration .yml file." +else + @echo " (install python3-yaml to enable support for yaml file to configure a VM.)" +endif + @echo " See conf_example_*.yml for file format details." vm-build-all: $(addprefix vm-build-, $(IMAGES)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index cfe20c58f7ef..fa56fbbb4b60 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -481,7 +481,6 @@ class BaseVM(object): cwd=cidir, stdin=self._devnull, stdout=self._stdout, stderr=self._stdout) - return os.path.join(cidir, "cloud-init.iso") def get_qemu_path(arch, build_path=None): @@ -497,6 +496,41 @@ def get_qemu_path(arch, build_path=None): qemu_path = "qemu-system-" + arch return qemu_path +def parse_config(config, args): + """ Parse yaml config and populate our config structure. + The yaml config allows the user to override the + defaults for VM parameters. In many cases these + defaults can be overridden without rebuilding the VM.""" + if args.config: + config_file = args.config + elif 'QEMU_CONFIG' in os.environ: + config_file = os.environ['QEMU_CONFIG'] + else: + return config + if not os.path.exists(config_file): + raise Exception("config file {} does not exist".format(config_file)) + # We gracefully handle importing the yaml module + # since it might not be installed. + # If we are here it means the user supplied a .yml file, + # so if the yaml module is not installed we will exit with error. + try: + import yaml + except ImportError: + print("The python3-yaml package is needed "\ + "to support config.yaml files") + # Instead of raising an exception we exit to avoid + # a raft of messy (expected) errors to stdout. + exit(1) + with open(config_file) as f: + yaml_dict = yaml.safe_load(f) + + if 'qemu-conf' in yaml_dict: + config.update(yaml_dict['qemu-conf']) + else: + raise Exception("config file {} is not valid"\ + " missing qemu-conf".format(config_file)) + return config + def parse_args(vmcls): def get_default_jobs(): @@ -536,6 +570,9 @@ def parse_args(vmcls): help="run tests with a snapshot") parser.add_option("--genisoimage", default="genisoimage", help="iso imaging tool") + parser.add_option("--config", "-c", default=None, + help="Provide config yaml for configuration. "\ + "See config_example.yaml for example.") parser.disable_interspersed_args() return parser.parse_args() @@ -547,6 +584,7 @@ def main(vmcls, config=None): if not argv and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 + config = parse_config(config, args) logging.basicConfig(level=(logging.DEBUG if args.debug else logging.WARN)) vm = vmcls(args, config=config) diff --git a/tests/vm/conf_example_aarch64.yml b/tests/vm/conf_example_aarch64.yml new file mode 100644 index 000000000000..9d44ae356f7c --- /dev/null +++ b/tests/vm/conf_example_aarch64.yml @@ -0,0 +1,51 @@ +# +# Example yaml for use by any of the scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: qemu + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # If one key is provided, both must be provided. + #ssh_key: /complete/path/of/your/keyfile/id_rsa + #ssh_pub_key: /complete/path/of/your/keyfile/id_rsa.pub + + cpu: max + machine: virt,gic-version=max + memory: 16G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=16,sockets=2,cores=8 + -numa node,cpus=0-3,nodeid=0 -numa node,cpus=4-7,nodeid=1 + -numa node,cpus=8-11,nodeid=2 -numa node,cpus=12-15,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: 1.234.567.89 + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + #ssh_port: 5555 + + # To install a different set of packages, provide a command to issue + #install_cmds: "apt-get update ; apt-get build-dep -y qemu" + + # Or to skip the install entirely, just provide "" + #install_cmds: "" diff --git a/tests/vm/conf_example_x86.yml b/tests/vm/conf_example_x86.yml new file mode 100644 index 000000000000..78d3f5830fa2 --- /dev/null +++ b/tests/vm/conf_example_x86.yml @@ -0,0 +1,50 @@ +# +# Example yaml for use by any of the x86 based scripts in tests/vm. +# Can be provided as an environment variable QEMU_CONFIG +# +qemu-conf: + + # If any of the below are not provided, we will just use the qemu defaults. + + # Login username and password(has to be sudo enabled) + guest_user: "qemu" + guest_pass: "qemupass" + + # Password for root user can be different from guest. + root_pass: "qemupass" + + # Provide default ssh keys of current user. + # You need to edit the below for your user. + #ssh_key_file: /home//.ssh/id_rsa + #ssh_pub_key_file: /home//.ssh/id_rsa.pub + + cpu: max + machine: pc + memory: 8G + + # The below is a example for how to configure NUMA topology with + # 4 NUMA nodes and 2 different NUMA distances. + qemu_args: "-smp cpus=8,sockets=2,cores=4 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node0 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=0,id=ram-node1 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node2 + -object memory-backend-ram,size=4G,policy=bind,host-nodes=1,id=ram-node3 + -numa node,cpus=0-1,nodeid=0 -numa node,cpus=2-3,nodeid=1 + -numa node,cpus=4-5,nodeid=2 -numa node,cpus=6-7,nodeid=3 + -numa dist,src=0,dst=1,val=15 -numa dist,src=2,dst=3,val=15 + -numa dist,src=0,dst=2,val=20 -numa dist,src=0,dst=3,val=20 + -numa dist,src=1,dst=2,val=20 -numa dist,src=1,dst=3,val=20" + + # By default we do not set the DNS. + # You override the defaults by setting the below. + #dns: "1.234.567.89" + + # By default we will use a "block" device, but + # you can also boot from a "scsi" device. + # Just keep in mind your scripts might need to change + # As you will have /dev/sda instead of /dev/vda (for block device) + boot_dev_type: "block" + + # By default the ssh port is not fixed. + # A fixed ssh port makes it easier for automated tests. + ssh_port: 5555 From patchwork Tue Jul 7 07:08:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234920 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp655441ilg; Tue, 7 Jul 2020 00:15:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJws3GxpkEC4q81DIalyr4OHYch7Wi/Soi76k8DvEL35jlwObimXudVMlro5YiV28KvWiVL6 X-Received: by 2002:a25:a026:: with SMTP id x35mr87978254ybh.114.1594106123797; Tue, 07 Jul 2020 00:15:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106123; cv=none; d=google.com; s=arc-20160816; b=yhnwN9Mr7LOPIvfRsb5bBXLj1pgtAXBKmEE90od2V4g4uJcUjwn5QXcTdJdtrz+QQ3 TvaIk5vIwoVA/trtAVEYrCXgWFoUTp6g2H0AXMuE5j0lWTUjcE0NkzEeGTm8ITQB0uvt 84tKHRSIXnPFmfar8zrXvXViAF8yuo7ghcoHoAwt/CnCc4ANrNX3jTL5z0/dWbNbLYER UVKt5XTyrg1XE9nCmRkoUfSqni+o4WblduuGAcA5JklUamCI07uEayCVHXqcPQyazJe5 SfvJqzmBQCjnhEBCklKRCIHyp7rsfjvqzF6dVEaZ697S3ie8qKSYb/VcPT1HZ8yeCRnZ Vtlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=y/qmE2rvdW+2X2fGtUlm2uoe3BCyY7VQ2o9+eHdvkyA=; b=RIUPWj+PZtf9Q2i5Z8OULnfxXu4r4ZzjcLglPUaVKRnEvQqB+mlEZfb0xi1wITTae9 AWHakn3thW35VVmpizbixydlXe932vobqOJo/yqYoYunmNnwuZCGV0sFPA2DMwEoTggN FcN7NUPZnD9stf7TnMOWSWdQYqI+2Gx82da/Jx6ebJSRq6alIMyfulQo3fO9/Hn/Pzum qbtmltRblN//g56K0YAuIpKIeZUFAfJP6F83q5dA+t/MtLIczpn8KnRrQ3eC1ClSHXiv BPZnOJvl9OH/UOnPC0Qka8M9OPKicrp6K8gOx7Vfebt/M6nrrhH/QpNFJ3AhFvyxrk+9 JAbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VO85lIIB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c8si20468778ybi.68.2020.07.07.00.15.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:15:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VO85lIIB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60068 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshp5-0006hs-6s for patch@linaro.org; Tue, 07 Jul 2020 03:15:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj6-0004bN-Vt for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:13 -0400 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:50369) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj3-00044x-If for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:12 -0400 Received: by mail-wm1-x332.google.com with SMTP id l17so42117055wmj.0 for ; Tue, 07 Jul 2020 00:09:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y/qmE2rvdW+2X2fGtUlm2uoe3BCyY7VQ2o9+eHdvkyA=; b=VO85lIIBVWBNwy4Rzco5/hd7alzrd37H7gQll84vVwymGS6kn+RAX14uX9zFVJR47u WjV1joSBV30r3ifE0zRN8Pze+nYnsAywfSX2zx1NLjRDbh9taoQY+qvyVOG0Ph5qDN9d vu4IzLVuXsSDKUWa82yw1xA4wdvxG/zTshkRVEse/XpnGBtKs5dgQ5PHSyDRXUrg8G9W REcOWUCO2A5jc8VlaNaBdr08l7wfhEkXMX6vnf4keQcS5/MsMWVy65wanlP2OeEbAunb FFqyN92jj0Kj6uP/JLkcqcAHClUia1lQpix8TvYCQOp8G1bEzR+fXWe2fH2PaDO4/OOC SeIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y/qmE2rvdW+2X2fGtUlm2uoe3BCyY7VQ2o9+eHdvkyA=; b=fMImh8RZ5InlhJdg5poUfzKD39WdaexxgUn54MkvG7huYqj/emETlqpSt6PolgKzm4 MXiHHNcPBRlToVizjoxvkGTx1XXdKvhUzqQOEC6241qcy/mNyvL+RUxBBau0tkrfwuA6 6Sb3TcajnVFrS96PiaUZgYNackh5G+AgxUVtrhLQXtsIASJZXE6biex6Yu7KwiwZJZnM UcKJsoFnUrn0C4rJmi7LGOlRqm9OW2m7BQgTMsdv48bHnRHpyW+RbZyg6sWAmILZJJVe xMKpQ5OEezLj8wq0ffOW0pmpWGOT8fW1HTUyw9K1ftWgyVM2GQsVYchBwscKmETjq84T ABzA== X-Gm-Message-State: AOAM530UlDqGODmUMIQFt54iJSy5x6zVl8h+sCx+E9YwPVWp76a1m74r CSQ5EcSMPJHmxOX0LG+osPasOQ== X-Received: by 2002:a7b:c0c9:: with SMTP id s9mr2505577wmh.166.1594105748078; Tue, 07 Jul 2020 00:09:08 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z6sm2155864wmf.33.2020.07.07.00.09.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 123231FF92; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 06/41] tests/vm: Add common Ubuntu python module Date: Tue, 7 Jul 2020 08:08:23 +0100 Message-Id: <20200707070858.6622-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Robert Foley Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Add a common Ubuntu python module and make use of it with the ubuntu.i386 script. This is preparation for adding an Ubuntu script ubuntu.aarch64. Splitting out the common logic such as build_image() will reduce duplication. Signed-off-by: Robert Foley Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-5-robert.foley@linaro.org> Message-Id: <20200701135652.1366-9-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 24527cc78c77..5ce72610a6c8 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -11,15 +11,22 @@ # the COPYING file in the top-level directory. # -import os import sys -import subprocess import basevm -import time +import ubuntuvm -class UbuntuX86VM(basevm.BaseVM): +DEFAULT_CONFIG = { + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y qemu,"\ + "apt-get install -y libfdt-dev language-pack-en", +} + +class UbuntuX86VM(ubuntuvm.UbuntuVM): name = "ubuntu.i386" arch = "i386" + image_link="https://cloud-images.ubuntu.com/releases/bionic/"\ + "release-20191114/ubuntu-18.04-server-cloudimg-i386.img" + image_sha256="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef" BUILD_SCRIPT = """ set -e; cd $(mktemp -d); @@ -29,34 +36,5 @@ class UbuntuX86VM(basevm.BaseVM): make --output-sync {target} -j{jobs} {verbose}; """ - def build_image(self, img): - cimg = self._download_with_cache( - "https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img", - sha256sum="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef") - img_tmp = img + ".tmp" - subprocess.check_call(["cp", "-f", cimg, img_tmp]) - self.exec_qemu_img("resize", img_tmp, "50G") - self.boot(img_tmp, extra_args = [ - "-device", "VGA", - "-cdrom", self.gen_cloud_init_iso() - ]) - self.wait_ssh() - self.ssh_root_check("touch /etc/cloud/cloud-init.disabled") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get install -y cloud-initramfs-growroot") - # Don't check the status in case the guest hang up too quickly - self.ssh_root("sync && reboot") - time.sleep(5) - self.wait_ssh() - # The previous update sometimes doesn't survive a reboot, so do it again - self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list") - self.ssh_root_check("apt-get update") - self.ssh_root_check("apt-get build-dep -y qemu") - self.ssh_root_check("apt-get install -y libfdt-dev language-pack-en") - self.ssh_root("poweroff") - self.wait() - os.rename(img_tmp, img) - return 0 - if __name__ == "__main__": - sys.exit(basevm.main(UbuntuX86VM)) + sys.exit(basevm.main(UbuntuX86VM, DEFAULT_CONFIG)) diff --git a/tests/vm/ubuntuvm.py b/tests/vm/ubuntuvm.py new file mode 100644 index 000000000000..6689ad87aa83 --- /dev/null +++ b/tests/vm/ubuntuvm.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# +# Ubuntu VM testing library +# +# Copyright 2017 Red Hat Inc. +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. + +import os +import subprocess +import basevm + +class UbuntuVM(basevm.BaseVM): + + def __init__(self, args, config=None): + self.login_prompt = "ubuntu-{}-guest login:".format(self.arch) + basevm.BaseVM.__init__(self, args, config) + + def build_image(self, img): + """Build an Ubuntu VM image. The child class will + define the install_cmds to init the VM.""" + os_img = self._download_with_cache(self.image_link, + sha256sum=self.image_sha256) + img_tmp = img + ".tmp" + subprocess.check_call(["cp", "-f", os_img, img_tmp]) + self.exec_qemu_img("resize", img_tmp, "+50G") + ci_img = self.gen_cloud_init_iso() + + self.boot(img_tmp, extra_args = [ "-device", "VGA", "-cdrom", ci_img, ]) + + # First command we issue is fix for slow ssh login. + self.wait_ssh(wait_root=True, + cmd="chmod -x /etc/update-motd.d/*") + # Wait for cloud init to finish + self.wait_ssh(wait_root=True, + cmd="ls /var/lib/cloud/instance/boot-finished") + self.ssh_root("touch /etc/cloud/cloud-init.disabled") + # Disable auto upgrades. + # We want to keep the VM system state stable. + self.ssh_root('sed -ie \'s/"1"/"0"/g\' '\ + '/etc/apt/apt.conf.d/20auto-upgrades') + self.ssh_root("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list") + + # If the user chooses not to do the install phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + # Issue the install commands. + # This can be overriden by the user in the config .yml. + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.graceful_shutdown() + os.rename(img_tmp, img) + return 0 From patchwork Tue Jul 7 07:08:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234913 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp652841ilg; Tue, 7 Jul 2020 00:11:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxyYm2DDKThZmHoktu1CJqSTGm+zMvgSXm+GozAY0xU8ip1RtdkTSEviIwNw5/17T0KJa6a X-Received: by 2002:a25:d652:: with SMTP id n79mr92322653ybg.406.1594105867341; Tue, 07 Jul 2020 00:11:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105867; cv=none; d=google.com; s=arc-20160816; b=yLuboFfT5HkE4hoVrG0ICo2vCOOErCHAvrkEIQl4jNLrZSeQBYLQ0YNQp+ODf3Gu4C UXrP7fLul2txy/R88ppF2qweq5dvIRUIyM2+tjhI6iYHnoE/XmzvwnUmj8IUuynPvfdC ID1CekHkO0CfwKpdm5dT5fSHn95LPgFXnWfiuR3O8f9rT5Jf4zP2Q4pqDZVw12FOwl2O 2EGM8wumxAZg4NkYpA/W8wbkaJCWE7SlKqI1eMOtczGHaab89PbhTNuapMuM1b2Lda/l qyqtuiqQJT3fhlto6X0HKvfJFJiSU9+rDdG0WH4l8F0VkDINltTz27f31mcmug/bavxz 12vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=YKrkcy3TnItuyzjmWvaP6irqkpfpFfve+p64hNdgFao=; b=RIHq1YnkQFuWRWqiY+cEfygj8Hyq/nHCwljlceSq8FgZZ3kRO+bRX3lofecWbpLFbE cwRzX3QLoeMuokTg01iFgnxoTxH3sgPZl1pscKtlq+eRkMQRK1IzXlXGPtSblTN06e8r AnByCU2dPQtnwO/ImrdU27vyGSHUWATKXGT6PplcfftTVGzhAn1iuyAQyn07K5Bn7Cj7 fAXlUXtYi7/kUEzo8vau3OKBfqrLCP0wDIsSCu/pZV2S6Q09F6B+CyyzXKL3NLKcTuO5 3gsKFEK0QoskQ3Z+sEb1kWG2ApdF8C+l/xhSwlwu8d231vLb0ahDPsYc6JwIEfjjIEjE zOhA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="rfW/QYfq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a19si15429911ybg.332.2020.07.07.00.11.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:11:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="rfW/QYfq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshkw-0007yW-Lo for patch@linaro.org; Tue, 07 Jul 2020 03:11:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42542) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj8-0004hI-QC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:14 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:36054) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj6-00045A-0y for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:14 -0400 Received: by mail-wr1-x431.google.com with SMTP id k6so43991651wrn.3 for ; Tue, 07 Jul 2020 00:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YKrkcy3TnItuyzjmWvaP6irqkpfpFfve+p64hNdgFao=; b=rfW/QYfqnAbN7+NPzVXDBJnyr61+Z9mgzyK0+7oJwd6Vdxe/rHoX9eFAHsRVXsix8v 7GR/Lsm1pVj3aS56rX6m/4lhtsGFKEmEzK9HLFXqlC5SrcmbZwr7k+nC014iI8PVSTq+ 6pRsAHbz4psJGOLV3xTrx+9t3aj/X4jLoqY4jzGBfdsf+n944LkzaV8FXO8Tx6iNEZAV PJTIFFkILiMmNe/bP8uz1B0gj9M/fjXyc7m0hjny1pZeKL61zCL2oNYwlp9fB4RX9Uie DaHQbiSeey1NS9nBJ4If8WNhNYPEyRoB0/jEjsxxJXQToYiJ1O+uJyXjmLGIWJ6m+5cc 4lYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YKrkcy3TnItuyzjmWvaP6irqkpfpFfve+p64hNdgFao=; b=hk2IHLKJWfxm/XvtPcIh1QBG4F9Yr9UBOTAss3rN4V/lu0UwtNjL+knBwtQn6Sa98R u+KXTm1yt5DNFimL8PrJzBw9UvLq0sVL2BufkYBCLz5r5mErFfW+git5vEDR2yxculQV ni9ED8nryMd4bGy5/2Ql7SI2/rjHtztOJXfr0+4u9S4H2RatMUNDDmwTk8d5txNb27dH +3fCVUUMh68DaZCR8gqp0p/dnZaRbHbRJwi4FSQvNaTg31ZrWjfNnctuxfZCNcRZNcxh LQWE88gKTzZ+E378CKRTessHV1CNA8meZBJmZCSDyLR/hg41U4lYBryyYMsDY9oiawAh KJSA== X-Gm-Message-State: AOAM531DLQ/++K6eTUb/Onn0qGNcRhXbXRpSlmU3QePVKwqkqfgMG8Fk bab3C2+XPg9nOUxH8mlAsmmRIA== X-Received: by 2002:adf:92c4:: with SMTP id 62mr51051258wrn.290.1594105750387; Tue, 07 Jul 2020 00:09:10 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u65sm2273668wmg.5.2020.07.07.00.09.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2DF911FF93; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 07/41] tests/vm: Added a new script for ubuntu.aarch64. Date: Tue, 7 Jul 2020 08:08:24 +0100 Message-Id: <20200707070858.6622-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-6-robert.foley@linaro.org> Message-Id: <20200701135652.1366-10-alex.bennee@linaro.org> -- 2.20.1 diff --git a/configure b/configure index 3cabd004e329..d2d93ae51e47 100755 --- a/configure +++ b/configure @@ -418,6 +418,7 @@ prefix="/usr/local" mandir="\${prefix}/share/man" datadir="\${prefix}/share" firmwarepath="\${prefix}/share/qemu-firmware" +efi_aarch64="" qemu_docdir="\${prefix}/share/doc/qemu" bindir="\${prefix}/bin" libdir="\${prefix}/lib" @@ -1109,6 +1110,8 @@ for opt do ;; --firmwarepath=*) firmwarepath="$optarg" ;; + --efi-aarch64=*) efi_aarch64="$optarg" + ;; --host=*|--build=*|\ --disable-dependency-tracking|\ --sbindir=*|--sharedstatedir=*|\ @@ -1787,6 +1790,7 @@ Advanced options (experts only): --sysconfdir=PATH install config in PATH$confsuffix --localstatedir=PATH install local state in PATH (set at runtime on win32) --firmwarepath=PATH search PATH for firmware files + --efi-aarch64=PATH PATH of efi file to use for aarch64 VMs. --with-confsuffix=SUFFIX suffix for QEMU data inside datadir/libdir/sysconfdir [$confsuffix] --with-pkgversion=VERS use specified string as sub-version of the package --enable-debug enable common debug build options @@ -3606,6 +3610,20 @@ EOF fi fi +############################################ +# efi-aarch64 probe +# Check for efi files needed by aarch64 VMs. +# By default we will use the efi included with QEMU. +# Allow user to override the path for efi also. +if ! test -f "$efi_aarch64"; then + if test -f $source_path/pc-bios/edk2-aarch64-code.fd.bz2; then + # valid after build + efi_aarch64=$PWD/pc-bios/edk2-aarch64-code.fd + else + efi_aarch64="" + fi +fi + ########################################## # libcap-ng library probe if test "$cap_ng" != "no" ; then @@ -6850,6 +6868,7 @@ if test "$docs" != "no"; then echo "sphinx-build $sphinx_build" fi echo "genisoimage $genisoimage" +echo "efi_aarch64 $efi_aarch64" echo "python_yaml $python_yaml" echo "slirp support $slirp $(echo_version $slirp $slirp_version)" if test "$slirp" != "no" ; then @@ -7936,6 +7955,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak +echo "EFI_AARCH64=$efi_aarch64" >> $config_host_mak echo "PYTHON_YAML=$python_yaml" >> $config_host_mak echo "CC=$cc" >> $config_host_mak if $iasl -h > /dev/null 2>&1; then diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index f6c3892bb28e..4fa292765d5c 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -5,6 +5,9 @@ IMAGES := freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES += ubuntu.i386 centos +ifneq ($(EFI_AARCH64),) +IMAGES += ubuntu.aarch64 +endif endif IMAGES_DIR := $(HOME)/.cache/qemu-vm/images @@ -23,6 +26,11 @@ vm-help vm-test: ifneq ($(GENISOIMAGE),) @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker" @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" +ifneq ($(EFI_AARCH64),) + @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" +else + @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch64)" +endif else @echo " (install genisoimage to build centos/ubuntu images)" endif @@ -65,6 +73,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(V)$(DEBUG), --debug) \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$@" \ --force \ --build-image $@, \ @@ -80,6 +89,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -102,6 +112,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(J),--jobs $(J)) \ $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ + $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py new file mode 100644 index 000000000000..bb04cb19c911 --- /dev/null +++ b/tests/vm/aarch64vm.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# +# VM testing aarch64 library +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import os +import sys +import subprocess +import basevm +from qemu.accel import kvm_available + +# This is the config needed for current version of QEMU. +# This works for both kvm and tcg. +CURRENT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", +} + +# The minimum minor version of QEMU we will support with aarch64 VMs is 3. +# QEMU versions less than 3 have various issues running these VMs. +QEMU_AARCH64_MIN_VERSION = 3 + +# The DEFAULT_CONFIG will default to a version of +# parameters that works for backwards compatibility. +DEFAULT_CONFIG = {'kvm' : {'cpu' : "host", + 'machine' : "virt,gic-version=host"}, + 'tcg' : {'cpu' : "cortex-a57", + 'machine' : "virt"}, +} + +def get_config_defaults(vmcls, default_config): + """Fetch the configuration defaults for this VM, + taking into consideration the defaults for + aarch64 first, followed by the defaults for this VM.""" + config = default_config + config.update(aarch_get_config_defaults(vmcls)) + return config + +def aarch_get_config_defaults(vmcls): + """Set the defaults for current version of QEMU.""" + config = CURRENT_CONFIG + args, argv = basevm.parse_args(vmcls) + qemu_path = basevm.get_qemu_path(vmcls.arch, args.build_path) + qemu_version = basevm.get_qemu_version(qemu_path) + if qemu_version < QEMU_AARCH64_MIN_VERSION: + error = "\nThis major version of QEMU {} is to old for aarch64 VMs.\n"\ + "The major version must be at least {}.\n"\ + "To continue with the current build of QEMU, "\ + "please restart with QEMU_LOCAL=1 .\n" + print(error.format(qemu_version, QEMU_AARCH64_MIN_VERSION)) + exit(1) + if qemu_version == QEMU_AARCH64_MIN_VERSION: + # We have an older version of QEMU, + # set the config values for backwards compatibility. + if kvm_available('aarch64'): + config.update(DEFAULT_CONFIG['kvm']) + else: + config.update(DEFAULT_CONFIG['tcg']) + return config + +def create_flash_images(flash_dir="./", efi_img=""): + """Creates the appropriate pflash files + for an aarch64 VM.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + fd_null = open(os.devnull, 'w') + subprocess.check_call(["dd", "if=/dev/zero", "of={}".format(flash0_path), + "bs=1M", "count=64"], + stdout=fd_null, stderr=subprocess.STDOUT) + # A reliable way to get the QEMU EFI image is via an installed package or + # via the bios included with qemu. + if not os.path.exists(efi_img): + sys.stderr.write("*** efi argument is invalid ({})\n".format(efi_img)) + sys.stderr.write("*** please check --efi-aarch64 argument or "\ + "install qemu-efi-aarch64 package\n") + exit(3) + subprocess.check_call(["dd", "if={}".format(efi_img), + "of={}".format(flash0_path), + "conv=notrunc"], + stdout=fd_null, stderr=subprocess.STDOUT) + subprocess.check_call(["dd", "if=/dev/zero", + "of={}".format(flash1_path), + "bs=1M", "count=64"], + stdout=fd_null, stderr=subprocess.STDOUT) + fd_null.close() + +def get_pflash_args(flash_dir="./"): + """Returns a string that can be used to + boot qemu using the appropriate pflash files + for aarch64.""" + flash0_path = get_flash_path(flash_dir, "flash0") + flash1_path = get_flash_path(flash_dir, "flash1") + pflash_args_str = "-drive file={},format=raw,if=pflash "\ + "-drive file={},format=raw,if=pflash" + pflash_args = pflash_args_str.format(flash0_path, flash1_path) + return pflash_args.split(" ") + +def get_flash_path(flash_dir, name): + return os.path.join(flash_dir, "{}.img".format(name)) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index fa56fbbb4b60..5fd66f6b26a1 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -92,6 +92,7 @@ class BaseVM(object): self._guest = None self._genisoimage = args.genisoimage self._build_path = args.build_path + self._efi_aarch64 = args.efi_aarch64 # Allow input config to override defaults. self._config = DEFAULT_CONFIG.copy() if config != None: @@ -496,6 +497,14 @@ def get_qemu_path(arch, build_path=None): qemu_path = "qemu-system-" + arch return qemu_path +def get_qemu_version(qemu_path): + """Get the version number from the current QEMU, + and return the major number.""" + output = subprocess.check_output([qemu_path, '--version']) + version_line = output.decode("utf-8") + version_num = re.split(' |\(', version_line)[3].split('.')[0] + return int(version_num) + def parse_config(config, args): """ Parse yaml config and populate our config structure. The yaml config allows the user to override the @@ -573,6 +582,9 @@ def parse_args(vmcls): parser.add_option("--config", "-c", default=None, help="Provide config yaml for configuration. "\ "See config_example.yaml for example.") + parser.add_option("--efi-aarch64", + default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help="Path to efi image for aarch64 VMs.") parser.disable_interspersed_args() return parser.parse_args() diff --git a/tests/vm/ubuntu.aarch64 b/tests/vm/ubuntu.aarch64 new file mode 100755 index 000000000000..21d454c27f1a --- /dev/null +++ b/tests/vm/ubuntu.aarch64 @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +# +# Ubuntu aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.i386 Fam Zheng +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import sys +import basevm +import aarch64vm +import ubuntuvm + +DEFAULT_CONFIG = { + 'cpu' : "cortex-a57", + 'machine' : "virt,gic-version=3", + 'install_cmds' : "apt-get update,"\ + "apt-get build-dep -y --arch-only qemu,"\ + "apt-get install -y libfdt-dev pkg-config language-pack-en", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM + # especially using softmmu. + 'ssh_timeout' : 60, +} + +class UbuntuAarch64VM(ubuntuvm.UbuntuVM): + name = "ubuntu.aarch64" + arch = "aarch64" + image_name = "ubuntu-18.04-server-cloudimg-arm64.img" + image_link = "https://cloud-images.ubuntu.com/releases/18.04/release/" + image_name + image_sha256="0fdcba761965735a8a903d8b88df8e47f156f48715c00508e4315c506d7d3cb1" + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(UbuntuAarch64VM, self).boot(img, extra_args=extra_args) + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(UbuntuAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(UbuntuAarch64VM, defaults)) From patchwork Tue Jul 7 07:08:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234917 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp654354ilg; Tue, 7 Jul 2020 00:13:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwAWVJpXnVXgWpDgYJ51AET2KtgQnJPyF8cYyrKm0OMmUq18hJ67KAARfK2iODsZcv01LGr X-Received: by 2002:a25:c5c3:: with SMTP id v186mr81329909ybe.233.1594106012369; Tue, 07 Jul 2020 00:13:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106012; cv=none; d=google.com; s=arc-20160816; b=AVKvhLtlV096V1no0eJikSUbTl2QrcFFxcIBkE1Sz8X2Qjd4p0XsduQLDsFn/qujQX FVSgWecx6quooMSLwrDl1WrKtF34lNi/wO/mh5ZORQTOyurUXmz+0YeKcOCXLBSriOXW 22GHw/8+lhuz5HnyxCvHKIXZ0BmSXalI7mh1Kze/hZJVLFkGdyRTz28dHOnDbVSWsDFT slYyl6AQ2KHXZ2fjNxHTrMHWXfxAacJwysqGsfLi0fiu9fCTxazhfPFbYLru4+gHTi7q 7r1JQIkUMiqe+bepNAqT9xfPjgbn1TueMaavObLJi+eKN0rR9IUXvG/vWBUIkw1jc0wl KfBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=73jIj3W6bZqTXI1sT5L61O7DmiRCvL26uReniu5GeKQ=; b=AbftuvM0fT5Ai45WuRzmqblhN6gGtDwpK4T1Z77cTkFSR3+mogJIL6p7aypdruC2Ms MQVYBcLQAnzJjd9ffc/hsCYZYpH1GXRzIXHyC1vD2rKYgLe1uWluViB5eSe0iIgXx8x1 MJIgnWndGYARlLIwKwep3HEoP06Jwgjm7zOuNxOH2axUNiIVTnhCsI2sZBI5C+6fs6lI TZxoOa9jrhGCd2PLT9p8oj2rIWM4v9hvUS7ucTc4chvO9+Z5iW80n6vZGNHCphrJegIC uRuazxu8bnFIQRA9Y9v2J3sE9pihtsLta73ojBn7rs3kHM8KABjyTfqGpwDiNRo0rHb1 P5Ow== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FPssDLyn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j15si202456ybj.489.2020.07.07.00.13.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:13:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FPssDLyn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51500 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshnH-0002qL-OM for patch@linaro.org; Tue, 07 Jul 2020 03:13:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshj8-0004ey-1K for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:14 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:38349) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj5-000453-0J for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:13 -0400 Received: by mail-wm1-x330.google.com with SMTP id f18so45333882wml.3 for ; Tue, 07 Jul 2020 00:09:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=73jIj3W6bZqTXI1sT5L61O7DmiRCvL26uReniu5GeKQ=; b=FPssDLyn2/gipPR3pYN1nBLkNZuzjDwtFic8qVWaCRZaz2LJ00RPQdKFyJmrmL+R+g 1dG42YAXXRzb7jJC94eTOcMusTY05yaYsALhrwK/KvbtwzINjEma3BTEFZrGbthbMdz2 xJimqUgLS4D4EwQrETszeNy1bFXPhQJBRwiCg2ktMdYzfpsDWbPgbGleaobj2pNqYp2Q g9vG6sr1y2YBBAUeVMkQ2rA+eP75Dj+/w/UFJJ7d+4Foq5uZfz1P2RO5CtckUT35jHig WxMYghH8z2vtu2F2DuV0a8jQAj4fnfrkEFi9AnjJl5x7HBpsM5kCzJ+y3k3/ZmhqJblR H/1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=73jIj3W6bZqTXI1sT5L61O7DmiRCvL26uReniu5GeKQ=; b=uamLAkDS/hp+CmxnIP8ukB36xuApmessDT1I0vTYNvXAxQrRjFLo/lSq+g2L5fTOAT QSXzLNAbP6ZpO6QwkXZwICsJ9ir6cbxh/7gtxfkvcE2E8b7mLj9DBGORks9KxzhAy/1q EcUWikugolTK9a91bzuk7ng2HowwLY2cQAnYjAYRWsZQAc4z+dj13r2izYaIWS5SZaN6 0NJZf8WICveSxM1nkcXqjNS4mEdA8TkLvUml/12QDIYAjGBP5n7sN7nWgYsejMVm0312 zrlxjySdgybF1XXC3kdaD3mMGgL1UmpRKQqUrT3TOkBTM90zoHnUrbDN9L90wtYIwWEN kZjA== X-Gm-Message-State: AOAM532t8593swHF1PLc4wMi5QE1d4OKqm60UExeZ29Uy8UYBMxEVcZ/ hGWmmjO0RQEFSJvcEbtTCDYbdw== X-Received: by 2002:a1c:4408:: with SMTP id r8mr2645734wma.100.1594105749314; Tue, 07 Jul 2020 00:09:09 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id q188sm2226723wma.46.2020.07.07.00.09.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:04 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 468C71FF96; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 08/41] tests/vm: Added a new script for centos.aarch64. Date: Tue, 7 Jul 2020 08:08:25 +0100 Message-Id: <20200707070858.6622-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x330.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-7-robert.foley@linaro.org> Message-Id: <20200701135652.1366-11-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 4fa292765d5c..39f918a430a5 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -6,7 +6,7 @@ IMAGES := freebsd netbsd openbsd centos fedora ifneq ($(GENISOIMAGE),) IMAGES += ubuntu.i386 centos ifneq ($(EFI_AARCH64),) -IMAGES += ubuntu.aarch64 +IMAGES += ubuntu.aarch64 centos.aarch64 endif endif @@ -28,6 +28,7 @@ ifneq ($(GENISOIMAGE),) @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM" ifneq ($(EFI_AARCH64),) @echo " vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 VM" + @echo " vm-build-centos.aarch64 - Build QEMU in CentOS aarch64 VM" else @echo " (to build centos/ubuntu aarch64 images use configure --efi-aarch64)" endif diff --git a/tests/vm/centos-8-aarch64.ks b/tests/vm/centos-8-aarch64.ks new file mode 100644 index 000000000000..fd6ebe4d49ed --- /dev/null +++ b/tests/vm/centos-8-aarch64.ks @@ -0,0 +1,51 @@ +# CentOS aarch64 image kickstart file. +# This file is used by the CentOS installer to +# script the generation of the image. +# +# Copyright 2020 Linaro +# +ignoredisk --only-use=vda +# System bootloader configuration +bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda +autopart --type=plain +# Partition clearing information +clearpart --linux --initlabel --drives=vda +# Use text mode install +text +repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream +# Use CDROM installation media +cdrom +# Keyboard layouts +keyboard --vckeymap=us --xlayouts='' +# System language +lang en_US.UTF-8 + +# Network information +network --bootproto=dhcp --device=enp0s1 --onboot=off --ipv6=auto --no-activate +network --hostname=localhost.localdomain +# Run the Setup Agent on first boot +firstboot --enable +# Do not configure the X Window System +skipx +# System services +services --enabled="chronyd" +# System timezone +timezone America/New_York --isUtc + +# Shutdown after installation is complete. +shutdown + +%packages +@^server-product-environment +kexec-tools + +%end + +%addon com_redhat_kdump --enable --reserve-mb='auto' + +%end +%anaconda +pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty +pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok +pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty +%end diff --git a/tests/vm/centos.aarch64 b/tests/vm/centos.aarch64 new file mode 100755 index 000000000000..d5232ecdb8c0 --- /dev/null +++ b/tests/vm/centos.aarch64 @@ -0,0 +1,227 @@ +#!/usr/bin/env python3 +# +# Centos aarch64 image +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# Originally based on ubuntu.aarch64 +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# + +import os +import sys +import subprocess +import basevm +import time +import traceback +import aarch64vm + +DEFAULT_CONFIG = { + 'cpu' : "max", + 'machine' : "virt,gic-version=max", + 'install_cmds' : "yum install -y make git python3 gcc gcc-c++ flex bison, "\ + "yum install -y glib2-devel pixman-devel zlib-devel, "\ + "yum install -y perl-Test-Harness, "\ + "alternatives --set python /usr/bin/python3, "\ + "sudo dnf config-manager "\ + "--add-repo=https://download.docker.com/linux/centos/docker-ce.repo,"\ + "sudo dnf install --nobest -y docker-ce.aarch64,"\ + "systemctl enable docker", + # We increase beyond the default time since during boot + # it can take some time (many seconds) to log into the VM. + 'ssh_timeout' : 60, +} + +class CentosAarch64VM(basevm.BaseVM): + name = "centos.aarch64" + arch = "aarch64" + login_prompt = "localhost login:" + prompt = '[root@localhost ~]#' + image_name = "CentOS-8-aarch64-1905-dvd1.iso" + image_link = "http://mirrors.usc.edu/pub/linux/distributions/centos/8.0.1905/isos/aarch64/" + image_link += image_name + BUILD_SCRIPT = """ + set -e; + cd $(mktemp -d); + sudo chmod a+r /dev/vdb; + tar --checkpoint=.10 -xf /dev/vdb; + ./configure {configure_opts}; + make --output-sync {target} -j{jobs} {verbose}; + """ + def set_key_perm(self): + """Set permissions properly on certain files to allow + ssh access.""" + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v /root/.ssh\n") + self.console_wait_send(self.prompt, + "/usr/sbin/restorecon -R -v "\ + "/home/{}/.ssh\n".format(self._config["guest_user"])) + + def create_kickstart(self): + """Generate the kickstart file used to generate the centos image.""" + # Start with the template for the kickstart. + ks_file = "../tests/vm/centos-8-aarch64.ks" + subprocess.check_call("cp {} ./ks.cfg".format(ks_file), shell=True) + # Append the ssh keys to the kickstart file + # as the post processing phase of installation. + with open("ks.cfg", "a") as f: + # Add in the root pw and guest user. + rootpw = "rootpw --plaintext {}\n" + f.write(rootpw.format(self._config["root_pass"])) + add_user = "user --groups=wheel --name={} "\ + "--password={} --plaintext\n" + f.write(add_user.format(self._config["guest_user"], + self._config["guest_pass"])) + # Add the ssh keys. + f.write("%post --log=/root/ks-post.log\n") + f.write("mkdir -p /root/.ssh\n") + addkey = 'echo "{}" >> /root/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"]) + f.write(addkey_cmd) + f.write('mkdir -p /home/{}/.ssh\n'.format(self._config["guest_user"])) + addkey = 'echo "{}" >> /home/{}/.ssh/authorized_keys\n' + addkey_cmd = addkey.format(self._config["ssh_pub_key"], + self._config["guest_user"]) + f.write(addkey_cmd) + f.write("%end\n") + # Take our kickstart file and create an .iso from it. + # The .iso will be provided to qemu as we boot + # from the install dvd. + # Anaconda will recognize the label "OEMDRV" and will + # start the automated installation. + gen_iso_img = 'genisoimage -output ks.iso -volid "OEMDRV" ks.cfg' + subprocess.check_call(gen_iso_img, shell=True) + + def wait_for_shutdown(self): + """We wait for qemu to shutdown the VM and exit. + While this happens we display the console view + for easier debugging.""" + # The image creation is essentially done, + # so whether or not the wait is successful we want to + # wait for qemu to exit (the self.wait()) before we return. + try: + self.console_wait("reboot: Power down") + except Exception as e: + sys.stderr.write("Exception hit\n") + if isinstance(e, SystemExit) and e.code == 0: + return 0 + traceback.print_exc() + finally: + self.wait() + + def build_base_image(self, dest_img): + """Run through the centos installer to create + a base image with name dest_img.""" + # We create the temp image, and only rename + # to destination when we are done. + img = dest_img + ".tmp" + # Create an empty image. + # We will provide this as the install destination. + qemu_img_create = "qemu-img create {} 50G".format(img) + subprocess.check_call(qemu_img_create, shell=True) + + # Create our kickstart file to be fed to the installer. + self.create_kickstart() + # Boot the install dvd with the params as our ks.iso + os_img = self._download_with_cache(self.image_link) + dvd_iso = "centos-8-dvd.iso" + subprocess.check_call(["cp", "-f", os_img, dvd_iso]) + extra_args = "-cdrom ks.iso" + extra_args += " -drive file={},if=none,id=drive1,cache=writeback" + extra_args += " -device virtio-blk,drive=drive1,bootindex=1" + extra_args = extra_args.format(dvd_iso).split(" ") + self.boot(img, extra_args=extra_args) + self.console_wait_send("change the selection", "\n") + # We seem to need to hit esc (chr(27)) twice to abort the + # media check, which takes a long time. + # Waiting a bit seems to be more reliable before hitting esc. + self.console_wait("Checking") + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + time.sleep(5) + self.console_wait_send("Checking", chr(27)) + print("Found Checking") + # Give sufficient time for the installer to create the image. + self.console_init(timeout=7200) + self.wait_for_shutdown() + os.rename(img, dest_img) + print("Done with base image build: {}".format(dest_img)) + + def check_create_base_img(self, img_base, img_dest): + """Create a base image using the installer. + We will use the base image if it exists. + This helps cut down on install time in case we + need to restart image creation, + since the base image creation can take a long time.""" + if not os.path.exists(img_base): + print("Generate new base image: {}".format(img_base)) + self.build_base_image(img_base); + else: + print("Use existing base image: {}".format(img_base)) + # Save a copy of the base image and copy it to dest. + # which we will use going forward. + subprocess.check_call(["cp", img_base, img_dest]) + + def boot(self, img, extra_args=None): + aarch64vm.create_flash_images(self._tmpdir, self._efi_aarch64) + default_args = aarch64vm.get_pflash_args(self._tmpdir) + if extra_args: + extra_args.extend(default_args) + else: + extra_args = default_args + # We always add these performance tweaks + # because without them, we boot so slowly that we + # can time out finding the boot efi device. + if '-smp' not in extra_args and \ + '-smp' not in self._config['extra_args'] and \ + '-smp' not in self._args: + # Only add if not already there to give caller option to change it. + extra_args.extend(["-smp", "8"]) + # We have overridden boot() since aarch64 has additional parameters. + # Call down to the base class method. + super(CentosAarch64VM, self).boot(img, extra_args=extra_args) + + def build_image(self, img): + img_tmp = img + ".tmp" + self.check_create_base_img(img + ".base", img_tmp) + + # Boot the new image for the first time to finish installation. + self.boot(img_tmp) + self.console_init() + self.console_wait_send(self.login_prompt, "root\n") + self.console_wait_send("Password:", + "{}\n".format(self._config["root_pass"])) + + self.set_key_perm() + self.console_wait_send(self.prompt, "rpm -q centos-release\n") + enable_adapter = "sed -i 's/ONBOOT=no/ONBOOT=yes/g'" \ + " /etc/sysconfig/network-scripts/ifcfg-enp0s1\n" + self.console_wait_send(self.prompt, enable_adapter) + self.console_wait_send(self.prompt, "ifup enp0s1\n") + self.console_wait_send(self.prompt, + 'echo "qemu ALL=(ALL) NOPASSWD:ALL" | '\ + 'sudo tee /etc/sudoers.d/qemu\n') + self.console_wait(self.prompt) + + # Rest of the commands we issue through ssh. + self.wait_ssh(wait_root=True) + + # If the user chooses *not* to do the second phase, + # then we will jump right to the graceful shutdown + if self._config['install_cmds'] != "": + install_cmds = self._config['install_cmds'].split(',') + for cmd in install_cmds: + self.ssh_root(cmd) + self.ssh_root("poweroff") + self.wait_for_shutdown() + os.rename(img_tmp, img) + print("image creation complete: {}".format(img)) + return 0 + +if __name__ == "__main__": + defaults = aarch64vm.get_config_defaults(CentosAarch64VM, DEFAULT_CONFIG) + sys.exit(basevm.main(CentosAarch64VM, defaults)) From patchwork Tue Jul 7 07:08:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234923 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp656015ilg; Tue, 7 Jul 2020 00:16:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw0kf4LP6fWIqfYG6LLGBFNRh6IWwp+pS39vTMx/YtvqbAjJlK2mfC+VnXj4dHZdwFYbu/i X-Received: by 2002:a05:6902:102e:: with SMTP id x14mr30701176ybt.461.1594106183772; Tue, 07 Jul 2020 00:16:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106183; cv=none; d=google.com; s=arc-20160816; b=se/ollGNNg9Ok7jiXKwbs3hYXR97M99uSU1KgKwAaC7uRCxEbXeJKSm/SJ+6n8Pgy/ RzS7iXyaRR0CID3HR4K7qzGCpXxjtf1CIjYGdPDKV2CIo0f/GaI8SJupLkEPRiWJfV8J D9RBhDjDU8U692w0mAoZkBtaF0kM2eHx5Oho+kBQppAOQxWIt4/iTOf6JZ9C5yQukG5b qNnF7is0JaEEIm5l9mdUMq5+GJIb7qhgkW/7VUFPHSHdXuKJCJieZCert6klRoNuns4L CcKg591qf947IBmkOc0m/bRSnvbpAoxIpn/Cy7VcAdOCe7brNuOkSwZG259Z0qUwLMre eVXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wXN1VAPLVX42DG/MXYUsKGX14V7wDDkXJy0egf9Mkr4=; b=Gh6xN4ZLm7KrT1AdtuSDMWzfpVW4HG662+medqrgZ8WbgEK27K32TqbOUox671toAf lX5RWhSLNAM4qm8q/aRhnZkrq9sF6mFWiAyWQBWd0l8MJCKSorb3RyARaN01ZKmrdw+D 0iA/D61UCDm16SzPBSLo48n1GXbE/cHa7eeNQr+h1cRK5WFQocVSQxEw6+x8lEJkJy9T 4Rw8cvyUwFyUt3vAze+lvz2gQULC5e5aZEIsmIUU0hqI/4cdyy1QsDsVz0Ep0sGdj3vu vTf/AJQoOVN0q5+Kh1jsn6F9Furm2/JgJBbin9nR8ucOoAsCgw7yZY9wbKVQIqLHrwuz uUeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qdRpy8KB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w11si19975117ybl.23.2020.07.07.00.16.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:16:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qdRpy8KB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35224 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshq3-000879-4z for patch@linaro.org; Tue, 07 Jul 2020 03:16:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjA-0004lO-5Y for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:16 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:54936) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj7-00045I-68 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:15 -0400 Received: by mail-wm1-x344.google.com with SMTP id o8so42101474wmh.4 for ; Tue, 07 Jul 2020 00:09:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wXN1VAPLVX42DG/MXYUsKGX14V7wDDkXJy0egf9Mkr4=; b=qdRpy8KBobvBXvxk1HsQN2NFBeJYQabgfqImXreh2XCZK2rYB2OaJ4FFb2qMnbm4Vw xDa+Uw0ZATSvm5Doa9uO8axekWToYljAdLWkggA4+F1IGY0pFkZuJYa2OBhSj7XM2fkQ 74v9NDo563dx6goMyhHSJC+Y5ScSlzAcq6+D5aN5bOj1shVlx7a00IDEz87vkdg0Enox VEDMgRTggECUIMTc69+gVdk9Z7HslF8LX0LyfIdNzO8RlOai5GS2Kzb42LofcLnX45K4 AkmSQn3fv3MkkW299BMuR3+D90pnT4aV+QK/aL7rhajNVBmTfVgLiOF4FC65JiLGGk+1 Rqdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wXN1VAPLVX42DG/MXYUsKGX14V7wDDkXJy0egf9Mkr4=; b=cL4JCVaY8UuhkS7ra5GmpdSRDi4zR6Sy7UXVWPkCuoIwkh1SzKi2afuHx9fCWKMEZu 32qLQHzF3D0uvczSjFA+MtFjBgDqCp5AdVWrrtJG3+zrEGwbPtjsrJ7Ko8SPv1DkxOEU CcMfOdChVoqzIAP2T8LPNkr9lnBOIWqD+yGGGphQvbV2Y9KCVAxiTQFyLuXUJzaDCGuU W6n0Dm5GxvFNQZ1xcgtt/YDuRMYeF3WMXoufz8lBdGU6fnuw1zKWXLpsmiKCJzkEEuDN 3gVHVUdmmFeS0ZD3oTOBFB0/LZiiJbv29+yLVH4YJaz69U114AskKUG2QU5sucNGSMOM 0QQg== X-Gm-Message-State: AOAM532e9gEOoFzAHcDCSVCNIa57vIBKoQhZPjqRCya4enuGbv5UkC1i JsrJ2J+8ntg6x/nzLepTeT93RJSkYTE= X-Received: by 2002:a1c:6805:: with SMTP id d5mr2571159wmc.19.1594105751682; Tue, 07 Jul 2020 00:09:11 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id g3sm18597491wrb.59.2020.07.07.00.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:05 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 625F11FF98; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 09/41] tests/vm: change scripts to use self._config Date: Tue, 7 Jul 2020 08:08:26 +0100 Message-Id: <20200707070858.6622-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::344; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x344.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200601211421.1277-8-robert.foley@linaro.org> Message-Id: <20200701135652.1366-12-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 5fd66f6b26a1..f716798b405e 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -183,13 +183,6 @@ class BaseVM(object): self.console_init(timeout=timeout) self.console_wait(wait_string) - def __getattr__(self, name): - # Support direct access to config by key. - # for example, access self._config['cpu'] by self.cpu - if name.lower() in self._config.keys(): - return self._config[name.lower()] - return object.__getattribute__(self, name) - def _download_with_cache(self, url, sha256sum=None, sha512sum=None): def check_sha256sum(fname): if not sha256sum: @@ -239,13 +232,13 @@ class BaseVM(object): return r def ssh(self, *cmd): - return self._ssh_do(self.GUEST_USER, cmd, False) + return self._ssh_do(self._config["guest_user"], cmd, False) def ssh_root(self, *cmd): return self._ssh_do("root", cmd, False) def ssh_check(self, *cmd): - self._ssh_do(self.GUEST_USER, cmd, True) + self._ssh_do(self._config["guest_user"], cmd, True) def ssh_root_check(self, *cmd): self._ssh_do("root", cmd, True) diff --git a/tests/vm/fedora b/tests/vm/fedora index a9195670f4bd..b2b478fdbcac 100755 --- a/tests/vm/fedora +++ b/tests/vm/fedora @@ -108,20 +108,20 @@ class FedoraVM(basevm.BaseVM): self.console_wait_send("7) [!] Root password", "7\n") self.console_wait("Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("8) [ ] User creation", "8\n") self.console_wait_send("1) [ ] Create user", "1\n") self.console_wait_send("3) User name", "3\n") - self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER) + self.console_wait_send("ENTER:", "%s\n" % self._config["guest_user"]) self.console_wait_send("4) [ ] Use password", "4\n") self.console_wait_send("5) Password", "5\n") self.console_wait("Password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password (confirm):") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("7) Groups", "c\n") while True: @@ -139,7 +139,7 @@ class FedoraVM(basevm.BaseVM): if good: break time.sleep(10) - self.console_send("r\n" % self.GUEST_PASS) + self.console_send("r\n" % self._config["guest_pass"]) self.console_wait_send("'b' to begin install", "b\n") @@ -150,12 +150,13 @@ class FedoraVM(basevm.BaseVM): # setup qemu user prompt = " ~]$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = " ~]#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/freebsd b/tests/vm/freebsd index f87db2b126e6..29252fa4a64a 100755 --- a/tests/vm/freebsd +++ b/tests/vm/freebsd @@ -113,9 +113,9 @@ class FreeBSDVM(basevm.BaseVM): # post-install configuration self.console_wait("New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype New Password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Network Configuration", "\n") self.console_wait_send("IPv4", "y") @@ -134,9 +134,9 @@ class FreeBSDVM(basevm.BaseVM): # qemu user self.console_wait_send("Add User Accounts", "y") self.console_wait("Username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait_send("Uid", "\n") self.console_wait_send("Login group", "\n") self.console_wait_send("Login group", "\n") @@ -148,9 +148,9 @@ class FreeBSDVM(basevm.BaseVM): self.console_wait_send("Use an empty password", "\n") self.console_wait_send("Use a random password", "\n") self.console_wait("Enter password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Enter password again:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Lock out", "\n") self.console_wait_send("OK", "yes\n") self.console_wait_send("Add another user", "no\n") @@ -164,12 +164,12 @@ class FreeBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "root@freebsd:~ #" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup serial console diff --git a/tests/vm/netbsd b/tests/vm/netbsd index cdac502dad86..dc0e45c1d45a 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -120,24 +120,24 @@ class NetBSDVM(basevm.BaseVM): self.console_wait_send("d: Change root password", "d\n") self.console_wait_send("a: Yes", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("o: Add a user", "o\n") self.console_wait("username") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("to group wheel") self.console_wait_send("a: Yes", "a\n") self.console_wait_send("a: /bin/sh", "a\n") self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("New password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Retype new password:") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("a: Configure network", "a\n") self.console_wait_send("a: vioif0", "a\n") @@ -170,12 +170,13 @@ class NetBSDVM(basevm.BaseVM): # setup qemu user prompt = "localhost$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "localhost#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) diff --git a/tests/vm/openbsd b/tests/vm/openbsd index 13e7f9a6d56f..dfe633e45325 100755 --- a/tests/vm/openbsd +++ b/tests/vm/openbsd @@ -98,9 +98,9 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which network interface", "done\n") self.console_wait_send("DNS domain name", "localnet\n") self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait("Password for root account") - self.console_send("%s\n" % self.ROOT_PASS) + self.console_send("%s\n" % self._config["root_pass"]) self.console_wait_send("Start sshd(8)", "yes\n") self.console_wait_send("X Window System", "\n") self.console_wait_send("xenodm", "\n") @@ -108,13 +108,13 @@ class OpenBSDVM(basevm.BaseVM): self.console_wait_send("Which speed", "\n") self.console_wait("Setup a user") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Full name") - self.console_send("%s\n" % self.GUEST_USER) + self.console_send("%s\n" % self._config["guest_user"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait("Password") - self.console_send("%s\n" % self.GUEST_PASS) + self.console_send("%s\n" % self._config["guest_pass"]) self.console_wait_send("Allow root ssh login", "yes\n") self.console_wait_send("timezone", "UTC\n") @@ -135,12 +135,13 @@ class OpenBSDVM(basevm.BaseVM): # setup qemu user prompt = "$" - self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_ssh_init(prompt, self._config["guest_user"], + self._config["guest_pass"]) self.console_wait_send(prompt, "exit\n") # setup root user prompt = "openbsd#" - self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_ssh_init(prompt, "root", self._config["root_pass"]) self.console_sshd_config(prompt) # setup virtio-blk #1 (tarfile) From patchwork Tue Jul 7 07:08:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234919 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp654388ilg; Tue, 7 Jul 2020 00:13:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzBtPIQ7GJiD6XocSvex8TxW94JSuCCStib+BT7TMAPt+WHNFAh++r6ORWp88pPXHECV6TO X-Received: by 2002:a25:9907:: with SMTP id z7mr3304136ybn.185.1594106015465; Tue, 07 Jul 2020 00:13:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106015; cv=none; d=google.com; s=arc-20160816; b=ZRICNCvDq3wHthAPcclkk3ZQu56qlqLT72/9iJfGwrIounfqb8q1lQj7hs4vu+X3HG ppeo7+YAg+hJqiUcj1C3a72wk1m3fabuz8rCyol46YQzA9rpYDCM5LR+RnGfrxRFlzc6 2+ZLLjPJII2gtCKgqiG7Hj3RgNVxaO3LDQ93Iljy4kC5b4ota7gdmUdjqjEA3e1goWL3 5p/hIRClxzHHCYj6GDvDj4TMtpXoEIHEoU2tH9j7pboYkvvQbqUwCg4mD5uKzmh5DsTU PoOR17l2N1A4nSgqz2Vp4X/pBPWyiU627CltD6fUTamrmMfUWKMbiSZk2ThEk7W5oPn3 PYbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gDtGcpZv1Rvonc6KqJMOc0i8D0gnp8nnG/k6JUeeZcs=; b=dMpPxMrjohPgRz2H2yHW+uG7yvDcmDC8RlSG/ocWiY4SHYocDwlZ6rRjg/vqOeJo30 oE2K455nxYffbWe+feYXlQByRqjcolNRvE7jQZqbrQj3ZMpR4kALvZymSkKIWqNhWEmC PpGDr8uwf+ghRhrdHmg+Qd3nx9jJPNTZ1s8c4U3RFes4w6MpgJ7i6AeMG798UT6fay0+ VN0SZ8XxHihEQcK059f6dTkWXaDg3/QJNw7AaOtPy48UFa7Z+FKSqx4mwvEOqbYHQ4hB 5wUGzn61kdMFUkEFi5c+JWNfQo+aADJkmErCUX6eb47+UNl+v/ZWUeVQZIVe+C7v5YK8 1N2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DuB2go+S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 203si10656451ybl.405.2020.07.07.00.13.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:13:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DuB2go+S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshnK-00032u-UL for patch@linaro.org; Tue, 07 Jul 2020 03:13:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42586) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjB-0004oq-PC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:17 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:44809) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshj8-00045T-4p for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:17 -0400 Received: by mail-wr1-x436.google.com with SMTP id b6so43950843wrs.11 for ; Tue, 07 Jul 2020 00:09:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gDtGcpZv1Rvonc6KqJMOc0i8D0gnp8nnG/k6JUeeZcs=; b=DuB2go+SQJv57EfL3j3/236c9S28H6/YCp5R3Q0HGig60gbZTyddROdVRVbQhLX9QG QdiIT+tEcXC5vZvjWQabR8vkEj5EvTPCos3waRQseTrB55ZUEiTPszAKCc6Ep3NY86fc 1mDL5fImXKzP99QclIWy8qFjmPFvB607Sj1gkY935crM4SHE8+dGTRhs6SaQAt3s+XxO sqYFY8U8Qer4b7YiF/Q2ZHM9ehN3wBxpxeH+Y8Lvzz571whzbBXnuk07weOsQg0JSW/d /uPcUZUt99kyKqebqTRQ9Y7b6LThqdcYvKEVUIllAmt/fa5yThQmwrFbTURhQPzTqfxc tSkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gDtGcpZv1Rvonc6KqJMOc0i8D0gnp8nnG/k6JUeeZcs=; b=IRZg5Aax2Amvhw1KD8cpt6X/Zke50W0sb0DEYvpvMR6RthUKhM/V6JOMTnzMJOcjcc W9HO6Xg6vY2MsYvwrqRdou6b8MTtc15nxTTVc9CQA6EQLruwE0eannevDhU+KW74mzSF TJJPfcOX8c0tylkc1gL/yzTeGHb5H3GMJLE+tzae54Md8c19BPZ5ezn3TqXZu+v+Mz4c N+x/rSx7NaxvLmKUkuzJOucqiNWC5pBv3+JDJnFSjwNvITwd3cyWkfemV+sQTZ0sp1DQ v2vdvD2IOo1gnQFFqNh4z16AZtvoUZtllR7bmXYchChZPYjungCTUVsFXxDUtmNVWAQL 1c2w== X-Gm-Message-State: AOAM532/rBiNDr3GeH3NOz4IYn9TtjqoovpvE6y94ZV0l1cz9AeCymLg M0h4nWxzJQaqzfnMttDgm0XpS1dbeug= X-Received: by 2002:adf:8024:: with SMTP id 33mr56289077wrk.117.1594105752686; Tue, 07 Jul 2020 00:09:12 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u2sm2137834wml.16.2020.07.07.00.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:05 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 7A15A1FF99; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 10/41] python/qemu: Add ConsoleSocket for optional use in QEMUMachine Date: Tue, 7 Jul 2020 08:08:27 +0100 Message-Id: <20200707070858.6622-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , qemu-devel@nongnu.org, Cleber Rosa , Peter Puhov , =?utf-8?q?Alex_Benn=C3=A9e?= , Robert Foley Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley We add the ConsoleSocket object, which has a socket interface and which will consume all arriving characters on the socket, placing them into an in memory buffer. This will also provide those chars via recv() as would a regular socket. ConsoleSocket also has the option of dumping the console bytes to a log file. We also give QEMUMachine the option of using ConsoleSocket to drain and to use for logging console to a file. By default QEMUMachine does not use ConsoleSocket. This is added in preparation for use by basevm.py in a later commit. This is a workaround we found was needed for basevm.py since there is a known issue where QEMU will hang waiting for console characters to be consumed. Cc: Eduardo Habkost Cc: Cleber Rosa Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée Tested-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-9-robert.foley@linaro.org> Message-Id: <20200701135652.1366-13-alex.bennee@linaro.org> -- 2.20.1 diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py new file mode 100644 index 000000000000..830cb7c62825 --- /dev/null +++ b/python/qemu/console_socket.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +# +# This python module implements a ConsoleSocket object which is +# designed always drain the socket itself, and place +# the bytes into a in memory buffer for later processing. +# +# Optionally a file path can be passed in and we will also +# dump the characters to this file for debug. +# +# Copyright 2020 Linaro +# +# Authors: +# Robert Foley +# +# This code is licensed under the GPL version 2 or later. See +# the COPYING file in the top-level directory. +# +import asyncore +import socket +import threading +import io +import os +import sys +from collections import deque +import time +import traceback + +class ConsoleSocket(asyncore.dispatcher): + + def __init__(self, address, file=None): + self._recv_timeout_sec = 300 + self._buffer = deque() + self._asyncore_thread = None + self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self._sock.connect(address) + self._logfile = None + if file: + self._logfile = open(file, "w") + asyncore.dispatcher.__init__(self, sock=self._sock) + self._open = True + self._thread_start() + + def _thread_start(self): + """Kick off a thread to wait on the asyncore.loop""" + if self._asyncore_thread is not None: + return + self._asyncore_thread = threading.Thread(target=asyncore.loop, + kwargs={'timeout':1}) + self._asyncore_thread.daemon = True + self._asyncore_thread.start() + + def handle_close(self): + """redirect close to base class""" + # Call the base class close, but not self.close() since + # handle_close() occurs in the context of the thread which + # self.close() attempts to join. + asyncore.dispatcher.close(self) + + def close(self): + """Close the base object and wait for the thread to terminate""" + if self._open: + self._open = False + asyncore.dispatcher.close(self) + if self._asyncore_thread is not None: + thread, self._asyncore_thread = self._asyncore_thread, None + thread.join() + if self._logfile: + self._logfile.close() + self._logfile = None + + def handle_read(self): + """process arriving characters into in memory _buffer""" + try: + data = asyncore.dispatcher.recv(self, 1) + # latin1 is needed since there are some chars + # we are receiving that cannot be encoded to utf-8 + # such as 0xe2, 0x80, 0xA6. + string = data.decode("latin1") + except: + print("Exception seen.") + traceback.print_exc() + return + if self._logfile: + self._logfile.write("{}".format(string)) + self._logfile.flush() + for c in string: + self._buffer.extend(c) + + def recv(self, n=1, sleep_delay_s=0.1): + """Return chars from in memory buffer""" + start_time = time.time() + while len(self._buffer) < n: + time.sleep(sleep_delay_s) + elapsed_sec = time.time() - start_time + if elapsed_sec > self._recv_timeout_sec: + raise socket.timeout + chars = ''.join([self._buffer.popleft() for i in range(n)]) + # We choose to use latin1 to remain consistent with + # handle_read() and give back the same data as the user would + # receive if they were reading directly from the + # socket w/o our intervention. + return chars.encode("latin1") + + def set_blocking(self): + """Maintain compatibility with socket API""" + pass + + def settimeout(self, seconds): + """Set current timeout on recv""" + self._recv_timeout_sec = seconds diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 041c615052e4..c25f0b42cf60 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,7 @@ import socket import tempfile from typing import Optional, Type from types import TracebackType +from qemu.console_socket import ConsoleSocket from . import qmp @@ -75,7 +76,8 @@ class QEMUMachine: def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None, sock_dir=None): + socket_scm_helper=None, sock_dir=None, + drain_console=False, console_log=None): ''' Initialize a QEMUMachine @@ -86,6 +88,9 @@ class QEMUMachine: @param test_dir: where to create socket and log file @param monitor_address: address for QMP monitor @param socket_scm_helper: helper program, required for send_fd_scm() + @param sock_dir: where to create socket (overrides test_dir for sock) + @param console_log: (optional) path to console log file + @param drain_console: (optional) True to drain console socket to buffer @note: Qemu process is not started until launch() is used. ''' if args is None: @@ -122,6 +127,12 @@ class QEMUMachine: self._console_address = None self._console_socket = None self._remove_files = [] + self._console_log_path = console_log + if self._console_log_path: + # In order to log the console, buffering needs to be enabled. + self._drain_console = True + else: + self._drain_console = drain_console def __enter__(self): return self @@ -580,7 +591,11 @@ class QEMUMachine: Returns a socket connected to the console """ if self._console_socket is None: - self._console_socket = socket.socket(socket.AF_UNIX, - socket.SOCK_STREAM) - self._console_socket.connect(self._console_address) + if self._drain_console: + self._console_socket = ConsoleSocket(self._console_address, + file=self._console_log_path) + else: + self._console_socket = socket.socket(socket.AF_UNIX, + socket.SOCK_STREAM) + self._console_socket.connect(self._console_address) return self._console_socket From patchwork Tue Jul 7 07:08:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234924 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp656364ilg; Tue, 7 Jul 2020 00:16:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxZ4fjVCzWef4FWHbU+sdBhnFi4AKfcWKzRdEG4zvsZ4hs7lVEilMHJstiWx30ugwoJ2HDB X-Received: by 2002:a25:4f85:: with SMTP id d127mr1344989ybb.155.1594106216262; Tue, 07 Jul 2020 00:16:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106216; cv=none; d=google.com; s=arc-20160816; b=bWmACQ9Tqq7oteDNsY7FXTa09XcZMyKfaSX4n8c0Y9QghzRZCOnhYVpwK1uzicziS0 rCZAU/BGAQfet+5htKaBXcZkoVyL+Bhz/CsfZv4jk38qGqqNEWk6i9M6fat+Zo3m6iB1 aKxs7kYKz9wMUIJ6rpk2j/vLQk/XuvVJn2lz1m4krusRKdl08q5yNi/p/7AsFNV2VXHG R5K3PnghaKpAx96hPDVw3KZQD4VP5hFQGQ7+Ms9GSp0P9Snx2hXVMMK6lZ/96wr6e6bU 4P6hhSNjfdUVB8T4o6G7mhQyUOiLsBAaZTt2xkS22rndhfqpqP4T/CSrL0iQ+mD2X176 f1Hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=3f/VHOspddBHT4k11DZu6PifAAchG3ueuum87F8Ii4I=; b=Fp8cuncYGU22oziisSQUPP2loY601F/2vmBJ2eqPy9erOLQ1hH4l6c5i49Pfp+Sr7L BWtPYxEtsVQDfIRS93Qt2i9QlH5c1/Q0yy7fBdt++608ulZ2ejioW+OBuJGSSo8pk5ny RWK7GnN4nVB2JMWDebS+bTJtO24JgFaL2UkS5Hmxi2pYJs1IQowYQPwF54IB96e0lRWR i1AVxynl5Zff/+1VCRnJYywfNGu+YG9KqiHzuIfHnDuXGFtX4bkFVXkzqNCMr62sb38A BmS8h1F05VajKYBlGltGQRFSeoeZd54ke8/jI5DJybmBDVaA5RDIOI+XoWXyNcwnHM1y Z/qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=e4GgYJSG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m9si18878773ybt.420.2020.07.07.00.16.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:16:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=e4GgYJSG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshqZ-0001oG-MV for patch@linaro.org; Tue, 07 Jul 2020 03:16:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42660) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjF-0004wn-Kl for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:21 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]:40327) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjD-00046L-Ku for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:21 -0400 Received: by mail-wr1-x42a.google.com with SMTP id f2so15988767wrp.7 for ; Tue, 07 Jul 2020 00:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3f/VHOspddBHT4k11DZu6PifAAchG3ueuum87F8Ii4I=; b=e4GgYJSGSdUDdTvWp4DXuGcsoHDKKRQjBXo34oPgu/tD75FJETV0mNfIvfGa98zmK4 EM+MSZWMyZaaMN5MGhTCfBd9s7VIaFD8W/NLpkX38ZlIOxnp4hMxzmL+Yt68lxboZ2Nl y8bCL151xoeOHjR9YvcDB7+j2CK7LUrVhMczcFUbk3Hdqazl12dUkF2C6EvhSe+tJLoJ /TnGxgwWRxt/mh3kFjuSUqiR0Zj4PKao9g8GbSeQNqVrki+X7zoxTQzOOYMLQjTSV+k/ JEjqE7NFyjVJsPm6HGfEAx6vzgsVLsR+SHFirvh9QkQUdsSJkoiCqFor6yvME+8skBM4 EEAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3f/VHOspddBHT4k11DZu6PifAAchG3ueuum87F8Ii4I=; b=Uo0wYZu4cwpS1FvRr73DlL9zNTsuiZXlQaibDaPMyjSSqdEJJM0xrB01PJ9SBsjpEj tzZ1Hc7WcXTgP8T+mvukl9nSSuKC2zRQMvxZi7BSZDy5HZ+iH1vW5DbFAyt+Rbx+2HBM KXpbo09RqRwh/vRvM3SO4ijXXNNp9W/BZ0Fn+oWYktk+plHILTQOHc0WISMVYJtH6RTJ ihFLf3oblK30Xfeia5SGOCa/65grwmsd5zY8uozH/2LZVrBoEnDX1ufQvn3syJL4xMKR 5Umb2s7i0ys7jJf2jSI2O0eyst73lOvC62Rf5kCqMOUAO8BnrDP6lPcUt/gQiyBNT9q0 o2hw== X-Gm-Message-State: AOAM5334Vzh/RJwthQA2E3o+qACUXooZGT4X3id0UZQ85g7cCrfdZBJR fzITa6nUJNcmKZR5ct1pt40Wlg== X-Received: by 2002:adf:f14e:: with SMTP id y14mr52429711wro.151.1594105758202; Tue, 07 Jul 2020 00:09:18 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u2sm2137896wml.16.2020.07.07.00.09.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:10 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 922191FF9A; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 11/41] tests/vm: Add workaround to consume console Date: Tue, 7 Jul 2020 08:08:28 +0100 Message-Id: <20200707070858.6622-12-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42a; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Peter Puhov , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley This adds support to basevm.py so that we always drain the console chars. This makes use of support added in an earlier commit that allows QEMUMachine to use the ConsoleSocket. This is a workaround we found was needed since there is a known issue where QEMU will hang waiting for console characters to be consumed. We also added the option of logging the console to a file. LOG_CONSOLE=1 will now log the output to a file. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200601211421.1277-10-robert.foley@linaro.org> Message-Id: <20200701135652.1366-14-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index 39f918a430a5..f21948c46a52 100644 --- a/tests/vm/Makefile.include +++ b/tests/vm/Makefile.include @@ -49,6 +49,7 @@ endif @echo ' EXTRA_CONFIGURE_OPTS="..."' @echo " J=[0..9]* - Override the -jN parameter for make commands" @echo " DEBUG=1 - Enable verbose output on host and interactive debugging" + @echo " LOG_CONSOLE=1 - Log console to file in: ~/.cache/qemu-vm " @echo " V=1 - Enable verbose ouput on host and guest commands" @echo " QEMU_LOCAL=1 - Use QEMU binary local to this build." @echo " QEMU=/path/to/qemu - Change path to QEMU binary" @@ -75,6 +76,7 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \ $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$@" \ --force \ --build-image $@, \ @@ -91,6 +93,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(V),--verbose) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ --snapshot \ @@ -114,6 +117,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(if $(V)$(DEBUG), --debug) \ $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \ $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \ + $(if $(LOG_CONSOLE),--log-console) \ --image "$<" \ --interactive \ false, \ diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index f716798b405e..04d083409a5f 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -117,6 +117,11 @@ class BaseVM(object): "w").write(self._config['ssh_pub_key']) self.debug = args.debug + self._console_log_path = None + if args.log_console: + self._console_log_path = \ + os.path.join(os.path.expanduser("~/.cache/qemu-vm"), + "{}.install.log".format(self.name)) self._stderr = sys.stderr self._devnull = open(os.devnull, "w") if self.debug: @@ -271,7 +276,13 @@ class BaseVM(object): args += self._data_args + extra_args + self._config['extra_args'] logging.debug("QEMU args: %s", " ".join(args)) qemu_path = get_qemu_path(self.arch, self._build_path) - guest = QEMUMachine(binary=qemu_path, args=args) + + # Since console_log_path is only set when the user provides the + # log_console option, we will set drain_console=True so the + # console is always drained. + guest = QEMUMachine(binary=qemu_path, args=args, + console_log=self._console_log_path, + drain_console=True) guest.set_machine(self._config['machine']) guest.set_console() try: @@ -285,6 +296,8 @@ class BaseVM(object): raise atexit.register(self.shutdown) self._guest = guest + # Init console so we can start consuming the chars. + self.console_init() usernet_info = guest.qmp("human-monitor-command", command_line="info usernet") self.ssh_port = None @@ -296,7 +309,9 @@ class BaseVM(object): raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \ usernet_info) - def console_init(self, timeout = 120): + def console_init(self, timeout = None): + if timeout == None: + timeout = self.socket_timeout vm = self._guest vm.console_socket.settimeout(timeout) self.console_raw_path = os.path.join(vm._temp_dir, @@ -578,6 +593,8 @@ def parse_args(vmcls): parser.add_option("--efi-aarch64", default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", help="Path to efi image for aarch64 VMs.") + parser.add_option("--log-console", action="store_true", + help="Log console to file.") parser.disable_interspersed_args() return parser.parse_args() From patchwork Tue Jul 7 07:08:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234926 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp656809ilg; Tue, 7 Jul 2020 00:17:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwbAL3cCabsyWidoOxOIfUUrpNe6EI+G9rc4bcbF2JhpPdcFO2Of06avYqiimjrwvwScEmJ X-Received: by 2002:a25:8e08:: with SMTP id p8mr32828613ybl.63.1594106263449; Tue, 07 Jul 2020 00:17:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106263; cv=none; d=google.com; s=arc-20160816; b=AenxQ01E5ZAU+XyVSMNy/GXQZyvW+nQJcm+Gy9DfWvS79E0PL/8442TL7E5zTRKw7q lWArT4eqbx2TJihhl2dj6L0mrhfXg2rJV3QYFbfl04Le5sT75DWziYfrv3e9QCfo9w4T WwUaSuQOXxhqxJ+R+fSbCUueHlXAXYUXRMb7NsVzLJDLADSykgn2+MjsOZT3zQnT0PGh rV/AEJcPJT0vL5or2nLvR/l+88ZWWanmspUpt+W5P/uH8MHAvUPyeGBMGcV6kdVx2Hzc 3glLsKXDeBG6pQBR+Z5IeIa1mN4aBf2qJxR5bC4G90FQ2XpwtBi2Jv65N6sHXnY7A/c3 vcfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=msa/RJrC+RYm3qouK0hknnnvEiN6J+9sLGN+6EdqCHk=; b=tqYcvJsb0oJfjG3rSZzvLR1Y0D9Jq5Gzlrx9WeicEdLBn8rIoyR3pJ6n59DMdejuaJ VE6dg5bB8v/py7bbAiUAHj912nV3s4y2cxyaR1mDRp4Qu2SKE6BlDj2cUkTYM4uH+Spg /mRVUh3o/FYaL+hYJN7Y0bFayAez0uibe6uX+2d7bI502ndMaF9251BdXyfGnILEu8Rg MzlwnV6kSqL6k+JTIPn9OJ4JUSRSeujmBSVzAV0AXevkmdMFQxTTIsz+l/k0Fm0bmfVO +ub8R1sfC/tYdH6vXmY+/TcGQfmZmmow+ejZ4KPmZqByJIPEiqhPRuvARH5oK7z5TTcM WNfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=P51U85xk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d93si22483193ybi.258.2020.07.07.00.17.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:17:43 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=P51U85xk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshrK-0003Xk-To for patch@linaro.org; Tue, 07 Jul 2020 03:17:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42592) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjC-0004pe-4b for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:18 -0400 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]:38876) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjA-00045k-2y for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:17 -0400 Received: by mail-wr1-x442.google.com with SMTP id z13so43975376wrw.5 for ; Tue, 07 Jul 2020 00:09:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=msa/RJrC+RYm3qouK0hknnnvEiN6J+9sLGN+6EdqCHk=; b=P51U85xkjm5rkv3t994vGXWYkv5dEUj9rrCJdNqKW1W0TIjFBIPjIWDG/nYSSz9qsM nPu0MTnjBF0RwfQmzAGCUFbOtJNnN/yH0Ww/x6/zgAmUJ7yHlnd/2jbZ8fqevI1rFg3a rCkSpcjQMklHWJIeWkN0EimpGInHtfk2gPEkiDferloFiSqBe5OmpMcXuXxiCPDZjh7A xdjq7p0UC9R7VHFfPfhBJKTLBRDv+V4W/bMhE5kVjVFYAZNNir32DN3q/26Jwou0nZRb RLaMPiG5NgIvp0lxgxaKVxzUFN9oDsR/HrEmHingsiGeMEORcxJelk+Vg+GCneU0o8K+ tV8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=msa/RJrC+RYm3qouK0hknnnvEiN6J+9sLGN+6EdqCHk=; b=Qvd3y3W1UQFFYgyWUOsCuICJI3Jl+5llFIXRdwzbJ6NHyCN64Yqp+fv9JekcZWt9tj KyVBnRM4tiSvZPECHAiIj4RAjSzmD+0iS36xub/Yy3+i1KuL42Ms1D5EAQPKrMO+FixZ U5DZrzpwC11omR8NRmvjjTArc9vgF2JZrnPvxyEJ/vlCwuLxWgAvS/LvyIkLKjM60W8x +vLN0UdsyRrMRQOx788PY2xqV83KdwfqEoe6euuX/mD9sKQaNIhlohhvmANGQAD/R2RA romllR8gyFTZJRXbqhVMfIC2r1+7uhue9s0jSVlXof28CG/NSeYvkSTLlu4Y4zixvYWh fOiw== X-Gm-Message-State: AOAM532/BaytjT+BVMTM6zFGHUBNTlGpSrfEdh2VyqFS1AvUjC23jKnu K/E9ncgZuL1Tg7+cRkkfbZ8JJx0u7As= X-Received: by 2002:a5d:4611:: with SMTP id t17mr49727837wrq.243.1594105754647; Tue, 07 Jul 2020 00:09:14 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id y77sm2348715wmd.36.2020.07.07.00.09.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:10 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id A82311FF9B; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 12/41] tests/vm: switch from optsparse to argparse Date: Tue, 7 Jul 2020 08:08:29 +0100 Message-Id: <20200707070858.6622-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::442; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , Robert Foley , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" optparse has been deprecated since version 3.2 and argparse is the blessed replacement. Take the opportunity to enhance our help output showing defaults when called. Signed-off-by: Alex Bennée Reviewed-by: Daniel P. Berrangé Reviewed-by: Robert Foley Message-Id: <20200701135652.1366-15-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/aarch64vm.py b/tests/vm/aarch64vm.py index bb04cb19c911..d70ab843b6b8 100644 --- a/tests/vm/aarch64vm.py +++ b/tests/vm/aarch64vm.py @@ -46,7 +46,7 @@ def get_config_defaults(vmcls, default_config): def aarch_get_config_defaults(vmcls): """Set the defaults for current version of QEMU.""" config = CURRENT_CONFIG - args, argv = basevm.parse_args(vmcls) + args = basevm.parse_args(vmcls) qemu_path = basevm.get_qemu_path(vmcls.arch, args.build_path) qemu_version = basevm.get_qemu_version(qemu_path) if qemu_version < QEMU_AARCH64_MIN_VERSION: diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 04d083409a5f..890bbc5549a6 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -23,7 +23,7 @@ from qemu.accel import kvm_available from qemu.machine import QEMUMachine import subprocess import hashlib -import optparse +import argparse import atexit import tempfile import shutil @@ -556,54 +556,57 @@ def parse_args(vmcls): else: return 1 - parser = optparse.OptionParser( - description="VM test utility. Exit codes: " - "0 = success, " - "1 = command line error, " - "2 = environment initialization failed, " - "3 = test command failed") - parser.add_option("--debug", "-D", action="store_true", - help="enable debug output") - parser.add_option("--image", "-i", default="%s.img" % vmcls.name, - help="image file name") - parser.add_option("--force", "-f", action="store_true", - help="force build image even if image exists") - parser.add_option("--jobs", type=int, default=get_default_jobs(), - help="number of virtual CPUs") - parser.add_option("--verbose", "-V", action="store_true", - help="Pass V=1 to builds within the guest") - parser.add_option("--build-image", "-b", action="store_true", - help="build image") - parser.add_option("--build-qemu", - help="build QEMU from source in guest") - parser.add_option("--build-target", - help="QEMU build target", default="check") - parser.add_option("--build-path", default=None, - help="Path of build directory, "\ - "for using build tree QEMU binary. ") - parser.add_option("--interactive", "-I", action="store_true", - help="Interactively run command") - parser.add_option("--snapshot", "-s", action="store_true", - help="run tests with a snapshot") - parser.add_option("--genisoimage", default="genisoimage", - help="iso imaging tool") - parser.add_option("--config", "-c", default=None, - help="Provide config yaml for configuration. "\ - "See config_example.yaml for example.") - parser.add_option("--efi-aarch64", - default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", - help="Path to efi image for aarch64 VMs.") - parser.add_option("--log-console", action="store_true", - help="Log console to file.") - parser.disable_interspersed_args() + parser = argparse.ArgumentParser( + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + description="Utility for provisioning VMs and running builds", + epilog="""Remaining arguments are passed to the command. + Exit codes: 0 = success, 1 = command line error, + 2 = environment initialization failed, + 3 = test command failed""") + parser.add_argument("--debug", "-D", action="store_true", + help="enable debug output") + parser.add_argument("--image", "-i", default="%s.img" % vmcls.name, + help="image file name") + parser.add_argument("--force", "-f", action="store_true", + help="force build image even if image exists") + parser.add_argument("--jobs", type=int, default=get_default_jobs(), + help="number of virtual CPUs") + parser.add_argument("--verbose", "-V", action="store_true", + help="Pass V=1 to builds within the guest") + parser.add_argument("--build-image", "-b", action="store_true", + help="build image") + parser.add_argument("--build-qemu", + help="build QEMU from source in guest") + parser.add_argument("--build-target", + help="QEMU build target", default="check") + parser.add_argument("--build-path", default=None, + help="Path of build directory, "\ + "for using build tree QEMU binary. ") + parser.add_argument("--interactive", "-I", action="store_true", + help="Interactively run command") + parser.add_argument("--snapshot", "-s", action="store_true", + help="run tests with a snapshot") + parser.add_argument("--genisoimage", default="genisoimage", + help="iso imaging tool") + parser.add_argument("--config", "-c", default=None, + help="Provide config yaml for configuration. "\ + "See config_example.yaml for example.") + parser.add_argument("--efi-aarch64", + default="/usr/share/qemu-efi-aarch64/QEMU_EFI.fd", + help="Path to efi image for aarch64 VMs.") + parser.add_argument("--log-console", action="store_true", + help="Log console to file.") + parser.add_argument("commands", nargs="*", help="""Remaining + commands after -- are passed to command inside the VM""") + return parser.parse_args() def main(vmcls, config=None): try: if config == None: config = {} - args, argv = parse_args(vmcls) - if not argv and not args.build_qemu and not args.build_image: + args = parse_args(vmcls) + if not args.commands and not args.build_qemu and not args.build_image: print("Nothing to do?") return 1 config = parse_config(config, args) @@ -619,12 +622,12 @@ def main(vmcls, config=None): if args.build_qemu: vm.add_source_dir(args.build_qemu) cmd = [vm.BUILD_SCRIPT.format( - configure_opts = " ".join(argv), + configure_opts = " ".join(args.commands), jobs=int(args.jobs), target=args.build_target, verbose = "V=1" if args.verbose else "")] else: - cmd = argv + cmd = args.commands img = args.image if args.snapshot: img += ",snapshot=on" From patchwork Tue Jul 7 07:08:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234928 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp657585ilg; Tue, 7 Jul 2020 00:19:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz1BzMMTr9TsxlMGeO2o4w9FKddywS83QjjKDvNzN057ksLMCW4NUFRlolXFqEBIEKSufSV X-Received: by 2002:a25:c445:: with SMTP id u66mr21162028ybf.277.1594106346934; Tue, 07 Jul 2020 00:19:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106346; cv=none; d=google.com; s=arc-20160816; b=oVvzWIApJ+B0LZESheNxQpz7fnXtcqEVMcqI1hdB5MSZrE8IV00EEJl9zs3CnolV5F VWT4YB2pH8qaXjSfFJFG6y6K9qklVxr8tUJHHu9rJS5jjfnSKI+vgdhwgZhDpfzOZsCM vbOq8I2lQp8Oa21fRvGeWJiHvIwh4JYta4No7JaY0GEXoIlk7lxwmvRS2G1LZhL4fwEk TT6kR6Lzik6ZsbCQetLkWHntmhSx3pU1VNZz5Y4nfmHsDN3o/M5vgsWqYOyTkHi9tLqQ HEMjay1okLJsKgL877oQ38BySIOWJ22fX4xWC9saPW/sIq+FKRDuLHMZ+59kdB7RNKrc eIlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+0XGMSv3TRBxSJ9urnyaVy5rxmKlhWwkj3bUWyiYszA=; b=Ts5WaGb0XaIBK/4dquwTMNRL5t4XsmqXWc8zju96lixH+AOvYwqzbaXRSNS6KD/D0Q u3VFI2NEL4I70AgQyvrE6q2+z5rvSTdmiK0p32daSlQvzZICx31nsegA2o4zdclc80cq 0cCgQk3GeFz0vKH4hephIOtS02QHsJ73919u6l+MWwJzBCvKrBToCD/sQ5chwrp+GXmz y9ir54KxnpkzO0HaKDpNO4R3xVxZ1GuU+d8IgRmGyOtJTqZznYNJZL3jheJnNSQIQ1X4 9VWDfBVkaQiDjcpgyCDc8L3oa9WZ7Eh3toUgzvcu0MQ0WXb49bmp8SDhKcyTCsDoXNvu /4TA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E5hT2WR9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g4si22420022ybh.219.2020.07.07.00.19.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:19:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E5hT2WR9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshsg-0005zJ-Bh for patch@linaro.org; Tue, 07 Jul 2020 03:19:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42674) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjG-0004yR-DN for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:22 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:33746) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjE-00046Y-P0 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:22 -0400 Received: by mail-wr1-x42c.google.com with SMTP id f18so35909508wrs.0 for ; Tue, 07 Jul 2020 00:09:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+0XGMSv3TRBxSJ9urnyaVy5rxmKlhWwkj3bUWyiYszA=; b=E5hT2WR9I5bKLy+mT2XtTMUCmIJcWX/FjdORNIjdTbCjHTzA2f+k/Yl8hD23prS/o5 zqkysJG8jE7vZUBrPMVRR4R0EgAoMkDI/JX4ILUlPTeoCDn8QzrET34ROJToZzJt2zxt EN2DDqFXwmG6DcW9GkuJxRyJPPo6Kj+ojHCJ+qJfQExw7qxPNcmN2jqMgPzMYEuDirSx ttZtsfe62qROvPoiv79KrVZtOOijzELEebEmzhBPvTYY4envK3KnAKdCF/t9sBVEKCMo bnqXTXIqGRRCMtD4ZTirEK3aPbnhdxD/0sNxZnoeD//nZ0HMv4iddj6synJyS6C/H3O2 yblw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+0XGMSv3TRBxSJ9urnyaVy5rxmKlhWwkj3bUWyiYszA=; b=ATE2VaL3q22ZHHRUQ2jdgicCfUPjJ6kWobQOT93l0G1OrxpBCquHo9uTU6fAY6sZAu c+nJIRTrE0znJfIlBPUBmk/2wf3wAcVjfJ3FXKXNpl1GN+pDypWDEGKznXBtR/35ed4E lkV/OnAUXN9xFiWoxIQR0MCe+GqlYxFnmkZ+zSbffo2q1m0QFC0eOhZlTwISMo4v/iBn wuYXCR3qWSdXdD46F+0RFIZIMHlhuq5hS36VyCkXPKuFOylZkrr4fyWtRr844GjGvWqa LhA7b4fFzoJwVhBN3kQtTb5XougDZfrmPxMrjIE30659+76HbGp94Db7L0WMAvZ1uNj8 kTDg== X-Gm-Message-State: AOAM532vMS5pTKnOo+KoySzHOnDyYlf4HIKllsRgP5cgN08p+tEDhyuz UDpSg+3Y9s/8NyVa6PXCW+TJ8BGhQnQ= X-Received: by 2002:adf:e6c8:: with SMTP id y8mr55612131wrm.40.1594105759396; Tue, 07 Jul 2020 00:09:19 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 33sm28737937wri.16.2020.07.07.00.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:12 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id BCB221FF9C; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 13/41] tests/vm: allow us to take advantage of MTTCG Date: Tue, 7 Jul 2020 08:08:30 +0100 Message-Id: <20200707070858.6622-14-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Robert Foley Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We currently limit TCG guests to -smp 1 but now we have added some aarch64 guests we can do better when running on x86_64 hardware. Raise the limit for TCG guests when it is safe to do so. Signed-off-by: Alex Bennée Reviewed-by: Robert Foley Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-16-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 890bbc5549a6..21f46d1957d9 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -551,8 +551,15 @@ def parse_config(config, args): def parse_args(vmcls): def get_default_jobs(): - if kvm_available(vmcls.arch): - return multiprocessing.cpu_count() // 2 + if multiprocessing.cpu_count() > 1: + if kvm_available(vmcls.arch): + return multiprocessing.cpu_count() // 2 + elif os.uname().machine == "x86_64" and \ + vmcls.arch in ["aarch64", "x86_64", "i386"]: + # MTTCG is available on these arches and we can allow + # more cores. but only up to a reasonable limit. User + # can always override these limits with --jobs. + return min(multiprocessing.cpu_count() // 2, 8) else: return 1 From patchwork Tue Jul 7 07:08:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234931 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp657895ilg; Tue, 7 Jul 2020 00:19:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxVfzDztpFzUEyraAsf/YYofpQkeHsL8nqKXF2VpNQhUIimT3pSOz7hLiZEFiwLTevhA2ie X-Received: by 2002:a25:3849:: with SMTP id f70mr85159283yba.135.1594106380698; Tue, 07 Jul 2020 00:19:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106380; cv=none; d=google.com; s=arc-20160816; b=bD9Mr734Mqsvy9i3CJ2jrar/b0jSoEWu7RgMwrWWon2LJkSh9y4Q2MPE+oEDAFrmyI vIy5APmgQnzZsSqpDuRd8ULSu4Grr6JL00fWQJd9cXLZ8OA49qc8w9tEr5xfSpGegOn0 VeuuyOMDaTmQ+EEyhIE4+84sEecS0Emh/JKnZDATcP+GpdtY6Y+yCpvu/ueYl+k7jB+F gbQjQFVd9/WGAApn3QlLQicVHMSMvShY3oFftnXZdxtSdo79AGGYrjY/IYhzI0nXQcgz k1g42IvzG0vG9GNOlTH4YLymgXg3RwKfv6oSCWGC4mo/d+fm32oQDvvvQIHami8n0U3B WmkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=YDO/x/b94zhIGLCoE0QO5NlUEy4S35DFUXq1lVsSN7c=; b=1LAcFwgFKL/UFjWdmPrWlYTm7r9GSfzGw1SQEG8BICsy8qk6DH7uBMfEsJhbRtFbEO f6xHY5QYuskfdHIeiJE/lLCRyIcfpxwu6RF4Vx7qbuWsSoPSykJEIBh/DiA0CgxqiVTi OUZY6cz+5LBn0iYuH840yG1E7UXEERNzO0gL0a2+Z51WYjU796RIRAjCoa7CptBtIxP8 zcXJkG9ibtlhgeW8nRoNDnc+oKmTi12qbmAXoLWbXf9VUqVaqsfZcWgqo9KGbGUgCrH2 C4RAeWAmJ1NVqyM5ugvK6u6oG9ZYSKIaKEvcxw1wHp6oR9abnUg1j+MNmJosDX5iyvQh TMug== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SRv6DbNz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i192si21239314yba.341.2020.07.07.00.19.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:19:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SRv6DbNz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshtE-0007LR-3x for patch@linaro.org; Tue, 07 Jul 2020 03:19:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjD-0004t6-Nt for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:19 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:44807) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjC-000464-4f for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:19 -0400 Received: by mail-wr1-x433.google.com with SMTP id b6so43951047wrs.11 for ; Tue, 07 Jul 2020 00:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YDO/x/b94zhIGLCoE0QO5NlUEy4S35DFUXq1lVsSN7c=; b=SRv6DbNzX+7Wwk1aRSvwZJ4D4Vy80rygPlGFJ+EXIu/mU/bLf6ZDyD0NuUjQdDURiG 0lhv2N5UwoVhbqK66W68BcxWGIVPc0SeaZH72ZhmlSrDOcGhs58eAYJ4khg9OK/Pm1Qn JenOKeGEY1vKxlmnoH5EC+zhG8rnzEgbJpt6cpxCvZZlAaJzJe4i66waKwpfAOnStRmV aO0f8Ig/tD510En6s5ZLok2avf2bo/PBcl+qGYWUlpSFXKWQe4K7HCNFDX9iTfUyt917 D3cmXoxLfTqSR/qWnuhm5IcwB8dCuKYQ3sSI0mxS/V3/aUI4wLZgSEM7wJGnw8crkDDM rbVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YDO/x/b94zhIGLCoE0QO5NlUEy4S35DFUXq1lVsSN7c=; b=nGOboFVjhzyQh7AST62jZpy2Gb9N5m6Y0sm5SABF6BDPhHULW7Utu/J/hp6W7B2+Rv jT+BJCYCG+vo7j0So7m7FhZgATBgu3hUKfc5BTswtFlY2JcF4SmfcLJv4CMCSV2BuCj9 WUWCFxDnbyUPiEYmBAnRd0+Vis4rLVtLeZfT0XaqB6M9/W+OhwZpohal/ZENeT5z/Qbs qD9DfFHoPKO+R2Bif0Kz01VutJJSuSXLyXMLegZH3oun+Yy1EVB6OjVTnEQqWLp3GI3x JHunnmR6/n17rArWpGBbRaLoTTLLk6NHbiiZBu3LlbGtW0aRbkrSKvuoZmC7YXa6CqsB xxDQ== X-Gm-Message-State: AOAM531MyfuR4ZWSb/n7Tc+5qsK+QAQdFkjVIT2pPeJVMHujcLBt3ikT o2WXsmUBQc1YKfy1BMnzLipy9w== X-Received: by 2002:adf:e7c2:: with SMTP id e2mr55959035wrn.179.1594105756726; Tue, 07 Jul 2020 00:09:16 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 2sm2128427wmo.44.2020.07.07.00.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:12 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id D16711FF9D; Tue, 7 Jul 2020 08:08:59 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 14/41] tests/docker: check for an parameters not empty string Date: Tue, 7 Jul 2020 08:08:31 +0100 Message-Id: <20200707070858.6622-15-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reported-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Suggested-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-17-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/docker/common.rc b/tests/docker/common.rc index 02cd67a8c5e8..ebc5b97ecf91 100755 --- a/tests/docker/common.rc +++ b/tests/docker/common.rc @@ -47,7 +47,7 @@ build_qemu() check_qemu() { # default to make check unless the caller specifies - if test -z "$@"; then + if [ $# = 0 ]; then INVOCATION="check" else INVOCATION="$@" From patchwork Tue Jul 7 07:08:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234912 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp652104ilg; Tue, 7 Jul 2020 00:10:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyw+PjN7X4IWL/WjB1Zva19Pt5Bbbx7GMCF8Yp8ra9P86V0KfJzWU7omOx6fuKJ25KPKLPs X-Received: by 2002:a25:2c05:: with SMTP id s5mr30737050ybs.512.1594105801293; Tue, 07 Jul 2020 00:10:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105801; cv=none; d=google.com; s=arc-20160816; b=xBVUhbw9sIdmUObkDriA5A6nG3F+OdJrHz5JRw8LHW0UVjiQg39HKqstuwHS5BZCux Cbl5hY5ne1syWiJUSJYRFcbpes4RWm0XZhoLUGvBGmr8F5gDCEwKGkFcrnxLG3OseC5S zR7t6asg0F9fFxMLhX79Xp/n4MpBlPftcdmEGQpIDrCI4wcv9nXIV6rQ/FU3nC4mFhpt NnquE4qorHTRr2oxB66ujVgVuIzb7xA1d387XhsS1j1u4Kwv7GiuDw0jjTuyh/zMEbTe 0V+YulTCjrDirdDmQ3BZM24bNyLSGSQuJrtEhlWRzxD+2cFc1sTsMgyQI4ZtcwIqogRA dgJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HL9K/pMjzPXAmNS2D71Rv36Uwa+37eOIvWsGiIveaTM=; b=bzHKJSDkLgm77jmyoEKT4XpQbdrVcMEsZVVGW/eZrYGczJ3sgLR31YKDmZG5CRPaUL onpB0/ZE54JSWHytWUzPSqrx0B8de92U3OEZ4sljY3K0mOVHhe0tdEw9K3EyXI557Emg mFaM0qaghw9qKUltpO95dU54XgAj7b8a9o8SHua5mxZN1uYQ5iWkqUz7COAgJvmfwtmM JejMENkdFY1qHDtzAm8OdS4CpB7TUtJsQyJETNYOkGAtjaH3Bb3EAR/mSgZETIhHn1Im Frp6uu6Q8a3vhULHeiDtLH2aSjgishixqt/KZkYgzdgFS9sfbyl5FBl1m0UfCLNJk/fx iNBw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="bn7F/n+R"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e79si21324707yba.493.2020.07.07.00.10.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:10:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="bn7F/n+R"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshjs-0004xn-NC for patch@linaro.org; Tue, 07 Jul 2020 03:10:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42634) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjE-0004uR-DS for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:20 -0400 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]:39616) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjB-000460-PJ for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:20 -0400 Received: by mail-wm1-x32e.google.com with SMTP id w3so32851995wmi.4 for ; Tue, 07 Jul 2020 00:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HL9K/pMjzPXAmNS2D71Rv36Uwa+37eOIvWsGiIveaTM=; b=bn7F/n+RC+3vjGtfga5E/wUrB7ZdFkKtxGBMsxeKTXCCxX/Suztg+CYQqYM9qv6H1O XbIS7zUO86MW5U9lIuLNm0R3HgzHunus4C/stzHvxcR6eB3aexCB6PQAIG9/N/xNiu7D XIlKshrmZfCmMmGoGbSmckVADf9gJLx1RwOIMMyxGVmaqvQozQro6ARcceNAizUGOS2b rSmiQTveAdACg9GiaZWNq2uGL/ycX/Sm06FQzg7e427RwLTHG48wS/jaENby/GhrVzTU Iour9hc+LpVe0kd3QPYcJHoZuV53jhmwy9W9p/QUGUYYqnm3DZAe7fmP2WGF8pm+4JQf mwgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HL9K/pMjzPXAmNS2D71Rv36Uwa+37eOIvWsGiIveaTM=; b=S9IlmeGrtMdHHDQW9T0en0MmSEtsC5Ei8dfNLYtnUvosqqqIDdNdaLZG7xKvkyVzNK ef4QOzQsKZPaOVZ5OFLxsZ5/a8XN6v8/p0AkjVJWCEXfDAUGvnnL9mv/LSH4mE0uQ05U 7kjVVZVL1bOc3KMvXEvzBEkdTKHa6XfMHvmrX2/mdtJBcUQ8ta2IuTDsumpZASQsOZ/o JSxfNaikrrBbEUsu5R6rJ9FtWv7QXN4UY6dAk0MKQbGXoj3zn14V0xCLOwmMXfvcX8Za MtwmxNMFj7haC9+Ch3nQYvwhSoCl3/3Io4PgFU9e1dy+P1vzHW+FogHPWEeHi2umGYQp jdVQ== X-Gm-Message-State: AOAM533XZmmtCbzDHtxKbbwr/FQoRS4sa0Me1e1NgpyRphsfcYallxCz F5zZf9k70RJVzmanfJ/8xl67Xw== X-Received: by 2002:a05:600c:218f:: with SMTP id e15mr2593000wme.187.1594105755793; Tue, 07 Jul 2020 00:09:15 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id s10sm2257659wme.31.2020.07.07.00.09.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:12 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 20C921FF9E; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 15/41] tests/docker: change tag naming scheme of our images Date: Tue, 7 Jul 2020 08:08:32 +0100 Message-Id: <20200707070858.6622-16-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32e; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We've been misusing the tag naming scheme for some time by overloading the post : section with the image type. Really it should be saved for the revision of that particular build. Move the details to the other side so we have: qemu/image-name with the implied :latest version added by the tooling. Suggested-by: Daniel P. Berrangé Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-18-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.shippable.yml b/.shippable.yml index 2cce7b568905..81905727d14c 100644 --- a/.shippable.yml +++ b/.shippable.yml @@ -29,8 +29,8 @@ build: pre_ci: - make docker-image-${IMAGE} V=1 pre_ci_boot: - image_name: qemu - image_tag: ${IMAGE} + image_name: qemu/${IMAGE} + image_tag: latest pull: false options: "-e HOME=/root" ci: diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 3e3617816ec6..e23b4af20ea6 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -50,12 +50,12 @@ docker-image: ${DOCKER_TARGETS} ifdef SKIP_DOCKER_BUILD docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command, \ - $(DOCKER_SCRIPT) check --quiet qemu:$* $<, \ + $(DOCKER_SCRIPT) check --quiet qemu/$* $<, \ "CHECK", "$*") else docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ - $(DOCKER_SCRIPT) build -t qemu:$* -f $< \ + $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ @@ -75,14 +75,14 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker DEB_ARCH=$(DEB_ARCH) \ DEB_TYPE=$(DEB_TYPE) \ $(if $(DEB_URL),DEB_URL=$(DEB_URL),) \ - $(DOCKER_SCRIPT) build qemu:debian-$* $< \ + $(DOCKER_SCRIPT) build qemu/debian-$* $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES)) \ $(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)), \ "BUILD","binfmt debian-$* (debootstrapped)"), \ $(call quiet-command, \ - $(DOCKER_SCRIPT) check --quiet qemu:debian-$* $< || \ + $(DOCKER_SCRIPT) check --quiet qemu/debian-$* $< || \ { echo "You will need to build $(EXECUTABLE)"; exit 1;},\ "CHECK", "debian-$* exists")) @@ -258,7 +258,7 @@ docker-run: docker-qemu-src docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/') docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/') docker-run-%: - @$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE) + @$(MAKE) docker-run TEST=$(CMD) IMAGE=qemu/$(IMAGE) docker-clean: $(call quiet-command, $(DOCKER_SCRIPT) clean) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index e630aae10897..cc6f76caa602 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -204,7 +204,7 @@ def _dockerfile_preprocess(df): for l in df.splitlines(): if len(l.strip()) == 0 or l.startswith("#"): continue - from_pref = "FROM qemu:" + from_pref = "FROM qemu/" if l.startswith(from_pref): # TODO: Alternatively we could replace this line with "FROM $ID" # where $ID is the image's hex id obtained with diff --git a/tests/docker/dockerfiles/debian-alpha-cross.docker b/tests/docker/dockerfiles/debian-alpha-cross.docker index 74bcabfdb12b..10fe30df0dd1 100644 --- a/tests/docker/dockerfiles/debian-alpha-cross.docker +++ b/tests/docker/dockerfiles/debian-alpha-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker b/tests/docker/dockerfiles/debian-amd64-cross.docker index 5d8904192519..870109ef6afe 100644 --- a/tests/docker/dockerfiles/debian-amd64-cross.docker +++ b/tests/docker/dockerfiles/debian-amd64-cross.docker @@ -4,7 +4,7 @@ # This docker target is used on non-x86_64 machines which need the # x86_64 cross compilers installed. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Alex Bennée # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker index 957f0bc2e79b..8fdfd6a6b046 100644 --- a/tests/docker/dockerfiles/debian-amd64.docker +++ b/tests/docker/dockerfiles/debian-amd64.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. Further # libraries which are not widely available are installed by hand. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé RUN apt update && \ diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker index 09ca0a1ba797..166e24df138c 100644 --- a/tests/docker/dockerfiles/debian-arm64-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-arm64-test-cross.docker b/tests/docker/dockerfiles/debian-arm64-test-cross.docker index a44e76d94211..53a9012beb8a 100644 --- a/tests/docker/dockerfiles/debian-arm64-test-cross.docker +++ b/tests/docker/dockerfiles/debian-arm64-test-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Bullseye base image. # -FROM qemu:debian11 +FROM qemu/debian11 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture arm64 diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker index e3794a61c959..b7b1a3585f8d 100644 --- a/tests/docker/dockerfiles/debian-armel-cross.docker +++ b/tests/docker/dockerfiles/debian-armel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé # Add the foreign architecture we want and install dependencies diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker index e163b8b95699..25d76188337f 100644 --- a/tests/docker/dockerfiles/debian-armhf-cross.docker +++ b/tests/docker/dockerfiles/debian-armhf-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture armhf diff --git a/tests/docker/dockerfiles/debian-hppa-cross.docker b/tests/docker/dockerfiles/debian-hppa-cross.docker index 5c68b2d3304a..3d6c65a3efcf 100644 --- a/tests/docker/dockerfiles/debian-hppa-cross.docker +++ b/tests/docker/dockerfiles/debian-hppa-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-m68k-cross.docker b/tests/docker/dockerfiles/debian-m68k-cross.docker index 25edc80e9a3d..fcb10e353475 100644 --- a/tests/docker/dockerfiles/debian-m68k-cross.docker +++ b/tests/docker/dockerfiles/debian-m68k-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker b/tests/docker/dockerfiles/debian-mips-cross.docker index 08a8e1c29cca..26c154014db2 100644 --- a/tests/docker/dockerfiles/debian-mips-cross.docker +++ b/tests/docker/dockerfiles/debian-mips-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-mips64-cross.docker b/tests/docker/dockerfiles/debian-mips64-cross.docker index 1a79505d696a..09c2ba584e51 100644 --- a/tests/docker/dockerfiles/debian-mips64-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker index 453b53ef72c8..c990b683b7a1 100644 --- a/tests/docker/dockerfiles/debian-mips64el-cross.docker +++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker @@ -4,7 +4,7 @@ # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker index 3b6e975c686d..0e5dd42d3c4b 100644 --- a/tests/docker/dockerfiles/debian-mipsel-cross.docker +++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-powerpc-cross.docker b/tests/docker/dockerfiles/debian-powerpc-cross.docker index 89dd4fbf8708..07e1789650b7 100644 --- a/tests/docker/dockerfiles/debian-powerpc-cross.docker +++ b/tests/docker/dockerfiles/debian-powerpc-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64-cross.docker b/tests/docker/dockerfiles/debian-ppc64-cross.docker index 4bf88ab02dd0..8efe68874ed6 100644 --- a/tests/docker/dockerfiles/debian-ppc64-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64-cross.docker @@ -2,7 +2,7 @@ # Docker ppc64 cross-compiler target # # This docker target builds on the debian Buster base image. -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker index cd386f01d93c..1146a06be625 100644 --- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker +++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the foreign architecture we want and install dependencies RUN dpkg --add-architecture ppc64el && \ diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker index 5e2d6ddb6004..2bbff19772a3 100644 --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/docker/dockerfiles/debian-s390x-cross.docker index 43fe59836f25..9f2ab51eb09a 100644 --- a/tests/docker/dockerfiles/debian-s390x-cross.docker +++ b/tests/docker/dockerfiles/debian-s390x-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian10 +FROM qemu/debian10 # Add the s390x architecture RUN dpkg --add-architecture s390x diff --git a/tests/docker/dockerfiles/debian-sh4-cross.docker b/tests/docker/dockerfiles/debian-sh4-cross.docker index 9d7663764e4d..fd3af8957596 100644 --- a/tests/docker/dockerfiles/debian-sh4-cross.docker +++ b/tests/docker/dockerfiles/debian-sh4-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-sparc64-cross.docker b/tests/docker/dockerfiles/debian-sparc64-cross.docker index 31fd34f120b2..f4bb9b561cfa 100644 --- a/tests/docker/dockerfiles/debian-sparc64-cross.docker +++ b/tests/docker/dockerfiles/debian-sparc64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Buster base image. # -FROM qemu:debian10 +FROM qemu/debian10 RUN apt update && \ DEBIAN_FRONTEND=noninteractive eatmydata \ diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/docker/dockerfiles/debian-tricore-cross.docker index 4a0f7706a39a..769d95c77bb2 100644 --- a/tests/docker/dockerfiles/debian-tricore-cross.docker +++ b/tests/docker/dockerfiles/debian-tricore-cross.docker @@ -7,7 +7,7 @@ # # SPDX-License-Identifier: GPL-2.0-or-later # -FROM qemu:debian9 +FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker index d16d6431bc8b..b045e821b910 100644 --- a/tests/docker/dockerfiles/debian-win32-cross.docker +++ b/tests/docker/dockerfiles/debian-win32-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker index b0bc960445b7..2fc9cfcbc663 100644 --- a/tests/docker/dockerfiles/debian-win64-cross.docker +++ b/tests/docker/dockerfiles/debian-win64-cross.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch MXE base image. # -FROM qemu:debian9-mxe +FROM qemu/debian9-mxe MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/docker/dockerfiles/debian9-mxe.docker b/tests/docker/dockerfiles/debian9-mxe.docker index 62ff1cecf2d2..ae2c222a6f08 100644 --- a/tests/docker/dockerfiles/debian9-mxe.docker +++ b/tests/docker/dockerfiles/debian9-mxe.docker @@ -3,7 +3,7 @@ # # This docker target builds on the debian Stretch base image. # -FROM qemu:debian9 +FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé diff --git a/tests/tcg/Makefile.qemu b/tests/tcg/Makefile.qemu index 9c23aeaa2a64..f8ad4c47be69 100644 --- a/tests/tcg/Makefile.qemu +++ b/tests/tcg/Makefile.qemu @@ -47,7 +47,7 @@ ifneq ($(DOCKER_IMAGE),) DOCKER_COMPILE_CMD="$(DOCKER_SCRIPT) cc \ --cc $(DOCKER_CROSS_CC_GUEST) \ - -i qemu:$(DOCKER_IMAGE) \ + -i qemu/$(DOCKER_IMAGE) \ -s $(SRC_PATH) -- " .PHONY: docker-build-guest-tests @@ -57,7 +57,7 @@ docker-build-guest-tests: docker-image-$(DOCKER_IMAGE) $(MAKE) -f $(TCG_MAKE) TARGET="$(TARGET)" CC=$(DOCKER_COMPILE_CMD) \ SRC_PATH="$(SRC_PATH)" BUILD_STATIC=y \ EXTRA_CFLAGS="$(CROSS_CC_GUEST_CFLAGS)"), \ - "BUILD","$(TARGET) guest-tests with docker qemu:$(DOCKER_IMAGE)") + "BUILD","$(TARGET) guest-tests with docker qemu/$(DOCKER_IMAGE)") GUEST_BUILD=docker-build-guest-tests From patchwork Tue Jul 7 07:08:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234941 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp660822ilg; Tue, 7 Jul 2020 00:24:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwyxRdPfhhZhaidl8f89Dd0Zk5hLgPSMKIbdEKrxevmrNwm3nhde+nW6lvjLHyGuniD+jq0 X-Received: by 2002:a25:bb82:: with SMTP id y2mr79745722ybg.392.1594106696451; Tue, 07 Jul 2020 00:24:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106696; cv=none; d=google.com; s=arc-20160816; b=v8Mt25tiW+G7xTg4SMSTBINf03Z/hAw32OLT7H5wv+5i4y9yzsCbfT5nbBclKkTndJ n6wuOUUiy7DBjA3B1B79G8/NXxcYtYRt8il4zRbakWzrgpjHV92lb0BV65AKge+1XXFg Untf7pflyhH61g6YGeJ2ceR48jh/45NsqoFmmb4qKK0bUhZr149r35n4lw6ePHu2ngav 2NpDBDx6PmPuxPQZtsZxCuPy7AVOStlbF4y8WGa//I2lz3qELzB2v7ZxioCNRE/4pYuB SayGZP1pD/5cB8PGePT4mVp7qV8W7hWRuQ+vDvBZA+K0OTs09X0wBPMgLeonBV9yyqu8 69aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=7lFr9zkkCuEtI7DcfBdHOASgR49SrvXGNoFz77BDRTE=; b=w8TgbrssQFv1QgEUWhIH1Ltg2HKgZjWVVXxCAeWMnEkuAaI7TaqAApITrKKI2t3FPc yWSZ48BELJDHgjty5GQG/d9zdqcgNg5XDGk64hsk4v1pKXVnbD4CKN62+/Zb/23SAopd TP4XJtLP7bVI4IQS9IeU1TpLx6au4XhvIok90+khmhS9YSX2Nl2mi14in0uFSoGQvwzV EPL3RR65vqc8yR2yL3lSB6q8iur66aw81v2cKC7LE8zm6UE7I237PQwRb/WUfQKxJtSN ZxrNy4VnjsU5IqIQ1lykXRkm6H7CQsCJZr5ky0IGZRZXoptfePPMotiFxskbpj1xrgKC FSsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Qsi2t6wc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y4si24983163ybk.397.2020.07.07.00.24.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:24:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Qsi2t6wc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52054 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshyJ-0002AB-Ub for patch@linaro.org; Tue, 07 Jul 2020 03:24:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43594) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoG-0006dm-Cy for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:32 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:36099) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoE-0004r4-U3 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:32 -0400 Received: by mail-wr1-x431.google.com with SMTP id k6so44006459wrn.3 for ; Tue, 07 Jul 2020 00:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7lFr9zkkCuEtI7DcfBdHOASgR49SrvXGNoFz77BDRTE=; b=Qsi2t6wcFmL8KwU9qnZG2amgnMNoeg1Fxx2ga+iobh91/t8lEntRT7/qun2RfJSE7H oVMHmipkoC3W+bAH3GTYNodG4aCnRhdd5xi7AHSFCeQwIQnGpgaKEX3uMdS7Dl2zOsb2 YzczX044I5OD9xIHehsB1Sr8h1dAdLtvhgUy8oBYRF8Y/42nuB/pyG3oJjhS/Wsrx7wz 9E9OrMnMWH0/DOg/Ei5P050t3+Gd8AB0iEsOpg4v34NkSfQTzereSWLRjBWrSl+UR9mE qhOp7PAI66KKKX3R4SoiIHiEU74bdPVuZ4EdFP1iyL7XpIKnn5RaEzpnxROGtw1xSZMk Yjxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7lFr9zkkCuEtI7DcfBdHOASgR49SrvXGNoFz77BDRTE=; b=YC9A++GkT/alshB1Gv42Te1vIl3b8PxXZyCCjIlycTB+rHi8LvxbquCyaoeX/315Jn 9o7AzVKlaUv9yyQc72IKERyzY+2MqgsSmdWBc782RQ/nytUYiQCwa9nQ/GkpS9vn5orx kjpXbtT39KCMRgfJDbAK1AUkT2tFy0u7d56DG3qrhGDGUJ0ND+2mylU0yUwrVAcwWDwW Y27P8OgDq8WaXnIh/6Wo+GCDzGLMnnN4I2iUSMGjX48VE2iqF2G5gsfH7mi5G8cdduuR aNKecPMNSk0lb2XwlqnGHlEsLNLIWAtAINLPr8wrqz5nhC+kR8Cu2djXRaiukz9X90wt k5lQ== X-Gm-Message-State: AOAM533RCjGfMyImfyhKYrubtdwAfaDUXeGA4wF7KAbSeAlNnQv7+mtF 5RI4WkTmLy5yYHhSLqdGGq050g== X-Received: by 2002:a5d:6ac1:: with SMTP id u1mr51208277wrw.123.1594106069537; Tue, 07 Jul 2020 00:14:29 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id r10sm26564128wrm.17.2020.07.07.00.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:27 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 34E991FF9F; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 16/41] .gitignore: un-ignore .gitlab-ci.d Date: Tue, 7 Jul 2020 08:08:33 +0100 Message-Id: <20200707070858.6622-17-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The sooner we deprecate in-tree builds the sooner this mess of regexes can be thrown away. Signed-off-by: Alex Bennée Reviewed-by: Thomas Huth Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-19-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitignore b/.gitignore index 90acb4347d46..2992d15931ac 100644 --- a/.gitignore +++ b/.gitignore @@ -93,6 +93,7 @@ *.tp *.vr *.d +!/.gitlab-ci.d !/scripts/qemu-guest-agent/fsfreeze-hook.d *.o .sdk From patchwork Tue Jul 7 07:08:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234930 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp657774ilg; Tue, 7 Jul 2020 00:19:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpEROnB3hk/tY4ziaW+a80TGWdZriPtDGINCpZbIxJeegjq1lZ694e/G3bXY62nmXXsvAc X-Received: by 2002:a25:37ca:: with SMTP id e193mr84707843yba.505.1594106367603; Tue, 07 Jul 2020 00:19:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106367; cv=none; d=google.com; s=arc-20160816; b=Yy3W0GBag8TpR86gH3HFsTPiURXUGjgOWsohGwBuTXHiBn4K7vpKm2raiYpXcMf1hC ASn4dHeHeY4Sip37IAQwD/nUyTWgeA1unotVDO7tDtpHnWw+n3SshoeJNrE+z3bH1Vds bpiXJSCs0Mx8I/WAxDASq0t9oqD9YQz9fLdnwZS8kDnXYj5S+XIZLSEvQ0VafqulkzLF m/2jL7jVfpSpykvGyFhe8LF9gTN+rbzHZlLxAwXI732H/ngRpfdeWuccnSJtQK8X7K3p 0Lyq1ca1enASVhbCr01RGisINp2OkDUoWYFvGZGF4hh/FLPm5nA8MlcfzRvS103aN2YF a4WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=QcDsGkNTaeJACoHN9Zi8eoA/LmcFjDZTuPYv9J/13u0=; b=dxz8h79uJeHeeG2In9j0OtWHs60F+G/Aq6E2+lMog/pSGW/fZte/2gT6fCrnAi8fWJ fa56ag0byv5ng7mlq42TYtpWOp6guaIGFWpREC36kknMFdR8YG4CsiENRvj07uNZoE/c 84jVh05ZmmplWmWdw+0ZKCek03hn7b6RT2dTWKrhPxrmwswo7lSE4c3APzfMFUWvTtHQ YmW5kR3368SwV0j2yGcREGbyuO/hKlUxH4vNxbFRZ67fvyTEbjUQzh9AAl6CXmmFPsaC ORQyMOJEVhUPAQFD8jOBpgOSBVk9VBe1Xe2uEQXF9m/LElCHwrPN6p5Cq8t9TJYhafTR MbaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=CsamUib+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d205si22822553ybh.435.2020.07.07.00.19.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:19:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=CsamUib+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsht1-0006p8-0w for patch@linaro.org; Tue, 07 Jul 2020 03:19:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoI-0006hl-46 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:34 -0400 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]:46644) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoG-0004rK-Fg for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:33 -0400 Received: by mail-wr1-x430.google.com with SMTP id r12so43920459wrj.13 for ; Tue, 07 Jul 2020 00:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QcDsGkNTaeJACoHN9Zi8eoA/LmcFjDZTuPYv9J/13u0=; b=CsamUib+kh/ukEfM4sBKp7U6RVCw/zg0f0XctfckiI3X9EwWY6vSgR9pyhzVh3E5Q/ OmknDa8+3dY9kRH+CkJ4BaRqWBI9fau7TDaTLFkHWTHR32+C5LjjZoBbbmrjcJdaodw/ RA38T2OjQRtN/OJV0CQD6MbF6GpBtmUnTImKX+It89XY+h2RT7dNOJGagzmorb0FqEvr QeVLSCPWhAvxMo0bGv2z67qB/k0f6WT2eUZF4v2tJUQ7QPCzw1Ri3i054qDvymZzNngn mlg+NeNsyX3as+yhNh50EYX4+qiiTEwICHrVZEUCCX1ArjSwdNZDLPrbj9HXQD1OKI3q EP5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QcDsGkNTaeJACoHN9Zi8eoA/LmcFjDZTuPYv9J/13u0=; b=kkfIBuWYeCxhsvAKWho7l41d9ofQM8GwteJRV6Ze2U1QT+OGHBSwQ993W8ZjmnBffC jvdt/q2jjvN9SDXFd0c2e7jqLC61KXRAl7FcWEJ9OQkQgf/MeUiFYdVGP3F1vk5fJ4ri 21fozWHBOO/ZZiFOJJOT6Xk1hB3tTAxTEstCRR2Px7iginWcEDxHYQPNULGOOiPE+RuQ f55QjfEHMjJNA/GN2gymoeNqI32HJLQI/Ol4/GmcoXxHhii2vaTtWpIhBpK7T9HNbqyg fEnepo/3dYDluPXCXGewqK5QkocLKax1f6JXzjKqucz1kgJWzszxgmgqQE3WE3ZAbZOl 8+Gg== X-Gm-Message-State: AOAM5317n8dPjMAKeLvZd0IS+/qcX/oep+IKkkDdL5PTlAFo9+54USFR mHD1Mv8EFyvV7KGGbI9C0o1oXQ== X-Received: by 2002:adf:fa8b:: with SMTP id h11mr52440866wrr.391.1594106071075; Tue, 07 Jul 2020 00:14:31 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id j15sm17963248wrx.69.2020.07.07.00.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:31 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 4B1491FFA5; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 17/41] gitlab-ci: Fix the change rules after moving the YML files Date: Tue, 7 Jul 2020 08:08:34 +0100 Message-Id: <20200707070858.6622-18-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::430; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x430.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Laszlo Ersek Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Thomas Huth The edk2.yml and opensbi.yml files have recently been moved/renamed, but the change has not been reflected in the rules in the YML files yet. Fixes: 922febe2af ("Move edk2 and opensbi YAML files to .gitlab-ci.d folder") Signed-off-by: Thomas Huth Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Reviewed-by: Laszlo Ersek Message-Id: <20200625151627.24986-1-thuth@redhat.com> Message-Id: <20200701135652.1366-20-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index 088ba4b43a33..a9990b71475f 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -2,7 +2,7 @@ docker-edk2: stage: build rules: # Only run this job when the Dockerfile is modified - changes: - - .gitlab-ci-edk2.yml + - .gitlab-ci.d/edk2.yml - .gitlab-ci.d/edk2/Dockerfile when: always image: docker:19.03.1 diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index dd051c01245d..6a1750784ac1 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -2,7 +2,7 @@ docker-opensbi: stage: build rules: # Only run this job when the Dockerfile is modified - changes: - - .gitlab-ci-opensbi.yml + - .gitlab-ci.d/opensbi.yml - .gitlab-ci.d/opensbi/Dockerfile when: always image: docker:19.03.1 From patchwork Tue Jul 7 07:08:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234921 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp655576ilg; Tue, 7 Jul 2020 00:15:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMKaSqIOKGNSwBG6JQqtzrvKK+BKuII8fIffOnP2b5cHgN5UU42/PPcxKD6WEqEZLFK/qv X-Received: by 2002:a25:2d43:: with SMTP id s3mr634330ybe.341.1594106138463; Tue, 07 Jul 2020 00:15:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106138; cv=none; d=google.com; s=arc-20160816; b=H0k+6gE3VQzquL9J3g4CbEbgsZFmDiy0GmI3PskOYWBsrqrHeSATPnCwNWtVOV1hsT +eVptD5DGNcFVmu7SHuaVeUmzWgY6kfIaQoJgU6Ejpjc1jPS24+mBm+mYMHVnNExgxrw 0u0dIBB9deI/wVbJH0OZnarsFre5zUuG26yMwLCkQUUYYmkQ8GpD0vMFqc3nTFedRi+2 ZBY2hEv7Eb8tiLAnmVO2Op4hR2Z9aDvR+CC1AIPUhKyUF32mnDDCcyEJTCZbttvd2ZEG wtiSnfAuv25ipBpZ+cgBrdlpfl+atE1+6AWgyXf9fTWU7CDrMuJBRM/+l8Zsz3AxItxK aeqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=eqzXTFvwpMkciPOZ24DlL9wg1TWlgnxheOGFm1m1Zko=; b=fa7ibXc7QbRksoqBvMDDbYE2drgvvKt91r0PUwSzGNA8b0rw0/3xy/xtM43hX3bZB5 /Sm0CMCDIyVxEM5ROm1lqGeNuEP5U33ZWBEwtY0hXleICARVdDiMqqY8kVgQKgCtU5hI 4RPMVShMHbVeNBUsz3d+QCQhOA/3cy+hAzdlyrTKqr0gUBefk9xEca5U4fBwrKN5Xtos LHjwxvFGaxkmQOBvfEdiPZatLY/RariMmYP4XwfCQKgiS2cbdERl02OWOAiAaaidAywS soXI5sK2itZpXZ3Ljxk6Wzyul8x8WFLQhyYLe1jU6eaKjxkmC5VNME+l2/s4dmG+xNlG qUdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="FW6u/FcT"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k128si21996537ybk.28.2020.07.07.00.15.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:15:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="FW6u/FcT"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60792 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshpJ-000719-U8 for patch@linaro.org; Tue, 07 Jul 2020 03:15:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43576) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoE-0006aF-VC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:31 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:39654) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoC-0004qf-Vo for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:30 -0400 Received: by mail-wm1-x32a.google.com with SMTP id w3so32866572wmi.4 for ; Tue, 07 Jul 2020 00:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eqzXTFvwpMkciPOZ24DlL9wg1TWlgnxheOGFm1m1Zko=; b=FW6u/FcTipVkdqSZzFZKAq9AnznjWuqqoIrSafqtjxsj1J0rHKVIa0E55mwky604zW Ehj0sDxmJkhJs3y5APTC6QRdPKd8EyolXmHZv/7eGLK0bikJRZOdd9qg8nvFgM+BjmuY 5NuS1SDF78imxtF3Ma+DpEydTsfJYLzOmY64iFhWyjjL2GLsAVs/w+t/KBpUTCq8lZmW cIeRFRUWzw3JgVHf95n/kyCIRfGpr0kZpO+B1FX6HJovMs+OyAI7aWEYCk7GlmmBnbEf YR8XdK3bR0YXY9CPZg/rx07Iy+ysAIu4imJbmKnPmnoDOCuZG5mAGaKPNR7pwE9hV2JK 3m/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eqzXTFvwpMkciPOZ24DlL9wg1TWlgnxheOGFm1m1Zko=; b=DYTVDrGXETJatmFyAMTDl7M+gLMLU2iu5Kzee3wsWWpwhQcJFPOVi7IJigoKDotuP+ 0+oogvbDzBtxPwsglRp0z8fWbycnmbCzOb4yRbE36KAsKJj42Oc89CBz+8HCBo1DRlmv 7XkQbjGGCvgcJt+33s+oCH9yKq+M3KUnTe/F7RD/xWVVP+jhcAfVqfQkVs+le0LZ1tuR ZsaXSrlisHfc7pZsBIMKNurf4YVxzTBov94toZNxM6k5c85Q8zFyXAqyiLWA4gcl64sU OSwiram/3MLgCm0Ht5nQshvOIJweC/x02X0nAMPFpM7iqp8xDbi5bv3rcxYGUYC3QxGe w/Dg== X-Gm-Message-State: AOAM532vpF0urCwXpGwPcQg2doLf/dAoFfp1nZKBeK9QdM1FLJEXP8ds L72hlSqGut0Eqjlt9bBcP8/Mpg== X-Received: by 2002:a1c:5986:: with SMTP id n128mr2529967wmb.112.1594106067389; Tue, 07 Jul 2020 00:14:27 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id j16sm27183653wrt.7.2020.07.07.00.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 6342A1FFA6; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 18/41] gitlab: introduce explicit "container" and "build" stages Date: Tue, 7 Jul 2020 08:08:35 +0100 Message-Id: <20200707070858.6622-19-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Laszlo Ersek , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé If no stage is listed, jobs get put in an implicit "test" stage. Some jobs which create container images to be used by later stages are currently listed as in a "build" stages. Signed-off-by: Daniel P. Berrangé Acked-by: Laszlo Ersek Signed-off-by: Alex Bennée Reviewed-by: Thomas Huth Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200622153318.751107-2-berrange@redhat.com> Message-Id: <20200701135652.1366-21-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index a9990b71475f..e1e045241661 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -1,5 +1,5 @@ docker-edk2: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci.d/edk2.yml @@ -24,6 +24,7 @@ docker-edk2: - docker push $IMAGE_TAG build-edk2: + stage: build rules: # Only run this job when ... - changes: # ... roms/edk2/ is modified (submodule updated) - roms/edk2/* diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index 6a1750784ac1..62088ec5ec1b 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -1,5 +1,5 @@ docker-opensbi: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci.d/opensbi.yml @@ -24,6 +24,7 @@ docker-opensbi: - docker push $IMAGE_TAG build-opensbi: + stage: build rules: # Only run this job when ... - changes: # ... roms/opensbi/ is modified (submodule updated) - roms/opensbi/* diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 349c77aa580e..9fdc752ea636 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,3 +1,7 @@ +stages: + - containers + - build + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' @@ -17,6 +21,7 @@ include: - JOBS=$(expr $(nproc) + 1) build-system1: + stage: build image: ubuntu:19.10 <<: *before_script_apt script: @@ -31,6 +36,7 @@ build-system1: - make -j"$JOBS" check build-system2: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -46,6 +52,7 @@ build-system2: - make -j"$JOBS" check build-disabled: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -62,6 +69,7 @@ build-disabled: - make -j"$JOBS" check-qtest SPEED=slow build-tcg-disabled: + stage: build image: centos:8 <<: *before_script_dnf script: @@ -82,6 +90,7 @@ build-tcg-disabled: 260 261 262 263 264 270 272 273 277 279 build-user: + stage: build <<: *before_script_apt script: - mkdir build @@ -92,6 +101,7 @@ build-user: - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user build-clang: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -106,6 +116,7 @@ build-clang: - make -j"$JOBS" check build-tci: + stage: build image: centos:8 <<: *before_script_dnf script: From patchwork Tue Jul 7 07:08:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234942 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp660836ilg; Tue, 7 Jul 2020 00:24:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwJfSO7NF2013jU8GAhkn7Mwk6YhSpIBK22O/LH+tfu7r9sUvZmdefnbfeazf3Mu5hCZ7MM X-Received: by 2002:a25:f204:: with SMTP id i4mr81282943ybe.486.1594106697253; Tue, 07 Jul 2020 00:24:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106697; cv=none; d=google.com; s=arc-20160816; b=dYw6JLkPgD456PDnFA9bQnzS8rVRMHfGh05/StfDNZBkDV4AY/PytSjpeshcTzCkmB D2C77OsYYybkASOIdtBNIr7lvRNImXN/oNa14QR9Qkk/prHH3cx63QoSw0RYEKPpK9su 5BvaYweqrwA/RO4fwlCzVvvogtFhAkzELY2bWoTUru1QIdHs1E/VStjXQHQU/9W61lUn AVRqImxMPKavO8brGcbPY8t3PIPS0qRerZFWpFyb2gFTQwjMTfmJF5UfJ4JOPKBEb2jR z254PfIC9wAj2EvLKXM3z5TFkZ/w3NO1Y/lkVDu7zMYl2yJNObtu0hFpd4l32On+qMdZ U6Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=eRHrZQr4xxHTl8oAYYO/0TefLzk7Y8ib9xGkS3UKuCE=; b=qphti62BJ8Dbp+19Cmtx71yOQsFuIeEbgPX69DROen4GOFhtWTZN48UFmeTkofMk4+ zenPReMUDm74xGGNYo8ej/+8ke49aSOCfrfeHN16Y6JXOEbE8/Dyg1N7bLf/BVKYiwfj aWfbUz5SUxJuu2/tNmzZQgoOiyzpyB5r1t3DComira4OIUuoZQl/OSoQ4m38IwbFz05A c3Ix/FI3tibtgMdzW7peldwrR2D1fN3qrwiqHUnBlGWJuau4Mw7jQWqFtaeuyWDzJUxJ I/WMtQVFzFuveW5qYWBXBOn9q70iSeM3gOzIdDJ6uh/cC/cir4p8c0/gXFhjhnyopjYL 2vnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=fU3LyOAS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i24si27597174ybj.388.2020.07.07.00.24.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:24:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=fU3LyOAS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshyK-0002At-MN for patch@linaro.org; Tue, 07 Jul 2020 03:24:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45938) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxu-0001bh-J4 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:30 -0400 Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]:53825) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxs-0006gg-GA for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:30 -0400 Received: by mail-wm1-x32a.google.com with SMTP id j18so42153424wmi.3 for ; Tue, 07 Jul 2020 00:24:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eRHrZQr4xxHTl8oAYYO/0TefLzk7Y8ib9xGkS3UKuCE=; b=fU3LyOASuwUlx1qff9Jk3ntYLstmYpSzeZoKLEC+bc3f6mWZ4G5eQSQVsVn71IzFJr 5Jt63hd0AqxnCqDh1+mG0BjrOgivo9fFAM4qxhLK0WrewF6JYiT9CStvNPX71bVJxQrs 9/sDmP1s4GazBkgjWa++2ruWhhJSgXwehDu5dQs7Bbmkr4/NW8UZBYNgW2mZMO+7ic8k MAAp1kPprAggmdxm4/IZqFLgt+L4KVYszqOtNolmdzEf8IN9NnaTEk6Vn6qnLzba4274 2uQB8KmignVrlTk4ufFfu+4PYUPI6KlxGr4J1MsoLbhyhAvEmS3og3/z5XZglQ3tveMu s+MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eRHrZQr4xxHTl8oAYYO/0TefLzk7Y8ib9xGkS3UKuCE=; b=Y0BXS8y0viB301v5HjtRT+vOxvB7Cd2ETKV1g/Oe/vOwEy5Vepf9caNRVGVsYdenGA yXNwgMypsG7CPUK4ebm/vi4hkCy3JcaF/Mz1DLkayeIG5AzjxJ8pU+xnLV4DDxZUWZ5C QCDdrccOIpAfO3gfYK354749al/iVRXvOZrOecnGhAy2W7tPAkEwI6GNnyZYJFpAVcCP 6j/++XenvpX7GLwxiSs7U6RZshfBOMCa8/VnsoF4aYazsKuu4Nxy9jFbymeJmYTvl3p2 VL3eBmdh3qVQHRlLu6kndgs5x0BpILq5EHbX2wkBTYneVChP+/zTi7StAW520hvhOtkZ vFPA== X-Gm-Message-State: AOAM5323K3mCrlNf6Wm7ZC3Zog69pJo2ViLDaPq2hpl7bYDg9EOc/Owr d+aHDkECopg5V5bBGoEd7rgU5Q== X-Received: by 2002:a05:600c:4109:: with SMTP id j9mr2622040wmi.157.1594106667008; Tue, 07 Jul 2020 00:24:27 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id b184sm2343882wmc.20.2020.07.07.00.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 7B75C1FFA9; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 19/41] gitlab: build all container images during CI Date: Tue, 7 Jul 2020 08:08:36 +0100 Message-Id: <20200707070858.6622-20-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Alex_Ben?= =?utf-8?q?n=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée Acked-by: Thomas Huth Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 000000000000..2afd01f1d179 --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,245 @@ +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea636..72d688875fd7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' .update_apt_template: &before_script_apt before_script: From patchwork Tue Jul 7 07:08:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234922 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp655778ilg; Tue, 7 Jul 2020 00:15:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+rqSxBDh0hWaj8wcf6f+Iuc8Gdy7gKinGGW7D6wd7KM3QzxFY/VHPG7A8OJ6X/bPny2Rd X-Received: by 2002:a25:b98e:: with SMTP id r14mr88068171ybg.20.1594106158606; Tue, 07 Jul 2020 00:15:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106158; cv=none; d=google.com; s=arc-20160816; b=kkZ1Q0x/vblTOgep+4R0wIrx7z1jlRv4GhtyQ1eV5SCRxTmJQz0TN8nJm4evRW58U9 Hb5sXJMBuo++9MeNguThGsrdVBGmNSkTUfHc4v5KrVbCsVhxjpQXmMnBom7tTjMnutZk IufluB95tThvqpCC3Cyd/6lcVoEB3re34FQixHeLadtojgleANrf7zENTPhl50zHfLR1 lZXpgp+aTU19YNbZ8Rc73xQwUdOl2ma3N7C/HDhR6D56r4r3lRodLXMVeA8WoUvMEfUv BOSd5I8JzpZtgwh21h7mrxyHKiDYhId5EX/OJk+cxfKFNX8wlPf0iKJrh+BvzahWIqmy l7pA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=u0LOJgObk9rTvD/1EAGQv5c56FpOTGglsdotVrrqgmQ=; b=ByBRvIKQ3XNiVLs/+IUpdI+Ji/GyUI7b2/25+jsaenG5jNE2t7/LipBEQAuhPujWno 4rlY/3RqtlBFYLTAYisO5z62kQwneMn8v/hzF+4SYkKIqY1Qs4QVHlEAxaDjnMG4W/74 /OViNB6irZ4gj5IuGpjRCkCfNGmfhqqzjEtOq4U5IovdYaZGubIIq7IzSKz3s6IWA/3h FE9d6VDAe2bPgHDeHQKAw1ScjckcVWd/3ZyL/Lg+z4cZ9Tq7iXA1Kvij1xPeYMSp9jmI fIGDgN+TqzdcLeuLlW1RshgGZALqrEm08myAJOb3c4vCn3Vifx1ldC1Inx2BRCDv0jAx shqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tUV51HJL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v16si2852489ybp.294.2020.07.07.00.15.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:15:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tUV51HJL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshpe-0007dI-03 for patch@linaro.org; Tue, 07 Jul 2020 03:15:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoD-0006W6-CC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:29 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]:54869) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoA-0004qN-Lt for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:29 -0400 Received: by mail-wm1-x335.google.com with SMTP id o8so42115721wmh.4 for ; Tue, 07 Jul 2020 00:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u0LOJgObk9rTvD/1EAGQv5c56FpOTGglsdotVrrqgmQ=; b=tUV51HJL4Kodh/5UcDw8Xwj3KZBmQFwMrM2iG6JZNn+hwhwYIg2x5DQOUevBpJw9ZH xmPsH3kk7Eq3TCHP7xF/h/O3n+k66GgrqC9cqF57i8txwzyjAV6dQNPE88xxaDQVyAjT orm1iQkZqChEYK+ulNYaEGZELHdIfL2ghG2LrA7P1viNKQgFntG0nCHLZU/atBK4j5V2 eMRimtGy4O4d5RUBbS2BZl5pmTrKuwMMgIoHC/Ff3meVdOsP6GglJN6ye+ATGC4kj2h3 NGtQljkF1yl06XS+BST8Z3a6YU1FN8yX/nDz8NwKPIiGeMpG7ObRgM0no3smlXv/KBB7 ETnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=u0LOJgObk9rTvD/1EAGQv5c56FpOTGglsdotVrrqgmQ=; b=kmsDJm2Q33edazVWEDRNRO2Ch4ecb6A01iug4VvgvO5ARx9IfP9jttp6XsLQQCPIoR kxY8sZq911KdLkMOgds6MVkM2AvR3TJlPn5Ac0a3YHJQ0tzGagC5OpeUAcUzC1a5EHqv SJMEhHBdY3ITY+XfIGpaZtGwUUS23eRkvvWUASPeAuCCQZb2S9F+4AEnu6q3IvueOLXw /ZUO+TC+JSmBz8ndsoNNJupGQWzVAzoEvdg1ktan/j6Mm14ARmygpipmktavW3BdVwOP yyKr5SA5SOqS8rBK6Tm3vnwbTPb+g6J6YyIOIp3CIaTH/s4UgO/ZqONhgWleGQzeHeD1 74FA== X-Gm-Message-State: AOAM533DEP6Tg2v7ogF7HqBABcz2zX6nDxNIc1oMqDbCgg5Sfk47So+6 bpf/E3hN8/XMqBQyHsWLtJavgQ== X-Received: by 2002:a7b:c185:: with SMTP id y5mr2779585wmi.85.1594106065131; Tue, 07 Jul 2020 00:14:25 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n17sm26187222wrs.2.2020.07.07.00.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 91D1B1FFAA; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 20/41] gitlab: convert jobs to use custom built containers Date: Tue, 7 Jul 2020 08:08:37 +0100 Message-Id: <20200707070858.6622-21-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x335.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Alex_Ben?= =?utf-8?q?n=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé Now that we're building standard container images from dockerfiles in tests/docker/dockerfiles, we can convert the build jobs to use them. The key benefit of this is that a contributor can now more easily replicate the CI environment on their local machine. The container images are cached too, so we are not spending time waiting for the apt-get/dnf package installs to complete. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-4-berrange@redhat.com> [AJB: tweak naming convention] Signed-off-by: Alex Bennée Acked-by: Thomas Huth Message-Id: <20200701135652.1366-23-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 72d688875fd7..a7abc55a5c6a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,133 +9,108 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' -.update_apt_template: &before_script_apt - before_script: - - apt-get update -qq - - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make - genisoimage - - JOBS=$(expr $(nproc) + 1) - -.update_dnf_template: &before_script_dnf - before_script: - - dnf update -y - - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-devel - make python3 perl-podlators perl-Test-Harness pixman-devel zlib-devel - - JOBS=$(expr $(nproc) + 1) +.native_build_job_template: &native_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + before_script: + - JOBS=$(expr $(nproc) + 1) + script: + - mkdir build + - cd build + - if test -n "$TARGETS"; + then + ../configure --enable-werror $CONFIGURE_ARGS --target-list="$TARGETS" ; + else + ../configure --enable-werror $CONFIGURE_ARGS ; + fi + - make -j"$JOBS" + - make -j"$JOBS" $MAKE_CHECK_ARGS build-system1: - stage: build - image: ubuntu:19.10 - <<: *before_script_apt - script: - - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev - libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev libvdeplug-dev - - mkdir build - - cd build - - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu - cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu - mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu + moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu + riscv64-softmmu sparc-softmmu + MAKE_CHECK_ARGS: check build-system2: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel - libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-devel - libzstd-devel - - mkdir build - - cd build - - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu - microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu - sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu + riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu + xtensa-softmmu nios2-softmmu or1k-softmmu + MAKE_CHECK_ARGS: check build-disabled: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-rdma --disable-slirp --disable-curl + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --disable-rdma --disable-slirp --disable-curl --disable-capstone --disable-live-block-migration --disable-glusterfs --disable-replication --disable-coroutine-pool --disable-smartcard --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user - --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user" - - make -j"$JOBS" - - make -j"$JOBS" check-qtest SPEED=slow + TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + MAKE_CHECK_ARGS: check-qtest SPEED=slow build-tcg-disabled: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel - - mkdir build - - cd build - - ../configure --cc=clang --enable-werror --disable-tcg --audio-drv-list="" - - make -j"$JOBS" - - make check-unit - - make check-qapi-schema - - cd tests/qemu-iotests/ - - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 + <<: *native_build_job_definition + variables: + IMAGE: centos8 + script: + - mkdir build + - cd build + - ../configure --disable-tcg --audio-drv-list="" + - make -j"$JOBS" + - make check-unit + - make check-qapi-schema + - cd tests/qemu-iotests/ + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163 170 171 183 184 192 194 197 208 215 221 222 226 227 236 253 277 - - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 122 + - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 122 124 132 139 142 144 145 151 152 155 157 165 194 196 197 200 202 208 209 215 216 218 222 227 234 246 247 248 250 254 255 257 258 260 261 262 263 264 270 272 273 277 279 build-user: - stage: build - <<: *before_script_apt - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-system --disable-guest-agent - --disable-capstone --disable-slirp --disable-fdt - - make -j"$JOBS" - - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + CONFIGURE_ARGS: --disable-system --disable-guest-agent + --disable-capstone --disable-slirp --disable-fdt + MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user build-clang: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-devel - libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-devel - - mkdir build - - cd build - - ../configure --cc=clang --cxx=clang++ --enable-werror - --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --cc=clang --cxx=clang++ + TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu + ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user + MAKE_CHECK_ARGS: check build-tci: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64" - - mkdir build - - cd build - - ../configure --enable-tcg-interpreter - --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" - - make -j"$JOBS" - - make run-tcg-tests-x86_64-softmmu - - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test - - for tg in $TARGETS ; do - export QTEST_QEMU_BINARY="${tg}-softmmu/qemu-system-${tg}" ; - ./tests/qtest/boot-serial-test || exit 1 ; - ./tests/qtest/cdrom-test || exit 1 ; - done - - QTEST_QEMU_BINARY="x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/pxe-test - - QTEST_QEMU_BINARY="s390x-softmmu/qemu-system-s390x" - ./tests/qtest/pxe-test -m slow + <<: *native_build_job_definition + variables: + IMAGE: fedora + script: + - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64" + - mkdir build + - cd build + - ../configure --enable-tcg-interpreter + --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" + - make -j"$JOBS" + - make run-tcg-tests-x86_64-softmmu + - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test + - for tg in $TARGETS ; do + export QTEST_QEMU_BINARY="${tg}-softmmu/qemu-system-${tg}" ; + ./tests/qtest/boot-serial-test || exit 1 ; + ./tests/qtest/cdrom-test || exit 1 ; + done + - QTEST_QEMU_BINARY="x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/pxe-test + - QTEST_QEMU_BINARY="s390x-softmmu/qemu-system-s390x" ./tests/qtest/pxe-test -m slow From patchwork Tue Jul 7 07:08:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234944 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp661620ilg; Tue, 7 Jul 2020 00:26:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyQLbbCd9hrsIzYR8hbwnXpdVpkvMjcZ4lyjz3uvSHDMVIxY83oxMa/JNrsUeWH8T/3xhU1 X-Received: by 2002:a25:d1cf:: with SMTP id i198mr40816929ybg.23.1594106786436; Tue, 07 Jul 2020 00:26:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106786; cv=none; d=google.com; s=arc-20160816; b=n1sNlfXFxMGrrQbdYZRPvGgaSHBYYk8UCTa3yEMQyDnyn+woxrgSBeXzKIzoVIqhHu QXC9Wm4cDIV0QoFcRQv8pjV68RgBEfezasXLKkfRMOhyh0Hb1+6B0rv6XsYiVbLpemjw OubKgssNZwvua8K+ORYALHqUX2y8keJMNJzqV9jwIuExIsvPp65kiBJbG4IDCahzqXAc 2QUz8IRoLJa3XI8PxGtkxGrQwruXEikyZ1RgFIk6ZD00Lpks51ncSLfrNayc8UT0IPws s5whZbcDB34El17fQSLD0BlPowYehtv7opivumCtXFpfuFObAuMSNWtF/L/rpCQ3U9v1 2XVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=YO+flFf+2yJ5JAaqokF7PDcUOrzGxTLuTO5KQlNjjuo=; b=qG79k2WvruFol3xtLreXRq79al8ktyouCsLTXPmTO3iDqqvoMdW5/Wr1FAkvdtv5Tj u3dsgU9svWBS23zr4YG9tlSduHokQuD53Cjq64jbpereDGA3z0ufCvuGOAPIPgTCfbGo 0wzcVvPng+/Y44mCfqNkSTonosiGkBJfY1r2Uvnovp8Z39VLyHrvXu1e1o7xGQTrXON5 2vTNAvotuqh3zLo3Rz3CZh20fSrDW/FBVvRCsthqUH49QSgyLXMtUYxLWqglYakCcTj+ vZVP/rVjC1IR/n+vG7kc9tV9FuorD3bk7J6U4wWNYVuKZF619++RYQ8Fh2ZXjGzA9IKz tmnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=luhIICYQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p202si21435237ybc.482.2020.07.07.00.26.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:26:26 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=luhIICYQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshzl-0004OL-Tc for patch@linaro.org; Tue, 07 Jul 2020 03:26:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45882) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxr-0001Tr-D0 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:27 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:42490) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxo-0006fx-Tv for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:27 -0400 Received: by mail-wr1-x42f.google.com with SMTP id o11so44048378wrv.9 for ; Tue, 07 Jul 2020 00:24:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YO+flFf+2yJ5JAaqokF7PDcUOrzGxTLuTO5KQlNjjuo=; b=luhIICYQJVM69XdfZZPLZIIyZcWs7GLRG9Y5ZAsStsAwji0nvEajCF1b65J0s4Jdu1 sV4kW/4uDDRP4NUCd7VZQaGqwTG5imA5HDnkQsnot9aLkMLT2WRPMv7rS/iYmRHPnQhJ oCPeFypEIAqE8l1ojsux9Tw5XDGrp8tuJhrsSvndQJz1WvW7LRSDanVPBdQ9+ij5gKCT jTFykhjZdXHlx0UdXXS60z5mtSEBjcr9CEDl1+Fx2C80ra8XUqo6DHfaR3AY3ug/eag+ OHBexttdjoo4x3+I6uAx9dyWEWTGNrawOAH0PeTQNbUbr7voGu8T/8EJRO5AaMZL/n/x h/Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YO+flFf+2yJ5JAaqokF7PDcUOrzGxTLuTO5KQlNjjuo=; b=sgKO0hnf04kRKQsD2pMRrw+TCCuT/xxeN0Flz028kNjXGE/YkiIdf3oOIwrPhMAje9 f2GHmNdPQs82u2Azwuxy3l8ZUKd3F65oEk41cFpOchOY5XG+zAclXGkSCN2q6Cx8H6/B jGW/TDlYYDpfL8jiszGEUjT5HnswOt9iDWv1YdXwUEOpG33Ku62lq7t64fYRowXIS7xD hxteGBSif4EWQinr7r93zno8DDKTGY2Ba0gPxmMK6xm1ukJOfebc7UXC1hd8MDCrNScO 0xwL2GxDIqrAKGmqh3+8ixnpsj6IqURqorvpzp+LgIthWbS3kh4pEivV5UZ0b5c/LKdi OzeA== X-Gm-Message-State: AOAM531uJJfuLLoqXr6yNYCkq1kTwcURzHKk5CMY8iaoYFgCorplvVy4 mKeb8xCL6l0krp/dvuaA8kQQaw== X-Received: by 2002:a5d:688d:: with SMTP id h13mr17459418wru.303.1594106663631; Tue, 07 Jul 2020 00:24:23 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 22sm2386322wmb.11.2020.07.07.00.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:21 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id A811A1FF87; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 21/41] gitlab: build containers with buildkit and metadata Date: Tue, 7 Jul 2020 08:08:38 +0100 Message-Id: <20200707070858.6622-22-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" According to the documentation to be able to use --cache-from for remote registries you need to enable both buildkit and inline the metadata. We want to do this to support pulling from gitlab when users build their local docker images. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-24-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 2afd01f1d179..f353359fd8ba 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -11,7 +11,9 @@ script: - docker pull "$TAG" || docker pull "$COMMON_TAG" || true - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker - - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - DOCKER_BUILDKIT=1 docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" + --build-arg BUILDKIT_INLINE_CACHE=1 + -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles - docker push "$TAG" after_script: - docker logout From patchwork Tue Jul 7 07:08:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234946 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp662455ilg; Tue, 7 Jul 2020 00:28:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyxkoPaaJPlF7O8KWtWPNG6/FmbY56QRfhczLWdPKVin2nEDrnVJhmVFKyfU2Z2Y2IYbIxE X-Received: by 2002:a25:848f:: with SMTP id v15mr89117440ybk.473.1594106882538; Tue, 07 Jul 2020 00:28:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106882; cv=none; d=google.com; s=arc-20160816; b=MhJIlaoMZnrXiCm8AJ5qTi0aQrYyK9Hv0n0yGmj2ZrsSvpDO79FT/hs0h1agJ13fHA ojEQYp/5WAVNH3XKIZGzUx3MBmGDPhkBKUxhDDrUEBs0jk/1ZPnVsLtuse4jsQT3HF67 /I4bNqW0eF3whnhMSciSofPXABFRx3I1F1Nu6HtD7r8Pgbp2B5ojIosXrDTiRb5sN5et qeHuYvg5kLeQxu6PWulRR/f25cNSm0UjJO/VAIb6QaN5mli8CrMahcltFthkcKNiUj+P s9c0u0BBWkwQk/sE8b83slKpm9+fr0gZpiBK8i4wslgfMKMRpEKgSyuaPMzMVIA/r9ql wsXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=i8mLxxoVe+bccZ+lmxE5XmxcSbQk6bfwvCsechC4WCI=; b=aOGGPwj5Z9YcW1IMPMGLsZOpPihNDEXWNwaya9jovq+7414bxXfUXYzQlvgaaq/epO ZAgBIPmPW0CsVHeCOPvgMreUEF4TWC52uRQS743HEjLmJyWU9RqAE2xMmkVqQFJbyh8P jlUcbsn5XYuaY5Q47hlHfUPvDAbtILpeJdANbQX9ndIMBR6FotfzuapBtb2BmgDiCcfc svvVQLWTBCxJyHKshm1Ji5PHYby2aan6V4D85oE4gRm30uJBtywyUMKZwJ/6BAiFh4kn a6S+tvTkl38d5q9Q3NEULwm9k2RjBi7K2IEDWz11MI8aeGy9GZViUvYCnq+HFMat+GLP ZRbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SsCfuK56; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h3si14250270ybh.33.2020.07.07.00.28.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:28:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SsCfuK56; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34242 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi1K-0006Zn-2A for patch@linaro.org; Tue, 07 Jul 2020 03:28:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxs-0001VM-5X for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:28 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]:42492) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxq-0006g7-2W for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:27 -0400 Received: by mail-wr1-x431.google.com with SMTP id o11so44048423wrv.9 for ; Tue, 07 Jul 2020 00:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i8mLxxoVe+bccZ+lmxE5XmxcSbQk6bfwvCsechC4WCI=; b=SsCfuK56XDpHfsz/Ud4PuNLGmwoPpo5rs4KP1O07QUtotxAHkDantf/MV8qVPYvfF+ ZOFvRpUnLbfNrQHq0baGtjVMCSd0PaPiPBzMMO6Jlw3kvAz7AX1iCFhC5A5TMonf/ZoL Pby1/DKTaqb5FN80PQlqF32zSyCfZs0ezh8LApJR385syce4lD2EsyLR9OEanPWS3Wh3 uVmOdON71H9fdaBdfZeI8n5eRTI5Fw5Zy/ZviUEhjb1SyrsA8w+4M3CGF4XPZrhJeAdX X1OnmNpdHAqTjA1xvFQlqutdHNqrxwmofxEmxKodrDilMek0uw7RLjgebWGKxb9k8jrj 53Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i8mLxxoVe+bccZ+lmxE5XmxcSbQk6bfwvCsechC4WCI=; b=K0IDLEzxNO1brBExDwK4tOA7E3Lxerq8SmYSom7U3ZMnM42E0oJzxAy7d0RiF6yIzL yBCcVpN/uDNIU7VtxX3jr7S0O+H4AV+0G0jXXOokTqrQQuY3/Qrkxk3Lub0ZzJxipura g15uN9TL+xIZI2hTh6EZ70amoDLVTxULdeMbQeUSDohzqsErb3zdUHepHa5QV0jglDuT 9ZrLm3UlQGKIL7pWIkMKH7oN2HH3LONtK81Mzj6E1CCFk+5MabvH4oMXxusNdnECqS/s xX2Q6JwFvy/Df1HFRC/A/wtRwqJy2wcWRpxypN/vle+X7IYGKO7j/ZW0S7wmuAXb/WkS 9ZEQ== X-Gm-Message-State: AOAM53063aYUqCgyE/oFweKSZkVZv09bAIZV8fg/w77c/b7ojqBE1vyQ fD7hii01ZGc4whAXkyMfpD4UPA== X-Received: by 2002:a5d:5706:: with SMTP id a6mr16959655wrv.328.1594106664628; Tue, 07 Jul 2020 00:24:24 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 5sm2281208wmk.9.2020.07.07.00.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:21 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C16F01FFAB; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 22/41] tests/docker: add --registry support to tooling Date: Tue, 7 Jul 2020 08:08:39 +0100 Message-Id: <20200707070858.6622-23-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x431.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This allows us to point the tools towards a registry from which they can grab pre-built layers instead of doing everything from scratch each time. To enable this we need to be using the DOCKER_BUILDKIT engine. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-25-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index e23b4af20ea6..a26177abc258 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -13,6 +13,7 @@ DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.doc DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES)) # Use a global constant ccache directory to speed up repetitive builds DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache +DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),registry.gitlab.com/qemu-project/qemu) DOCKER_TESTS := $(notdir $(shell \ find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f)) @@ -56,7 +57,9 @@ else docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command,\ $(DOCKER_SCRIPT) build -t qemu/$* -f $< \ - $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ + $(if $V,,--quiet) \ + $(if $(NOCACHE),--no-cache, \ + $(if $(DOCKER_REGISTRY),--registry $(DOCKER_REGISTRY))) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))\ $(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\ @@ -213,6 +216,7 @@ endif @echo ' Include extra files in image.' @echo ' ENGINE=auto/docker/podman' @echo ' Specify which container engine to run.' + @echo ' REGISTRY=url Cache builds from registry (default:$(DOCKER_REGISTRY))' # This rule if for directly running against an arbitrary docker target. # It is called by the expanded docker targets (e.g. make diff --git a/tests/docker/docker.py b/tests/docker/docker.py index cc6f76caa602..9684f07bdebe 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -221,6 +221,13 @@ class Docker(object): """ Running Docker commands """ def __init__(self): self._command = _guess_engine_command() + + if "docker" in self._command and "TRAVIS" not in os.environ: + os.environ["DOCKER_BUILDKIT"] = "1" + self._buildkit = True + else: + self._buildkit = False + self._instance = None atexit.register(self._kill_instances) signal.signal(signal.SIGTERM, self._kill_instances) @@ -289,10 +296,24 @@ class Docker(object): return labels.get("com.qemu.dockerfile-checksum", "") def build_image(self, tag, docker_dir, dockerfile, - quiet=True, user=False, argv=None, extra_files_cksum=[]): + quiet=True, user=False, argv=None, registry=None, + extra_files_cksum=[]): if argv is None: argv = [] + # pre-calculate the docker checksum before any + # substitutions we make for caching + checksum = _text_checksum(_dockerfile_preprocess(dockerfile)) + + if registry is not None: + dockerfile = dockerfile.replace("FROM qemu/", + "FROM %s/qemu/" % + (registry)) + # see if we can fetch a cache copy, may fail... + pull_args = ["pull", "%s/%s" % (registry, tag)] + self._do(pull_args, quiet=quiet) + + tmp_df = tempfile.NamedTemporaryFile(mode="w+t", encoding='utf-8', dir=docker_dir, suffix=".docker") @@ -306,15 +327,23 @@ class Docker(object): (uname, uid, uname)) tmp_df.write("\n") - tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" % - _text_checksum(_dockerfile_preprocess(dockerfile))) + tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" % (checksum)) for f, c in extra_files_cksum: tmp_df.write("LABEL com.qemu.%s-checksum=%s" % (f, c)) tmp_df.flush() - self._do_check(["build", "-t", tag, "-f", tmp_df.name] + argv + - [docker_dir], + build_args = ["build", "-t", tag, "-f", tmp_df.name] + if self._buildkit: + build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"] + + if registry is not None: + cache = "%s/%s" % (registry, tag) + build_args += ["--cache-from", cache] + build_args += argv + build_args += [docker_dir] + + self._do_check(build_args, quiet=quiet) def update_image(self, tag, tarball, quiet=True): @@ -403,6 +432,8 @@ class BuildCommand(SubCommand): parser.add_argument("--add-current-user", "-u", dest="user", action="store_true", help="Add the current user to image's passwd") + parser.add_argument("--registry", "-r", + help="cache from docker registry") parser.add_argument("-t", dest="tag", help="Image Tag") parser.add_argument("-f", dest="dockerfile", @@ -458,7 +489,8 @@ class BuildCommand(SubCommand): for k, v in os.environ.items() if k.lower() in FILTERED_ENV_NAMES] dkr.build_image(tag, docker_dir, dockerfile, - quiet=args.quiet, user=args.user, argv=argv, + quiet=args.quiet, user=args.user, + argv=argv, registry=args.registry, extra_files_cksum=cksum) rmtree(docker_dir) From patchwork Tue Jul 7 07:08:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234947 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp662695ilg; Tue, 7 Jul 2020 00:28:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7uD+NpQrvkwHbTzSyw+lIbnidrYdNayfdjlN6ZVQyNUclDLi0NlMh6RZ0lc4hoGX4swcP X-Received: by 2002:a25:b511:: with SMTP id p17mr90149231ybj.86.1594106911002; Tue, 07 Jul 2020 00:28:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106910; cv=none; d=google.com; s=arc-20160816; b=kPKWHSFuR1ZlvCIVSiAefAJ/mU/YyBE9TzDlyocTwOk8CpvhCxq49xYdIggdXHxUNm dZAa4JXjhpqs9xP15/3mCEpCByXospWAt5QcK6lXNsZPC5zbFrJXFFgvnlAaF7q0z8ft roJ4QNJjTdCHlxl8jwmyc9G0dtV35awBQxHmGMm7SvbWD/d2aSzlB1sEBYBeAscyNL2i QlsIP5qOAdK6PB1nRWeuTseLXc99Tip7lqlHLDVKLbznWTMZTUkti4ivPyHBvIdRbVda VvdA0Xxm9uYC7dSXprEAm4McLmoW4gY6tzSCMopkXUT8pP0UXJJmRsHWYmq47mrobWlk L+3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BhAPRbyJ6wjGUojYdyED7mQgTxQ6WKTzi+98H1OGLvA=; b=B7Z0/pJjkklMd5Cq/f34Thlz/V++h6KPcFB84WhnCMnBVQGxBWJy+QF9uW0wujKZnv umACVE4eSj5RP+VPWo4Q+32Yud/+rxsqEeQPmzNmcLE3qaWgwnvN3FuaEefMz1R2/JbF kGzaFdWqaHDxoCc1bc1WRSg0cdr11klzsIuKpPUOzlHtDt1w7rJAot3EoBWxoYiUBMru kd3tan0SSpUo+fni1nwCubx5mj0Bmy0Gin/DGLdOze6hIyMLvhWcbTT7aPZ4FgwQXcoM 92Fwdemlt1QMQtPQGi7DvXVNT1ys9goB5wdvHcr9voApgsty/p6g+Xyezu7/ZbqgcGrC Ym8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tmvi6Rh6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b125si13651185yba.89.2020.07.07.00.28.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:28:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=tmvi6Rh6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi1m-0006v7-G1 for patch@linaro.org; Tue, 07 Jul 2020 03:28:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoQ-00073K-Py for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:42 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:40374) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoN-0004sZ-Nv for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:42 -0400 Received: by mail-wr1-x42f.google.com with SMTP id f2so16004074wrp.7 for ; Tue, 07 Jul 2020 00:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BhAPRbyJ6wjGUojYdyED7mQgTxQ6WKTzi+98H1OGLvA=; b=tmvi6Rh60yUjJoJdLRzyAuDs3Uetu+eqE6la3Po2XNfZOV2GmBOtY5yjLNzLOYFKJZ 3F3nmlxXU8WmbfjHSLXi4q6o8fk5zz1FnSdhZXyaa+hBiRH9gv0NwldCupYmsUYPMboS z4jVdoOrgR7breirfvpvYZf9BhFMkpw1lVB0qEDTCMZrGShE9Qf9h4aLGCgLQGyuM8i2 d8vOMEfAHR8aVJOlz1GaIay64MpiJdSwaDymepqnZrjGEkBWoYoMCDAve8CDHxzDbtFT WYgwfl31Hg8/iJfTBQW7p3Ct8DdR/H21YcV4QHXjkcgh7eb9Y8ByeWXr59qCyDpjB4C4 VuqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BhAPRbyJ6wjGUojYdyED7mQgTxQ6WKTzi+98H1OGLvA=; b=pcq8Xhu6dtuCZkfWJNLP+ymLmMNSOjpZVPUHhO3NGK5orrM3y23vdT+RBMhIKWLnGO 86ZhjSJ7JR9OkojSwCjPYZgXWXgrjxSALx8G4RDaBHrRodZURUwJ3yLpqGULnqI7Wi9F OOW8Grldn1a2nNAPibFbeNg672AmZpHsSe0CS/6xJf2tjb+rnXf3BlyeGKBTNjfmoFTC 4eW/6lHd7Sh9tHslPSiekE4GB5XbGY/kSloy3B2VjC9G4dR4pk3YI8b7PVvyHnj67wBJ g9tOsZqFCgg49xVNAr1b4/rzdycrC0iL09vob266tYWjiPapjgNyH50wm/hepzGFgivm lGiQ== X-Gm-Message-State: AOAM5331/Pn407Ivv91He96uk/qcQohi7vWMlZQbNAKQ4zqc/tt1vG8F rfWgx+nZxBdaLhjoSOXSuOrz/A== X-Received: by 2002:a5d:630c:: with SMTP id i12mr57737967wru.158.1594106078157; Tue, 07 Jul 2020 00:14:38 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id m16sm23078090wro.0.2020.07.07.00.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:28 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id DABEA1FFAC; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 23/41] tests/docker: add packages needed for check-acceptance Date: Tue, 7 Jul 2020 08:08:40 +0100 Message-Id: <20200707070858.6622-24-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We need additional python packages to run check-acceptance. Add them to the docker images we will be using later. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-Id: <20200701135652.1366-26-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index 798ddd2c3e0b..70b6186bd3ea 100644 --- a/tests/docker/dockerfiles/fedora.docker +++ b/tests/docker/dockerfiles/fedora.docker @@ -80,7 +80,12 @@ ENV PACKAGES \ pixman-devel \ python3 \ python3-PyYAML \ + python3-numpy \ + python3-opencv \ + python3-pillow \ + python3-pip \ python3-sphinx \ + python3-virtualenv \ rdma-core-devel \ SDL2-devel \ snappy-devel \ @@ -89,6 +94,8 @@ ENV PACKAGES \ systemd-devel \ systemtap-sdt-devel \ tar \ + tesseract \ + tesseract-langpack-eng \ texinfo \ usbredir-devel \ virglrenderer-devel \ diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker index 6050ce7e8a80..f7aac840bf86 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -46,9 +46,17 @@ ENV PACKAGES flex bison \ libxen-dev \ libzstd-dev \ make \ - python3-yaml \ + python3-numpy \ + python3-opencv \ + python3-pil \ + python3-pip \ python3-sphinx \ + python3-venv \ + python3-yaml \ + rpm2cpio \ sparse \ + tesseract-ocr \ + tesseract-ocr-eng \ texinfo \ xfslibs-dev\ vim From patchwork Tue Jul 7 07:08:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234943 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp660953ilg; Tue, 7 Jul 2020 00:25:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxm4E0Iz3U2qLQliGhwgMFsT5saRm8VcVA4khpAmwQfjBG6JR3hnxScMtIvLEN3ZZtFQr7F X-Received: by 2002:a25:acdd:: with SMTP id x29mr67199255ybd.58.1594106711794; Tue, 07 Jul 2020 00:25:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106711; cv=none; d=google.com; s=arc-20160816; b=ZxwOABRHBR7khV9Ke0DoLgFEwo0h6T8fA/AnVcVU2e3PbPBW26ujIQm3KtW77euSm0 LS/yxXTmmD1AzFbL/YMZ6iPxGfyx87PwYBB8AYxsdIKyfmD1zqcE38Wj9TmtqUafMats uFWWATvFbz9Bmb9npxSncfT+PxQg128TGlTU+HFrIYm10tpSwh6ySsFvxDoPzLGUvjBK izuYs2P2/IwZY7stonzf7QDwYXrjntPx2w0T8eEHRxodna1o23+jsS1l6uxg0vvWZ+K6 I7zF6iW4HJhWHsPRd/XwkGnkGWMqYwDrQVTA2HLWEv84es38gzEM69Ijs/Bego1plbqD DZMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CJmGTUIDpDAF9NLfy0YY76xgc1xY2TT3QfY+IV2fyrk=; b=TK8TEKU0B0OQjCaKVgF2y657gTqO/t/Hq5MxYmwy6cY5tNz1Io76EY8iu+M9S1XsCe ySO0fWSK6tOQC5KV4f+xEizY+32OsB2Tvz0BU3HvJtih+RZ0aWXkWUxU84odo35cjEuR 8G5AJ348ACV0niUapqkoqMRFA+UG8balECSzA/peHC5b/Y9Tq3dG1F65jtRpIDT8gLYX uMXxu5f7haupx47hS7cg1qQsEXw/kKJ0phlAk5Nn+RCajHI8Wwo3wLb5wKvztHXbxPsT X+xRiiuZzAaf0NAM47YSkCu+XfYCo3g1SutIzp901bl1CfCL4WcVdOmFKlx0sW6s9e/b 3WnQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=gTwbzNRR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n6si22291805ybn.213.2020.07.07.00.25.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:25:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=gTwbzNRR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshyV-0001b8-91 for patch@linaro.org; Tue, 07 Jul 2020 03:25:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxq-0001Rg-9P for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:26 -0400 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]:36184) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxo-0006fs-Kq for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:25 -0400 Received: by mail-wr1-x435.google.com with SMTP id k6so44033166wrn.3 for ; Tue, 07 Jul 2020 00:24:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CJmGTUIDpDAF9NLfy0YY76xgc1xY2TT3QfY+IV2fyrk=; b=gTwbzNRR1ZqKp4RUx0KY1/upcjiRfVGNG7g3WUso9pRHCucc5l2gRZaEOopQlKN6ox KX9kdFy4k5SkMcIkTIpIi0LgWL6SXblLUm6tzXqYovOaeFb1jgirMlmkltPnNZaFI3UN 5wMNnV7mHTIVcgN5UMXaxG6KZCCjzkjfpgAffJz2CQP+060F3KHCr9y7DKbdFuQTLlah QnK5IKFf+65YD3+Pal5fiNQZwuLpyE/GfOjQfdFbVr6U23+5yvSu4oeABftk7MQmJe94 zBJFAEs17QgPekubj2E0z9V5h9yZs9VLo/oMt+LQm+kZxKB0kXLNpqrYocsLvfZ3pvmj a+MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CJmGTUIDpDAF9NLfy0YY76xgc1xY2TT3QfY+IV2fyrk=; b=jZhwN9oaolGXp2BOika42h5SL1Q+tqG2fewyk5RP/hR6oFywXePK07ZZSBD2fpxsPY YwnyBBhLC2gEam5DJz39k04aBj0MUgYsk8J+uvVQz8KMrq2jt43CevpcEbPbMwjXN70u UuNB5NRMtsyVVdRBtiCcXXU+expCYQL+X0KMg2OrdZydOYpR7XcxuBuGR/EVbucEihy9 79ef6rtR6Vyr6d0uJ1LeeJQor9JTx5w+FQ/kPzqEzuB4xQ2mfabRb87eV5sERwc6vdcb gvV7WcK3lSo58mlAVLxuI9d4qXsCfVlAoyWd1Il+rsUgtUVzCWCHf1CDpIzdZG3qhMmr 0TtQ== X-Gm-Message-State: AOAM533Pvox5RAa/c0331px2AxVvKI0SzyFiAwDkNZ085o1IpLjup7jD InNLHstpStNlOHc+Pb0FIBYsPQ== X-Received: by 2002:adf:ff8c:: with SMTP id j12mr51860693wrr.230.1594106662802; Tue, 07 Jul 2020 00:24:22 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id r1sm26188861wrw.24.2020.07.07.00.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:21 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id F19181FFAE; Tue, 7 Jul 2020 08:09:00 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 24/41] tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab Date: Tue, 7 Jul 2020 08:08:41 +0100 Message-Id: <20200707070858.6622-25-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::435; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x435.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , Cornelia Huck , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , "open list:S390 general arch..." , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Currently the test takes more the 900 seconds on GitLab and then times out. Running on Travis seems to be OK. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-27-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py index 3aa57e88b0bf..0055dc7cee77 100644 --- a/tests/acceptance/boot_linux.py +++ b/tests/acceptance/boot_linux.py @@ -20,6 +20,7 @@ from avocado.utils import network from avocado.utils import vmimage from avocado.utils import datadrainer from avocado.utils.path import find_command +from avocado import skipIf ACCEL_NOT_AVAILABLE_FMT = "%s accelerator does not seem to be available" KVM_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "KVM" @@ -220,6 +221,7 @@ class BootLinuxS390X(BootLinux): chksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d' + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_s390_ccw_virtio_tcg(self): """ :avocado: tags=machine:s390-ccw-virtio From patchwork Tue Jul 7 07:08:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234934 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp658625ilg; Tue, 7 Jul 2020 00:21:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyT/vMkUWXu3c7CHWRAtUS+X3Y76ZNdwXbUxxc/71UdJT9elVd19lou6jKgVgKWhcldkzAK X-Received: by 2002:a25:512:: with SMTP id 18mr84375664ybf.432.1594106464357; Tue, 07 Jul 2020 00:21:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106464; cv=none; d=google.com; s=arc-20160816; b=aRsVVG5wmh31odYhCsVmZm/2e1uCkQBRlpUBTgY2jsEkp5NvckECnXigPPtTOy4ODS gMsLmKU6xkq46/r9GQrLUqX6YenxlKlTsbO7Laqk2K8pcVO5XoTbWrTSEfJ1Md/W747U RkEhFQ3tl8vo7bGJtLtMn5IWad4sYJktubOUDxnUodcfAPY21hnY3UaEcrLZb+7S0pjs Z6UrAsPJ9yba3WgaVDjmoWmNAwRTEjFX7zBu13RFv4YOFSVyiWtxIBQeX8EfLtQYSozg Jq5ZuJ4sknm9qThJ05r6Fv/OeH0czLTZTkIeJaC8H8+ZlRPm49MrOLylMKZfPFchNLxi M84A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FLGnmkqna9y24iYAUhuJaK9vct+iWHSTDBHb/dvLTOU=; b=SPliLdUWEffT/MlcSD59pypINU0orbnbs9fK9KbJF5sft9nfLkPFLyQJOauZ1u/N3g ZHXx3OE9srZB3nsxbXkbf9gDu6jh/acR9EiIVNmtKDldNhDtgh3hRHOblqcYCGquMVJY IL163+tpP9oUVs3lIU6GtiP02eofVj/aINpXgxxtFDoP/ZV+5K9U/0SrdDk3Zy5PkfSB f7UkW+/dmzt+Qa2RgcHcR/8NbjRiPeiLfqroD18qSR5eaA6Jo9T4BoRxzSrWH8AA2tG1 jgcLuUVtLItmxj5E8rEF0Mb8tOTk46IO3ip7f972wdkAs9Mqp+zkET2ZViql4vLAQZ5V T84A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=de0voltZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 5si17754117ybp.293.2020.07.07.00.21.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:21:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=de0voltZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshuZ-0001pw-O2 for patch@linaro.org; Tue, 07 Jul 2020 03:21:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoL-0006pd-7Y for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:37 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:34608) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoJ-0004rs-9h for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:36 -0400 Received: by mail-wr1-x436.google.com with SMTP id f7so40975410wrw.1 for ; Tue, 07 Jul 2020 00:14:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FLGnmkqna9y24iYAUhuJaK9vct+iWHSTDBHb/dvLTOU=; b=de0voltZRNqo+/KwkqE0gF6XrAgKXXQ4dQzW/I4fqtqgnYUSkT6IaolsEJTunn70aT Q8E1l17oYFOIcm/hshkkCrxu7cRrZVLhjucFhs0Z9FjR1FdAXBYgXW5ClM7YQ9jMvXyz YPmPZk7PowMtdgQCpjye/7rPJ0YPdJHAoF3/xKDZay8MSNLBOnDkl0T/2JmtYjmcpi+a u9xY9N6aFk3JKhWHfF1kE3LxHuk2uxXb/HxnuKM6J5fB06jkCqSSRNS1zouFgYw0nCcB Kz8KrCk4hYQVEbdHSKsHhRkQBQb2sHEMrfjBg32Abi/NCJbcCGRdLsu9YUXa6y4mnhZd 4Dow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FLGnmkqna9y24iYAUhuJaK9vct+iWHSTDBHb/dvLTOU=; b=SIImd8niiD5UEp2SRmJa7CYcdnA1y938A5RAe4qWq+pGC38VqM4AYS612fSwjNLk+R 2RX4zPvCv1e+2bbkX+MboTdRFHf4X7gIP6BBCEH0DMYQ7jEINjDygyiMvdk6Btd7Fggh 7IMC3sgCZOUojt2wZsURT9bXLi3h5tcGk3mLaw+wU4QJ16uWuvKDVdeX2/pXXPpxp2Ir bcMsK+dkAEgl0QCUA3xrRaiRvQqeE/APEzO3M1C3pQnapklryOkQ/o9Xr82dhNaY16na 7GXHEAnVMpXe5++iFF4900pM/UPUlaNcKTJWlJOUa3eUvvxU9ThMG1R59HeV5E0OGYJL mP6A== X-Gm-Message-State: AOAM532a+xLglPXEHTyFgq0DTpXfbWIpAHQizWpaRbareRq0k7WzciWo n2674er8xVjnbgRVUMONibSKdQ== X-Received: by 2002:a5d:51ce:: with SMTP id n14mr52727488wrv.155.1594106073964; Tue, 07 Jul 2020 00:14:33 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 2sm2145190wmo.44.2020.07.07.00.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:28 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 123621FFAF; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 25/41] tests/acceptance: fix dtb path for machine_rx_gdbsim Date: Tue, 7 Jul 2020 08:08:42 +0100 Message-Id: <20200707070858.6622-26-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yoshinori Sato , =?utf-8?q?Alex_Benn=C3=A9?= =?utf-8?q?e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The old path doesn't exist but the rx-virt.dtb file has the same checksum so lets use that. Signed-off-by: Alex Bennée Acked-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-28-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/acceptance/machine_rx_gdbsim.py b/tests/acceptance/machine_rx_gdbsim.py index a44f2c87da0c..bff63e421d54 100644 --- a/tests/acceptance/machine_rx_gdbsim.py +++ b/tests/acceptance/machine_rx_gdbsim.py @@ -50,7 +50,7 @@ class RxGdbSimMachine(Test): :avocado: tags=machine:gdbsim-r5f562n7 :avocado: tags=endian:little """ - dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-qemu.dtb') + dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb') dtb_hash = '7b4e4e2c71905da44e86ce47adee2210b026ac18' dtb_path = self.fetch_asset(dtb_url, asset_hash=dtb_hash) kernel_url = ('http://acc.dl.osdn.jp/users/23/23845/zImage') From patchwork Tue Jul 7 07:08:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234938 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp659516ilg; Tue, 7 Jul 2020 00:22:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwvrsykm0fe43UpiSJQ0bIg5e/t/0/pK9Q5aL3DUcLDFVfRR7zlOAfmtMvQrwVZeow/NHm/ X-Received: by 2002:a25:98c7:: with SMTP id m7mr80594805ybo.369.1594106562270; Tue, 07 Jul 2020 00:22:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106562; cv=none; d=google.com; s=arc-20160816; b=RrqnpcvYrxrs4BMvwe+Q+fRzzShRt+LLN6Zt47X2oxIGi789fPDsZPZLfrgODIH+Br +NxlyVLtVfbXrOcjp9Dw/vjjblpANjmT4Q3iwGar2hwiXMb580ZJLo2mRCieByH/gmKl 6HLyC1Dj51AJFR46+i113Qc8mDVehg8plpzsPXgtaM1fD50c80U9PE3glHlNjOkMpxbf g7/wWDy5Ngp1rguCNkNj9EABjHmcRWg/daZGlXUdY4qhBRamO56d+BHS9BBYAYQ+5O98 T1ylltPNER3NMbo4dTzZHCmEVQyJR9aISio2KZg76AHTUYUOBnQIFIedT9TtNbGtv3/0 plwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=fA+g3LrdgrFUSUvqRSEWu0BkgruGndhEDIAbeRZnv3g=; b=k9Bhf0uBLDz25EqiDPp3N+Wo6CaYC4cT+MF0/i1CahQv9tFkVUcTpW4DzNxlgfiBT4 agKut2STsG4gEZXoVZE0s0crmUZM/NwDkd6RsENR2Mw5kN/WzI8+ZdECDCrPzsaK73eO bPGuOmJAJVx3yLaBcahDxve6lgzfPo8HRwydIkOyDfvrG//W/V7J4yUoIF/7239OEoCK jdXcoCeea2/eorvXWqeAUfKbvEcp/1cMF0HlLZE3z0ZfP7k6tqqdzU0gRY4oFDQR+LnA KISWiIUuExcQQsZOAT+pGcxVYTnx0t1XWEMZX5Buwv0qbG90wJIanMUR7nv6hw1gw+Un aMTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yAMXv7or; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k2si31812740ybt.375.2020.07.07.00.22.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:22:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yAMXv7or; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:39448 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshw9-0005I2-MX for patch@linaro.org; Tue, 07 Jul 2020 03:22:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43836) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoX-00077m-E9 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:51 -0400 Received: from mail-wr1-x432.google.com ([2a00:1450:4864:20::432]:42414) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoT-0004vV-GA for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:48 -0400 Received: by mail-wr1-x432.google.com with SMTP id o11so44021125wrv.9 for ; Tue, 07 Jul 2020 00:14:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fA+g3LrdgrFUSUvqRSEWu0BkgruGndhEDIAbeRZnv3g=; b=yAMXv7orRZmRvCHJtjRNT9bVYsLmUCu9NfC+utNdbJxOTsryrfGKznJ+LrenMzk3IZ +RW1ZOglfVhTmA9T1rYkmZNI9pKDUj43jJrfu0Ad7uYI8BN35ziaza2D5Z9sSkAJZk5q GduDzZ1mBmfJqsDYDEHI4Wx0AHYNKjEwXmwKAy+jdNofaGCR6dGf8B0jWR4wvObw7qA0 uhJAe5Ew6Ry6/Ji9wSA5A1Am4bxcNoJwo2zCzj0wms7+1oleULKshQrxGcxlcb7A3sXC bGiyGW3lYjv13pgZzOfl4aEVJWIkyQXjDTIPBG83NgYE/b289trglZYX97kDlG2fud9O qQzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fA+g3LrdgrFUSUvqRSEWu0BkgruGndhEDIAbeRZnv3g=; b=PJz/20QdHEXBGd+zZo4eZrbxzHuDjhKoaDajUublhb11HFbDtyYxyC7vh2a0B5VEnt cZTDzJP17kKrSW4la0PyGPfrDF7VOdgER9pbxvjcdayE1Rlz+RCZz5lqu5a3rk75bxoH 3Don9sXbCxSSJJsW91ofLeaU69we9OpLTGBDX4nO2BhCbm+HAzopxblQt/9xXdAbZSs6 JVDIn4UoPbKhw+ma4mwGiYSeNNVTojTcEM/ZmEOuWyLZ1w6JJeryAU8TrlEabJ5+vUPx LyDKG/L0pdSLOYvBX+vEqoF8nzeKe01aomqDqglCsdLK++LtQP7OaD+EIW4ssGIL58z0 oxkw== X-Gm-Message-State: AOAM530o5UWGVkhlmVbvBoKFGz3IftZ5NRvCatfZri8hCdMGn3OuT/Y8 cdBusRBvZwUqUf9pPO2hYF54uamTnFw= X-Received: by 2002:adf:b608:: with SMTP id f8mr56804021wre.363.1594106084003; Tue, 07 Jul 2020 00:14:44 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id d81sm2677820wmc.0.2020.07.07.00.14.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:38 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 276471FFB1; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 26/41] tests/acceptance: skip multicore mips_malta tests on GitLab Date: Tue, 7 Jul 2020 08:08:43 +0100 Message-Id: <20200707070858.6622-27-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::432; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x432.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Aleksandar Rikalo , =?utf-8?q?Alex_Benn?= =?utf-8?b?w6ll?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , Aleksandar Markovic , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Aurelien Jarno Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For some reason these tests fail all the time on GitLab. I can re-create the hang around 3% of the time locally but it doesn't seem to be MTTCG related. For now skipIf on GITLAB_CI. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Cc: Aleksandar Markovic Message-Id: <20200701135652.1366-29-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/acceptance/machine_mips_malta.py b/tests/acceptance/machine_mips_malta.py index 92b4f28a1123..7c9a4ee4d2d6 100644 --- a/tests/acceptance/machine_mips_malta.py +++ b/tests/acceptance/machine_mips_malta.py @@ -15,6 +15,7 @@ from avocado import skipUnless from avocado_qemu import Test from avocado_qemu import wait_for_console_pattern from avocado.utils import archive +from avocado import skipIf NUMPY_AVAILABLE = True @@ -99,6 +100,7 @@ class MaltaMachineFramebuffer(Test): """ self.do_test_i6400_framebuffer_logo(1) + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_mips_malta_i6400_framebuffer_logo_7cores(self): """ :avocado: tags=arch:mips64el @@ -108,6 +110,7 @@ class MaltaMachineFramebuffer(Test): """ self.do_test_i6400_framebuffer_logo(7) + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_mips_malta_i6400_framebuffer_logo_8cores(self): """ :avocado: tags=arch:mips64el From patchwork Tue Jul 7 07:08:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234925 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp656795ilg; Tue, 7 Jul 2020 00:17:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyakXb6Wqy2mkpMIqoMqfFKaPjOGEJ9O7LUs3k3IhIY4PPJ2tHO6Bx8Y1P8dXxOSfe6Vqda X-Received: by 2002:a25:8008:: with SMTP id m8mr28462371ybk.104.1594106262113; Tue, 07 Jul 2020 00:17:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106262; cv=none; d=google.com; s=arc-20160816; b=Aq2049tJ0jtz2ELEafJLDicpLXXzMR8mEu8zGm/BwDIbiogd312v2yFHVjx33MDkAw 0ITX9y3EvZmeDSNXOfDIvkNu+2PSnZfWAdgKQFcI/PVJa4qSbtQeJgg+DqL2NcXhj4Le Tv9XLnqFJ6umPnsk7MUPE+ITm0DWSrJ74plwKQur0+49KX0htE/CIi42J0vvYMnyoxBi YbzZePwK5ahuP+o78x1sGooWs10iBJoOiVaBGd2WARvWFksHBSwySkj70iZ07SiIbu4m T+whMSHxt5wzRqt18KZO/uLBKp2ytkPiuEISTJ5ShIKmChwQk5eVJU5hIMKmouIJRgqu PPcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=fVpFU3Bn0gx8fweRN9XBf8loHnsbKZ9F1g1Uyun+Gsg=; b=yxbClXMvKppJhKldhk0tu4N3Tcw8Bo9v7sZoJw8czhdv9td0apLBf0lf1IzGrLkumv +MWyGxLJRSDMBeaWINkRZb0hJJO8zyWoyMnQVjBlpmTMZcwV7NxjdkXAj3EeHRz6s3/A uFqIAZoENVQG9+FEjxhhLKwofDgZ1HgGeigfioJiSTXh5PrV6l/ZP0oxUjLrzgq5DDe9 qFdacLLDmPWNHutKBArjjrxhc3asnODHHLNYbmI2oTOWyAYzkLfzM+nz8Fo1w3CJd92r cHa7cqgqtUETfs2LZ0YLvci5zr+UxZ00BnZpZkdaQop1dbt4iqeBc4AQCbNPOALZYmpF BMYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=R9v8p9Ae; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f142si21547204yba.230.2020.07.07.00.17.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:17:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=R9v8p9Ae; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshrJ-0002AU-FP for patch@linaro.org; Tue, 07 Jul 2020 03:17:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoH-0006hH-TP for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:33 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:36104) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoG-0004rI-8b for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:33 -0400 Received: by mail-wr1-x436.google.com with SMTP id k6so44006533wrn.3 for ; Tue, 07 Jul 2020 00:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fVpFU3Bn0gx8fweRN9XBf8loHnsbKZ9F1g1Uyun+Gsg=; b=R9v8p9AeUq2v6ydPU9XeLm7KEUpSWIYFUsXmqGq/pcbaCLgExX3UzYb8WSY1ZVO7vS R0N9dhWsat59BvoOOTpVaWsq+1k6+04KSYhwWYPOrSBpOMbJDz9mc+pN0Xi7yI33Pvvf BaAmOlCK2xYKRpG4koFGnaHFQhjGnZrCnw1BWd8o2toAK71FS24ephW2oWZCE9orVsz4 PcDNLtKH18/L3PO470NtrWJh2FuzMVAbuDgM1jpeCfw/uXHf2EBhTjhxxrgEbAXJZ3bw 75en7898w2I+hSoX/2ZPLXCmZdsQeWeE3LwHRXHvQEm81XLQVmlmdqOE0AESDX0S9mL8 ewPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fVpFU3Bn0gx8fweRN9XBf8loHnsbKZ9F1g1Uyun+Gsg=; b=bzvimRNmisSAHlk65IIVwkGZpREcyVja/94++UkT/ZuRnUBTSXCdfpmgKeSRLL36pS y9tDo5MH/BVNrNZVI4kfGrK9f8H9wiKo2Ri6tZ00GoBURHWlhatvwUSBhPTEp2+MlCtG t4/oqDUyBZ8Y5pTEzGAPnKJCXK+6zxCJPhitWFke3tupHxczoSSgtap8Co6KvwqGL2Lp 4JvjGGrO4crCp7D16tUYQSpWqO/OULPVHRZipXMO6mDWoNYj+usVhwvLj0UFuSNJ/E3a 0Osxuf7kOlU43wxfkU/wRVo5QDPY/SaQn2aJIBYAQt7zXKg6YRzhfVfCqBxc+cXR7xLb +tBg== X-Gm-Message-State: AOAM533WchUaytXSomYHuTQSWdBi3jKWiw3cxaq3ZJHmWagux9JSx2Az gM/TENOW9UzIzWjHOkSpfpsNbQ== X-Received: by 2002:adf:b1ca:: with SMTP id r10mr24403519wra.150.1594106070886; Tue, 07 Jul 2020 00:14:30 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id c25sm2107501wml.46.2020.07.07.00.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:27 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 3CCD21FFB3; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 27/41] tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab Date: Tue, 7 Jul 2020 08:08:44 +0100 Message-Id: <20200707070858.6622-28-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This fails on GitLab but not when run locally on the same container image. It's very confusing. Signed-off-by: Alex Bennée Acked-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-30-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py index a3e54d3fc915..a249e2f14a24 100644 --- a/tests/acceptance/linux_initrd.py +++ b/tests/acceptance/linux_initrd.py @@ -8,10 +8,12 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. +import os import logging import tempfile from avocado_qemu import Test +from avocado import skipIf class LinuxInitrd(Test): @@ -51,6 +53,7 @@ class LinuxInitrd(Test): max_size + 1) self.assertRegex(self.vm.get_log(), expected_msg) + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_with_2gib_file_should_work_with_linux_v4_16(self): """ QEMU has supported up to 4 GiB initrd for recent kernel From patchwork Tue Jul 7 07:08:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234940 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp660246ilg; Tue, 7 Jul 2020 00:23:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwcKm1Saxq+orx0d5ApciDzOhWoufLYbdBjxR6Fw3WE2MvNiSQanbn/AbpaMP2p0fM4EHEA X-Received: by 2002:a25:5d2:: with SMTP id 201mr62903930ybf.280.1594106635612; Tue, 07 Jul 2020 00:23:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106635; cv=none; d=google.com; s=arc-20160816; b=aj91MgJblaP1QP+gc2JGbNokBQc8Ish4auSdcxbh6fgiKGBup4LGYyS5KHMUJPBtiz LWiIXPrp95wUu4G9ie+FILzytNGP1lkegaYNDQHLygGBklXxrKl+7pqB6359rx2/r2es ciTrTQWm2g/WX+JaQlOCCNDn+gavbwIhCK32TTwOKvmm40mzQ7dpI2Exc493twR8QyVV fqvA6nRgtftJRu4tFd6j5f/lveiIRRSJ6k+Rs0ujiFK3vEjs95pTOMf/wro8KFneEcgY lfAJe7OWyGZJ/HUi7a7SbOgMNIAKH1FD2d0dM0ask4iaC5u4M6vDL6J10tGB8mdnajOi lOYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tcFFr6wqntF8eZqbijSLJ4lRc28E2R4EjbxYXolVTZw=; b=h0wV4Sotys8OGj8dojF3nNI1uA3lxOT0tzN6s2fipQn0nKyVad/VnaBR+B5NkWvPO3 AMjHjk1maGnqFTWtHtWPT1jCq4LQtW06ziHoXCp50b0Npte8hr8rowRV8vaBEltWlemz 11soD4c0+ql4HKrXi3rfoYu9/mkv9X70ZOzcY96UGM/u7q0bIcYIZ9Jmr6yvrRqIEbrs URdiQrWdrxUWGdXOhOzN1XbKn5p3teFcR0YQCMbJ+hcI90lsW9Mj+RuOTSkKvZ0HE3fX vtSsGKbJFgVLfZSCjLfm9URRlbzmHCEC7nkux4gYSLEgD19biKS9Ds4u6vikFKemvTG5 k6xg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=uSm9Utno; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m3si11435564ybb.70.2020.07.07.00.23.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:23:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=uSm9Utno; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshxL-00087o-2u for patch@linaro.org; Tue, 07 Jul 2020 03:23:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoC-0006T8-0B for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:28 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]:53758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jsho9-0004qA-R6 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:27 -0400 Received: by mail-wm1-x335.google.com with SMTP id j18so42125764wmi.3 for ; Tue, 07 Jul 2020 00:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tcFFr6wqntF8eZqbijSLJ4lRc28E2R4EjbxYXolVTZw=; b=uSm9UtnoQG3JZKw3s+8KHJyinyr40UTTxVMWcJgNWCt4y7E3xg10qHv15Xnxxma1d/ I9fqjaEV3bi/fduP/1yRi/xSnlFuCTtvk256uZbQkMjytV8zA40Hg/00o+KESJ9jv9yX oDiWho4r7zEM8JARh9+RJaujHAd9h41WBF+ie78WYbkh2JJYMdlJ1rRnZhRhEb5zwnaE MdbVLjWgSmNrp+/vJxTOyDnICAn+gamdbXpq/Ri0EwoosEXzu5Hs2vvTyEhOrw/mSien JliEeCGDhrd4pHp3TY3au05NwhjMDcQXJnm0PwaNS/rJnRGgQRRc50MRvitbeF5qn6bE Ig2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tcFFr6wqntF8eZqbijSLJ4lRc28E2R4EjbxYXolVTZw=; b=lt2yV5H76YnM8QCiQ28klGh8k6ZPCvN6oXBbqO8YdV0ALOzmHUHSpByN0ctYx3Pqg4 dLZR39YKTvPXHC44aJwwAni/eEqcfxlY6dniNQKN+zuahi4CCbyv2bySkd5TDc13CDzl JTWxw91xaK0Xh5+Z+eMLshG0US65mjFhZRDOB7Iul1seJ58I0NuESThdBONqOHTFcXGc lwt/QkVOV/XHd3IRBAMIjQrQA7GqoBRyeDLmjscAEoen9wM9X5usSDMXCwY/xXaKXQ3s nlnMoooguMQbJqp1LiQhOx3nqV/ancZcY6vJJWhAaLdRCQZzdNZywJ/KTBZu1/+ictLa mPFA== X-Gm-Message-State: AOAM531F7rN0syzXZo817YNQbfVO+mGc835YzrvdcHJOvWTOWYWecsMz AAHRfGr1esVZ+yyrFi5IHZ8kRQ== X-Received: by 2002:a7b:c92e:: with SMTP id h14mr2570639wml.36.1594106064198; Tue, 07 Jul 2020 00:14:24 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z6sm2174406wmf.33.2020.07.07.00.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 58D621FFB4; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 28/41] gitlab: add acceptance testing to system builds Date: Tue, 7 Jul 2020 08:08:45 +0100 Message-Id: <20200707070858.6622-29-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x335.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , Pavel Dovgalyuk , Cleber Rosa , Paolo Bonzini , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As part of migrating things from Travis to GitLab add the acceptance tests. To do this: - rename system1 to system-ubuntu-main - rename system2 to system-fedora-misc - split into build/check/acceptance - remove -j from check stages - use artifacts to save build stage - add post acceptance template and use Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a7abc55a5c6a..93baf98232d7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,8 +1,12 @@ +# Currently we have two build stages after our containers are built: +# - build (for traditional build and test or first stage build) +# - test (for test stages, using build artefacts from a build stage) stages: - containers - containers-layer2 - containers-layer3 - build + - test include: - local: '/.gitlab-ci.d/edk2.yml' @@ -24,26 +28,85 @@ include: ../configure --enable-werror $CONFIGURE_ARGS ; fi - make -j"$JOBS" - - make -j"$JOBS" $MAKE_CHECK_ARGS + - if test -n "$MAKE_CHECK_ARGS"; + then + make $MAKE_CHECK_ARGS ; + fi + +.native_test_job_template: &native_test_job_definition + stage: test + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - cd build + - find . -type f -exec touch {} + + - make $MAKE_CHECK_ARGS + +.post_acceptance_template: &post_acceptance + after_script: + - cd build + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat + - du -chs $HOME/avocado/data/cache -build-system1: +build-system-ubuntu-main: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu + artifacts: + paths: + - build + +check-system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build-system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check -build-system2: +acceptance-system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build-system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 + MAKE_CHECK_ARGS: check-acceptance + <<: *post_acceptance + +build-system-fedora-alt: <<: *native_build_job_definition variables: IMAGE: fedora TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu + artifacts: + paths: + - build + +check-system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build-system-fedora-alt + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check +acceptance-system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build-system-fedora-alt + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + <<: *post_acceptance + build-disabled: <<: *native_build_job_definition variables: diff --git a/.travis.yml b/.travis.yml index 74158f741b18..c24dfbe377f3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -289,29 +289,6 @@ jobs: python: 3.6 - # Acceptance (Functional) tests - - name: "GCC check-acceptance" - dist: bionic - env: - - CONFIG="--enable-tools --target-list=aarch64-softmmu,alpha-softmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-softmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sh4-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu" - - TEST_CMD="make check-acceptance" - - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-acceptance" - after_script: - - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat - - du -chs $HOME/avocado/data/cache - addons: - apt: - packages: - - python3-pil - - python3-pip - - python3-numpy - - python3-opencv - - python3-venv - - rpm2cpio - - tesseract-ocr - - tesseract-ocr-eng - - # Using newer GCC with sanitizers - name: "GCC9 with sanitizers (softmmu)" addons: diff --git a/tests/Makefile.include b/tests/Makefile.include index 3f4448a20bc7..c316e0d66475 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -944,7 +944,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ $(AVOCADO_TAGS) \ - --failfast=on tests/acceptance, \ + $(if $(GITLAB_CI),,--failfast=on) tests/acceptance, \ "AVOCADO", "tests/acceptance") # Consolidated targets diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py index 60621417ddf0..62d2db8c64be 100644 --- a/tests/acceptance/replay_kernel.py +++ b/tests/acceptance/replay_kernel.py @@ -73,7 +73,7 @@ class ReplayKernel(LinuxKernelTest): logger = logging.getLogger('replay') logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_x86_64_pc(self): """ :avocado: tags=arch:x86_64 From patchwork Tue Jul 7 07:08:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234933 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp658390ilg; Tue, 7 Jul 2020 00:20:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxPJodo/oJav1Kl50SFJwUEGyudVeEfz8gAMne3Df+O0ROIjtQFdqjAj3vNroTjXygyuRiB X-Received: by 2002:a05:6902:100f:: with SMTP id w15mr527999ybt.477.1594106434653; Tue, 07 Jul 2020 00:20:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106434; cv=none; d=google.com; s=arc-20160816; b=HuLbC80VSduin9w+fgC61G4sTFg+Jjl7NgHBuUOWYzUwiki4FQDGg7cApGEic4Ypdb 5lEu5UUfNt+fSC73ObKTUJGUDVK3fc0+IgYEXliSKZWrnyWB+3Kw3IekowtMVgKDjZb5 9CNw1/YGnmOk8QHrGtPo4QWkaI0qWLActr0/xmJlcvSPQxC5jLYFrowY2yyMLnHsDCx0 fZcy7eTppnpdjMaSb+5miYJSCl5+hWVZkDsfczsn3AZv+w00fXaWp2ehyPNfFNJZYWRW p57yfIUxtPg0mXws9R6uUwvU0oXBS/qFCU/Kv5lYYNWjHc7BrmU8McXTUViCNg+jBjoE cT1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Y0AvFK9jDhQslOqQ/+Gd4Ly1wJW6CnLI4eXMz6hvfIk=; b=0/ZAgQZi3lE+dSr8hsJj1eTpjw0L8i/GJonVf+8zBw0EK5XSSDbUupmsgIEkPfg1uq eeICFvvO9dCt4QPM50Ce2jcURoTjMSdBHg/MJRVjqRIsYCBYlTIRkQGRSbo19ykfjGtl po+5uIKWNk8V61mneOrB5ol8yynViJuEl5EUCQhNh9KjFZvU6tSaNSuu0bgrFjqGBn52 khIp9qugxlgXqXThnBipTc7OAa/Cd5eQRjh1ndxXxDvAR+Rp8Qkj3TW/+L5l2bbDT5kB 45wRV7GHlv98InLIV+RabCXSBFTINxUm5oW+SPHYQoAxcweO8vasjzlp0k8wkKvUtC5v IVzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ha9b2I93; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d69si21359624ybh.331.2020.07.07.00.20.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:20:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ha9b2I93; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshu6-0000ve-23 for patch@linaro.org; Tue, 07 Jul 2020 03:20:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42696) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjH-000528-S6 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:23 -0400 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]:33550) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjF-00046i-Vk for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:23 -0400 Received: by mail-wm1-x343.google.com with SMTP id a6so1070448wmm.0 for ; Tue, 07 Jul 2020 00:09:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y0AvFK9jDhQslOqQ/+Gd4Ly1wJW6CnLI4eXMz6hvfIk=; b=Ha9b2I93IMyiA/qZRUClTEq9hMNYNlV7RFMmE/GrnzS0CDKv0d7t5f96jZ2qknQ4sX +fImQQ1b7HpWPKnEqiCkJAxOwK4TaS86K9H2SqMB+MVJz7DH+LfTLT0kpqTcXuAqDHwx ztHy3KZDmHqcUxDQMdCdLQ0aN/O3ZRsdCkgL+F+eiMq5HcDuZJ2YE0LILxPR0trGiezP F2X84S6Nl5ZS/TDxqLtD1v6En8QyywJm6n6T6b4y7q26nweebo/XcbxSyPYWLlw1CcXn IUiYFbnp8VPscIzvIxHPHOJUzOHuywIDNXuBvwJlRjml8yEiPF44DuyXh1C2zaTmLK6+ Pzng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y0AvFK9jDhQslOqQ/+Gd4Ly1wJW6CnLI4eXMz6hvfIk=; b=K9NHfWoJ8ePWTroaGHJ++gUEsyYziWRdKQl58plZVA5DHyT0kifRpA5yY4INy1VnCC pBux3i8SQu0bH7Tgjfi5jD/abx9+Ch/wuA7RaZZUNSWZ7xY+s5Y7UBBdLWUVXkoNQJVF NSC+Eka1H6H2yak5jsN7witTKZZdVtr6juo/4Sduq8uUI6h9484ENXl7Q6KZYPP4e73D OX4ASZhPyNCrwPHpUOpJ6pFqR29R9O4zMolGWvLGYZ5Ef1dYzuDhhe+edMnrzb90w0No KMZGJQnd6t/75j7O1OGahwnFpjdBAJcb4g44upTncLy/0Y0TefV/LPIu454lzhbeJGmy rOvg== X-Gm-Message-State: AOAM5323jhkqiFIkjlj9FLNvhP4XTG6TYpdUM5wU4R/iL6AlXoKalHdv LLtiDykJ/tLC1qT0nQ23dL+/uz4BeNI= X-Received: by 2002:a1c:303:: with SMTP id 3mr2730816wmd.180.1594105760534; Tue, 07 Jul 2020 00:09:20 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id n17sm26170287wrs.2.2020.07.07.00.09.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:13 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 6F0461FFB5; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 29/41] tests/tcg: add more default compilers to configure.sh Date: Tue, 7 Jul 2020 08:08:46 +0100 Message-Id: <20200707070858.6622-30-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::343; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x343.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We were missing a bunch of compilers which we could use if they were locally installed. The defaults are based on Debian as they seem to be the best distro for well distributed cross-build compilers. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-32-alex.bennee@linaro.org> -- 2.20.1 diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh index 2326f978562f..37e49736ca40 100755 --- a/tests/tcg/configure.sh +++ b/tests/tcg/configure.sh @@ -46,20 +46,29 @@ fi : ${cross_cc_aarch64="aarch64-linux-gnu-gcc"} : ${cross_cc_aarch64_be="$cross_cc_aarch64"} : ${cross_cc_cflags_aarch64_be="-mbig-endian"} +: $(cross_cc_alpha="alpha-linux-gnu-gcc") : ${cross_cc_arm="arm-linux-gnueabihf-gcc"} : ${cross_cc_cflags_armeb="-mbig-endian"} +: ${cross_cc_hppa="hppa-linux-gnu-gcc"} : ${cross_cc_i386="i386-pc-linux-gnu-gcc"} : ${cross_cc_cflags_i386="-m32"} -: ${cross_cc_x86_64="x86_64-pc-linux-gnu-gcc"} -: ${cross_cc_cflags_x86_64="-m64"} +: ${cross_cc_m68k="m68k-linux-gnu-gcc"} +: $(cross_cc_mips64el="mips64el-linux-gnuabi64-gcc") +: $(cross_cc_mips64="mips64-linux-gnuabi64-gcc") +: $(cross_cc_mipsel="mipsel-linux-gnu-gcc") +: $(cross_cc_mips="mips-linux-gnu-gcc") : ${cross_cc_ppc="powerpc-linux-gnu-gcc"} : ${cross_cc_cflags_ppc="-m32"} -: ${cross_cc_ppc64="powerpc-linux-gnu-gcc"} -: ${cross_cc_cflags_ppc64="-m64"} +: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"} : ${cross_cc_ppc64le="powerpc64le-linux-gnu-gcc"} -: ${cross_cc_cflags_s390x="-m64"} +: $(cross_cc_riscv64="riscv64-linux-gnu-gcc") +: ${cross_cc_s390x="s390x-linux-gnu-gcc"} +: $(cross_cc_sh4="sh4-linux-gnu-gcc") : ${cross_cc_cflags_sparc="-m32 -mv8plus -mcpu=ultrasparc"} +: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"} : ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"} +: ${cross_cc_x86_64="x86_64-pc-linux-gnu-gcc"} +: ${cross_cc_cflags_x86_64="-m64"} for target in $target_list; do arch=${target%%-*} From patchwork Tue Jul 7 07:08:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234950 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp664172ilg; Tue, 7 Jul 2020 00:31:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwutDtD2aQH6oQEjT4aX/N2rMyuSSdZoBXIB344cn5FO4L47xQpUF1KFWHuepjce81pwxfi X-Received: by 2002:a25:2981:: with SMTP id p123mr14160113ybp.260.1594107062337; Tue, 07 Jul 2020 00:31:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594107062; cv=none; d=google.com; s=arc-20160816; b=KUI3CvN4vqFU6jzaEQgpIzEENNGs+ulCVuxif61ez0ApPsLoRzVIb5U/ilzyt8EaPA m4bhWCm9W9o6eO+jjzx5YZgKemzKhdty7+16zAnNoRN+Dv1WeftvNL6vtulxfgHewmAL 26rEIqlVtyhSYvslD/6eknVgG+s5Qj3BKODR+jaDaYn5mvyS/fBytb2D6E6OnCe44DVs ho3n0myKZScjz+rTTl0LAmZp/Lx1WBdQXkDz2Sxg78mKPJhvgTufIJsDsZxgidI44a9y 50Tpl/HPMv7uyj3XPEw5sKEddi8Z7WrUG5LbmB7UODiVuqqv2D17FPvEh2gTxt1v8E1b zI4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=m1Cz8IQnUh6NpcECGMGmsDtXwGSngF0Lzci6sx+px08=; b=FgWtMD0Zqr6PLnDjSWxvR/NZW67i+fPWh6D1XR7xqpt5KAdPbTrXUAE48xsJDRtwbb fNnROjYwnF75kNRlhJ/fiqCMGsOLvfn43WsUEAwez4PtKF6eEC/arw2xrK2CghwxAmun NJXTUQKSBqgy3FjBXvroFHLJRcog/khzrCVlVnDaDlSbG20yyNj3zssQHZnTcdUIVZwL Cw69cdCYtuB+OcBEhHswfvwq0rlQGLeNV+R7T2mjg5Ci9oz/y71HhfAreu0GMSVCJGzv w7+YMOVUIt61NZcT3bWvPX1PZLmLtlzMBh24MsaJZtnr4Hf07Gcgou19OdO6FHpDNK7y kgDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=o7o5dpiV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w6si22172183ybc.441.2020.07.07.00.31.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:31:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=o7o5dpiV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44884 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi4D-0003HK-JK for patch@linaro.org; Tue, 07 Jul 2020 03:31:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxv-0001dz-KW for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:31 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:43980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxt-0006hF-Th for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:31 -0400 Received: by mail-wr1-x42f.google.com with SMTP id j4so41598336wrp.10 for ; Tue, 07 Jul 2020 00:24:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m1Cz8IQnUh6NpcECGMGmsDtXwGSngF0Lzci6sx+px08=; b=o7o5dpiVbw/Gbqhr50c4Q6ywWPBUR+7yUm8mlIGpn+2HDhUah51XdnA1F8YaNpAi0l vnJh19F1ms18ONVAE/CtRdz6yU02bFJF9amH5CHmW2RbvmOl0UmjPcCp0REaLQNWbcfH +09Rw6ozJ+n74A4/pwMVNpc1tEaIOFOI9VDhQSRb++vnKd3U7L0ev7lXoOgJbDrPQ5ql lr+mY1Nt6T4TP+m3hDBNXT9z79JeiFZ/tX74VUUvOGW9r3jjTIwxEc4DqY9UetFKIC5i MGwDdtjLwS+Ob9wt3XT9MNyuYmuuV6kNk5qjkcEWTKsNjCrh5yk62KESnJ3/oEULciFf sGxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m1Cz8IQnUh6NpcECGMGmsDtXwGSngF0Lzci6sx+px08=; b=WVAaKR/xCznw0H9DA77kxqOcy5jSXQHhIsWAkNwT+ILSWmFYX+vsygstrGHNpcV2lQ kS1F1q1e+rKxqNkD41ZLtmARZkx5A3sJLGk3708ZfH4I5Vaop39QUBz70MnpOVSpRyBP fbi8pCSqNssRokyq5qViTIiw23U0u0aC+irE3WkULJIttXtQ6DP2L+OhqW3w9bHF975I KxJuCHi01kuLaQ2YRnOFkdsisT4fVFKoznbThRneO5JMqy3HDMSYnRFCrmb7QvkfH6v8 zlHfnGeW94fsWkoxfnZfeKzCeQ7tHTt32RqNphTNFegNC1NsQgkIKIA1teFiPFhHwf90 1nbQ== X-Gm-Message-State: AOAM532nHttFndodF8spab91GfW3/dIW5fj7x1ewc6sfJuSpDeHIkQx+ qXw3rqBVIbREujH8KCCDyIYUcw== X-Received: by 2002:adf:8524:: with SMTP id 33mr52091346wrh.366.1594106668552; Tue, 07 Jul 2020 00:24:28 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id x185sm2385198wmg.41.2020.07.07.00.24.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:27 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 883CD1FFB7; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 30/41] tests/docker: add a linux-user testing focused image Date: Tue, 7 Jul 2020 08:08:47 +0100 Message-Id: <20200707070858.6622-31-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We happily use all the cross images for both cross-building QEMU as well as building the linux-user tests. However calling docker from within docker seems not to work. As we can build in Debian anyway why not include an image that has all the compilers available for non-docker invocation. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index f353359fd8ba..a7621c4204c7 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -64,6 +64,13 @@ amd64-debian-cross-container: variables: NAME: debian-amd64-cross +amd64-debian-user-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-all-test-cross + amd64-debian-container: <<: *container_job_definition stage: containers-layer2 diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index a26177abc258..a104e9df2817 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -134,6 +134,7 @@ docker-image-travis: NOUSER=1 # Specialist build images, sometimes very limited tools docker-image-debian-tricore-cross: docker-image-debian9 +docker-image-debian-all-test-cross: docker-image-debian10 docker-image-debian-arm64-test-cross: docker-image-debian11 # These images may be good enough for building tests but not for test builds diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker new file mode 100644 index 000000000000..dedcea58b467 --- /dev/null +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker @@ -0,0 +1,53 @@ +# +# Docker all cross-compiler target (tests only) +# +# While the normal cross builds take care to setup proper multiarch +# build environments which can cross build QEMU this just installs the +# basic compilers for as many targets as possible. We shall use this +# to build and run linux-user tests on GitLab +# +FROM qemu/debian10 + +# What we need to build QEMU itself +RUN apt update && \ + DEBIAN_FRONTEND=noninteractive eatmydata \ + apt build-dep -yy qemu + +# Add the foreign architecture we want and install dependencies +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ + apt install -y --no-install-recommends \ + gcc-aarch64-linux-gnu \ + libc6-dev-arm64-cross \ + gcc-alpha-linux-gnu \ + libc6.1-dev-alpha-cross \ + gcc-arm-linux-gnueabihf \ + libc6-dev-armhf-cross \ + gcc-hppa-linux-gnu \ + libc6-dev-hppa-cross \ + gcc-m68k-linux-gnu \ + libc6-dev-m68k-cross \ + gcc-mips-linux-gnu \ + libc6-dev-mips-cross \ + gcc-mips64-linux-gnuabi64 \ + libc6-dev-mips64-cross \ + gcc-mips64el-linux-gnuabi64 \ + libc6-dev-mips64el-cross \ + gcc-mipsel-linux-gnu \ + libc6-dev-mipsel-cross \ + gcc-powerpc-linux-gnu \ + libc6-dev-powerpc-cross \ + gcc-powerpc64-linux-gnu \ + libc6-dev-ppc64-cross \ + gcc-powerpc64le-linux-gnu \ + libc6-dev-ppc64el-cross \ + gcc-riscv64-linux-gnu \ + libc6-dev-riscv64-cross \ + gcc-s390x-linux-gnu \ + libc6-dev-s390x-cross \ + gcc-sh4-linux-gnu \ + libc6-dev-sh4-cross \ + gcc-sparc64-linux-gnu \ + libc6-dev-sparc64-cross + +ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs --disable-tools +ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user From patchwork Tue Jul 7 07:08:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234937 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp659122ilg; Tue, 7 Jul 2020 00:21:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz11h8kletBSF7ASF58QXKcx6bgqxRsAo0dXWZyJK6VFmxHQk9mUlpd2Qk+KC7MVgYdzMWi X-Received: by 2002:a25:bb82:: with SMTP id y2mr79732672ybg.392.1594106518960; Tue, 07 Jul 2020 00:21:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106518; cv=none; d=google.com; s=arc-20160816; b=o38isp8q2T3MGlA4qMGxWmXKExjX+ZKONSeOLFlFNo9GvlM3QQ+RI5hOAk+MHQdZcB fqd/JXR7MG1c2h2C6K86005UThgAzZi3wsUmclta27V1XQd3uos5CH/v0au0ep+cKFbC bbHYTuCTuCWnCP9mkYyk6tBKaIXxxvHYnBdutyvhNsOCwg9V1yhpqJJM6vlXEJ9iRUsi 7oP34VzLBCB4dPZLKBMeB7ReodV8hciJsqvTFLOn1Bsl+NUVLC6Sn5W+32UPwrgb9HU8 Vrhpwk1BAMDDCcmYlPP57rPdh4FUMNJ/jNU0RO5tggeX4ZCw4aY20sxa6fYFF7Lwjif3 vrXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+v5zmTqMHanAlbP0rCuGswlB7G9ERwBp467Jydp4J1s=; b=gAehMA64Eeqj33+HasscgDzC+Qprb8hRETFmFfLlvr/HGcm0MtfHoFT4Qex+nMJnBi VW48yEDlxVpBdyhH9M9TxEu9SOorGBkT9F+leZ5EAinGi4Qw+pId6lSAJ2acjO4UJryG Zlm0fEWgQArss7zeQdXx6OzyaXSdWPRKiClsH5KWMkurWC0cax06b5M4RbeYdXM1Biax 78mXFPiMxfMNgaOm8dnlFmvjkXtilQedHErtC/zjqOk0v8rX1KHvDfanKHe3GxbKRMxA ME4rT74SrfFhGtLUQz0S09TnTPEy4Q552uVVKbyct+qLANUH41IdobYxEM8Ii1dIzvUB RY0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Hd9Rmt4S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 187si11789367ybv.470.2020.07.07.00.21.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:21:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Hd9Rmt4S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshvS-0004mx-CW for patch@linaro.org; Tue, 07 Jul 2020 03:21:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoB-0006R7-9T for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:27 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:45561) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jsho8-0004q3-Uo for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:26 -0400 Received: by mail-wr1-x434.google.com with SMTP id s10so43937961wrw.12 for ; Tue, 07 Jul 2020 00:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+v5zmTqMHanAlbP0rCuGswlB7G9ERwBp467Jydp4J1s=; b=Hd9Rmt4SPUXUgR0Qsa0W06b7poQu1JKVGeX9VguAR8b4SL1A39TybJOB3rgKo8kHiT FsyhWhqQ3/JaPvmAR401CrMQtAUoogplZy4NSP5KrfQPIfikEtlqu6wn8Gq23XHzNGgi We9XtmnpYYK4tlGTo08Pe6wLW9j5XUX3ebNWKn0fv1o3gs5yNToDOhsR3bDNIXX7SfaF 2e9XNbn+SwbXzmniemg0s95HIfB8Is0Ek/RzTEdgppZPCgpTm/AzErYK6rSKai5gfgkT VRGPi/UYR83ESvpET/tpY3/B4iSeX+w/+5Bjke+zysM4pvt55ScDOILiTkdpdY1yc7bn sOOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+v5zmTqMHanAlbP0rCuGswlB7G9ERwBp467Jydp4J1s=; b=E6op6RUy/rBzSHqcVfwbR36/e2UmRxqaZBxLa7xhH8bpvd0coKm0HWqTxz6UURGkHX mEeu6S1A8LzPbf+jZ7pwX77YqOIKmnRO99ourxoeMsFv3jqSdRB9zA53ABa7gH/yZNmU t3XiR5anfeuuGjdScUn7xaOPNmH7iTgII8dRZCRySuuNOQIsaSPezIdk3caS6mlMkPgY QSL1R7dvU+qB3tIvcWzOr3l0SBHs3NjDq5os1woXuXI/3Br5D7qoc4nqzeta8P8AN9uz GnS/Xr4V7vDXhINDloMn2tOAo/m0rEBW8u5brvcXjr7oaljaH4ra6yOE9kfXqtfTJe2j pL9Q== X-Gm-Message-State: AOAM533mJVeFHLIx01mifwe7t+7Csgy9+n3uH+CevW0SGly3bqiaUBQN C6GWJqWrl3bCBT8JCw5RC9lRtw== X-Received: by 2002:adf:f842:: with SMTP id d2mr56855091wrq.55.1594106063017; Tue, 07 Jul 2020 00:14:23 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id e17sm26417761wrr.88.2020.07.07.00.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 9DD1F1FFB8; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 31/41] linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va Date: Tue, 7 Jul 2020 08:08:48 +0100 Message-Id: <20200707070858.6622-32-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::434; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Riku Voipio , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Laurent Vivier Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Given we assert the requested address matches what we asked we should also make that clear in the mmap flags. Otherwise we see failures in the GitLab environment for some currently unknown but allowable reason. We use MAP_FIXED_NOREPLACE if we can so we don't just clobber an existing mapping. Also include the strerror string for a bit more info on failure. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-34-alex.bennee@linaro.org> -- 2.20.1 diff --git a/linux-user/elfload.c b/linux-user/elfload.c index b5cb21384a1c..7e7f642332dc 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -2294,7 +2294,7 @@ static void pgb_dynamic(const char *image_name, long align) static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr, abi_ulong guest_hiaddr, long align) { - const int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE; + int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE; void *addr, *test; if (guest_hiaddr > reserved_va) { @@ -2307,15 +2307,19 @@ static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr, /* Widen the "image" to the entire reserved address space. */ pgb_static(image_name, 0, reserved_va, align); +#ifdef MAP_FIXED_NOREPLACE + flags |= MAP_FIXED_NOREPLACE; +#endif + /* Reserve the memory on the host. */ assert(guest_base != 0); test = g2h(0); addr = mmap(test, reserved_va, PROT_NONE, flags, -1, 0); if (addr == MAP_FAILED) { error_report("Unable to reserve 0x%lx bytes of virtual address " - "space for use as guest address space (check your " + "space (%s) for use as guest address space (check your " "virtual memory ulimit setting or reserve less " - "using -R option)", reserved_va); + "using -R option)", reserved_va, strerror(errno)); exit(EXIT_FAILURE); } assert(addr == test); From patchwork Tue Jul 7 07:08:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234936 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp659117ilg; Tue, 7 Jul 2020 00:21:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxj4FZ7okgcV/WUmvwFDk+MpKeqX2KFi+CJPsFpRgVjDxBTlEGgh2Pmly8/aUtaffol4lsL X-Received: by 2002:a25:3f87:: with SMTP id m129mr89601752yba.371.1594106518113; Tue, 07 Jul 2020 00:21:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106518; cv=none; d=google.com; s=arc-20160816; b=Ryp+7Ql3/rHhOS7tkr0sMw9CDOA0uTwgEASKx09yMcxmuUXtsRVuwNk1VRU6wND3WI qjkKTA2Wb9QNrxeNUWzHLBUpaoOxT1nShTC2ma/Jivp4Nrrb3GXoI0WLMBOzUigUt7yo x3ek4FuIfzwZJKOuaAn9ITC9FPKMkBrjXVYINlBcpTCMTkXtw0FAxmzeovV1FLJeCZYO PGJ83VtOyYKMMY9LZ1riU+VWL2OI+j0Hhu4OdqeyqnJWK8G9SVpCVQ43duy5wKMqrI+A u0hXMEEbZY5FaN4N3DvMTFS+zD6ASbPNlV7/gdtNl18haRJKGRu1vWSv1rfD6u9CXtqT y2gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=zR5BJqrW2UupwJs+QSJfBgbca4bCAOfpAid0iolRfp8=; b=k6VeD73r/aA1y8vWZu4+8m2GOvraQOuW/aWhZFqs0usqmWf6dxXRLevAtUprtRfw+/ jsc+BL+JGg4giMdEpauyKGcWPIw8XOPpeok5q/EGk8E5cJEDFfC7q0I/eISTWT3ka91M bif4w1kYbkVeq58v/p+9R7oXL6GfNp9l0cSrEw3iTdsINcIweyI2B8wtViWJPloWE4Gj SC+Pb73fRe3s8KHtauTPsfadUgWCnxPyPtEWTvLZWpKwM7p/O7WIzD5+yCpb3EaQ/obu miO+rPasPXWWIbzJ0EZ60wbLyovNdqxNUxGyma4nC0aqfAMDkGVA9vKDFIPzyNYOx4vF D0Ow== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E40GmlbH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 12si198485ybf.410.2020.07.07.00.21.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:21:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=E40GmlbH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshvR-0004kG-Ja for patch@linaro.org; Tue, 07 Jul 2020 03:21:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43822) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoU-00076I-1e for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:46 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:38392) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoS-0004vB-C8 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:45 -0400 Received: by mail-wm1-x330.google.com with SMTP id f18so45349343wml.3 for ; Tue, 07 Jul 2020 00:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zR5BJqrW2UupwJs+QSJfBgbca4bCAOfpAid0iolRfp8=; b=E40GmlbH5NU8U5ZP+dpwlPsLKoqCj+kM280DWC/BvC0BCvQmZckY6SiD1nX2RTLkGh uQ8Bzp1t8BQBkMcaCElhRpcN1apmOr5kD6dZrIj1PAeOWbYQBBhd/QS+z+oylJc8N1xf Mt09FGGt4PwNQGn8GnY1/LONPGfu+BRjUJ0NFA4Z71h/RykUHUZN5ZPm8Txv326ZXKbJ MuA04qaPdVudbgIEdWC+XY4HwLrXZkA5/NL2pPuHDTug62QFeS3VFfeiOa+T1jyj0hmc 9qA/nlXfqzFVTIGY+3ylqzzJN+CtuC1Vg4tqjFXZ19O+HqpPvjOBx2Zt8YwZUEfjwyQQ UiGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zR5BJqrW2UupwJs+QSJfBgbca4bCAOfpAid0iolRfp8=; b=IxjPE/rFFOgwqarXaHqanM7Lls9/2zcA5M/TuQcJKNHNVjswdA7T6lI3mewGNDzf0A q/BKc1LbWfvC6U9EE7VaMQsDlwnEPXF82QF20bAVobmBt7g7xMenhISCADMuPKFQMIEc uvcYFiy54A1ZvZshyfrwppgv9k2RZMEyodshAodrC6PjvhGe4IYv3ytsWCm/0sdH393q AQ1XaLEALnI2bloYucctqQyvOHipkjYw76jvYuf46tKR8Ai7yiBFM+Y/zeBxaaGNqJWF o3FOWerIbdQOAgVLjwY42XTXe5bBdfSKwDMBK2PyjQDk9Vcq4TDj5Zyt4k4yZPr8Q1lz AC8Q== X-Gm-Message-State: AOAM530tPdNuL3DydejciuyfSHBZ9CtJ3fHdpo+aiaA3jrrY8hXAmgHA woMdFecZSxSJHFq+T7pAQXrgsg== X-Received: by 2002:a1c:19c5:: with SMTP id 188mr2627522wmz.124.1594106081675; Tue, 07 Jul 2020 00:14:41 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id e23sm2106022wme.35.2020.07.07.00.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:32 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id B3DF01FFBA; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 32/41] gitlab: enable check-tcg for linux-user tests Date: Tue, 7 Jul 2020 08:08:49 +0100 Message-Id: <20200707070858.6622-33-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x330.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Switch to building in the new debian-all-test-cross image which has most of the cross compilers inline. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-Id: <20200701135652.1366-35-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 93baf98232d7..257947853b72 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -143,10 +143,9 @@ build-tcg-disabled: build-user: <<: *native_build_job_definition variables: - IMAGE: ubuntu2004 - CONFIGURE_ARGS: --disable-system --disable-guest-agent - --disable-capstone --disable-slirp --disable-fdt - MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + IMAGE: debian-all-test-cross + CONFIGURE_ARGS: --disable-tools --disable-system + MAKE_CHECK_ARGS: check-tcg build-clang: <<: *native_build_job_definition From patchwork Tue Jul 7 07:08:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234948 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp663310ilg; Tue, 7 Jul 2020 00:29:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJynRWG/LvhaV55+h9McFFdaSeRsMnwAyh+BBkR/ON3e5RFJFY2ITUA3tTe3fj2aYtohhh4O X-Received: by 2002:a25:698a:: with SMTP id e132mr15343328ybc.177.1594106984789; Tue, 07 Jul 2020 00:29:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106984; cv=none; d=google.com; s=arc-20160816; b=SNU91pvWg+CGBIurGp5/5O/iMTKMyFHzs87eKUOVotsL124Qw/kHeCYpRruYFL2rDv wVqUmq4M7X1O5JiiFJkNQRsuA0VG8SKfz509IcPXoeZcLIEBc7MQ31GNjNV6CFsZXr/8 NVTCmQk9Au4SDY24n5RcjytxR3BIGuqvmj3eNOj9tILjNjz3meikVKXj09V5t0Dv2MCV QmAW3YN4gm+ijcqK8HzsiuQh3vMX7hCL0jw6p1u7a3QS0boeNGUCOTEH1oJy2c7rkznB EeyyvSTo2Lfojcg+STjz8IOwarWCC53IYcbCOy3yyrN39y+d/rCrqwPCirIdcwiAn7C4 JjfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gQ8vOrn8UzVSlnR8O9dGfbIrh6XgeWKqH3C6tmBlZYU=; b=LHL5SbS/pI5v840AClAI2zUBMqPBmLkC2FiSmkJdmneDXQ+We6i46BXkHTzla4BwEC dBYj1RLXU1hseXHqHfjp3e/FHBvGj7rTI3+sxA1pcKqrEfmf/XhNxlNcdnGotoe+rOa3 146pR9lti8UqavOPvUQDl9gS/NsESVLcel8YzM3T0dB4dbHjXoUGBX1dW9Ziuezg5Wgj tk86Y5VDBcSUiLt7OwIASPO2j8rMWvnK/+99kquJu2c3tEoRsHjpJjGyn3JhGiupz1S/ vmLhGrRzrkig6A5rc5WDpkZQ20iJwpSCj3jySlgRh9L/u0omVc+Y9EMGRgD0njxw/UHt RUTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=K07VAVkb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z18si9986621ybk.41.2020.07.07.00.29.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:29:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=K07VAVkb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38792 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi2y-00004v-9u for patch@linaro.org; Tue, 07 Jul 2020 03:29:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45908) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxs-0001Wc-Si for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:28 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:43983) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxr-0006gH-47 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:28 -0400 Received: by mail-wr1-x433.google.com with SMTP id j4so41598173wrp.10 for ; Tue, 07 Jul 2020 00:24:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gQ8vOrn8UzVSlnR8O9dGfbIrh6XgeWKqH3C6tmBlZYU=; b=K07VAVkb1kwsCPj34I4BuKcUO9oMG355TR8agcZ23u2cc+D1juVTfgXCJl4PTniMcz rfa09IAKcNu1eWNRHGJUcBaWp2TTdDQs16B/POtNxO07WHuMPP+SFEdXc2LgbqevS+/n VA51b+78HrrRFmLCt5EyFHce03XRYI+Jl+nyKJyZwiAOhaPC6YBE6kWDos1kQV6ZY4Md Pmcv5uE4M1d4rJqWIiUwLCP1o9xOOdjHZbnjTHUdxTZlTQ+wMA5yQvwJTojQokX4Zel8 vfHy4H/RCPSDH68Pzfh2Q2wkduNNLfJ8tuMNkBmhv1bcFQI0E4h9FmEasYvoEft180+W nGWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gQ8vOrn8UzVSlnR8O9dGfbIrh6XgeWKqH3C6tmBlZYU=; b=iJP6EUi6KxDqwwPd60K2YFV/bsqeDKwM2pOs4lXDFKit4wT/EevTO6LpvOX5Lw9b2T xRgWKD7MNcSFc4Z9mdS2hOWLAe+Fb6jOgaAneKRWbO5r2QJYhozWBtMAhXbVrvrq3qoN 4i81r08ayktXmoDnA/0PVyFX4xzNDeKa9B4gzOa9aLVqQXcIDGLl6VZAJBwlCRZna4f5 9BfFC8JqX+8qnN/Pj/IOgJkI4hr0hP2KSZ88DZ2jL0NUzWpsKA0pQZkb0M0qfwnMtGPJ tlzudCfmQUNk0myaW+zVqUrT+D76GMhD9uZzcwZ4eMaDb0kC1SkA8t/nROUm7tHKT9UF KciQ== X-Gm-Message-State: AOAM530g/ZFgo4vo+mJRzjGSKxSkZHUIfds5UESwF4w9GqhOyKCFVaXb ukK91gvzbxEhK943tkIHiXD84F6+W/A= X-Received: by 2002:adf:e546:: with SMTP id z6mr48953240wrm.99.1594106665660; Tue, 07 Jul 2020 00:24:25 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 92sm29119733wrr.96.2020.07.07.00.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id C8C071FFBB; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 33/41] gitlab: add avocado asset caching Date: Tue, 7 Jul 2020 08:08:50 +0100 Message-Id: <20200707070858.6622-34-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" These can be quite big so lets cache them. I couldn't find any nots on ccache in the gitlab docs so I've just ignored it for now. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-36-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 257947853b72..13e9531724d2 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -8,6 +8,12 @@ stages: - build - test +# We assume GitLab has it's own caching set up for RPM/APT repositories so we +# just take care of avocado assets here. +cache: + paths: + - $HOME/avocado/data/cache + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' From patchwork Tue Jul 7 07:08:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234945 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp661951ilg; Tue, 7 Jul 2020 00:27:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzsUiaOfwMQfSftxYL6gfTvvISXtO+eng5bsEuWUjUq9plmIIRlBgsp7gevb73ULQ3nxXnt X-Received: by 2002:a25:b4e:: with SMTP id 75mr77495008ybl.198.1594106823882; Tue, 07 Jul 2020 00:27:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106823; cv=none; d=google.com; s=arc-20160816; b=rbOJWtDdSVXuZ3PUcled/Ng154RG7M/ud3nbq7EFfaLMTeq+Q/UDnwqYh5Pmk4gFW6 hFf5Jqlw5O5FlUMtAjVKATCh03se43Je7yaTSetw5qje6Lra+UUGfllhdNYRY369XIy5 FsCT/Sp0QMeEv8t32vFs5mIEkpsd/ybtcDXzgIHNPanrP5Vj/KYQUniOUjfzIdRjweYk ncx4tE/KGOf0DZdcNcLrb5LYYUDSa8LD8Wmc+lXJFF/QdZEcEXv2L6ty7ShQf9gN9jaK CRDA7tjam2d/pYexnV+8KbDj+R1yp83AN3W7uUoYCSDkdWyJ2KGtGogMuWCWHfpBQHRe 8SAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nOeMTB5/6ChZR/18ZGCbrQXiHpw625FcDhLOCmlCX1o=; b=Gy6Id4NvnpZbepo6joQxGyyjea4bWZ6AsE6QRjE0AH5ewCxjM10l+UOkyZQtOm6upA FiDyA6Nmoh09SgFMP+JjsfW6Cc5mdMCUMabvy1jLt9l+YF4RG27VM0rQLjyCrPM++sT3 fMAx3GbkcyDtCsnJvTL/yRH5780cMPMZ5xnH5EDLrdBLqw6lZB6J0ldVqw+eRxpLfq8E hpV+HgtzJDFVPawzax0+Mfc29KoYDKjRJKwMq/6naHqjI0UaSGJsVDQ17DrSHOOkkYw2 I9gtAS33PZHrZ9DQJN3TiLnLZIRln1rLTYTEwdkdA9loE4onfcJb7BX8IScOHqnnHrmN YwzQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yAiPPEbX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i14si22723876ybk.252.2020.07.07.00.27.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:27:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yAiPPEbX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58364 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi0N-0004ux-CO for patch@linaro.org; Tue, 07 Jul 2020 03:27:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoM-0006rs-2u for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:38 -0400 Received: from mail-wr1-x436.google.com ([2a00:1450:4864:20::436]:43911) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoK-0004s1-GC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:37 -0400 Received: by mail-wr1-x436.google.com with SMTP id j4so41571838wrp.10 for ; Tue, 07 Jul 2020 00:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nOeMTB5/6ChZR/18ZGCbrQXiHpw625FcDhLOCmlCX1o=; b=yAiPPEbX2CYDYS2BhR0qKxR2ADauDnMYTz56i0ATRL5hQii9XZi3rt0tf+pg0w/k6K wuQGw7dBTQaI4VMLCFFtMGRjB19BFxt4SvA7NPpriXcdh3ZRmSrjWVsHs2pbi/S9k6Br QyHEfdKRGBDnEUNEkGP21mdKfZu6PLWHzRP5RAAa2fnTQvkE3HGBTgQGqzm5lN2p5/mc /q+Wk5cauAlToqqSu+mnB3Ng653PWX5I+DCyZuv/b8bfQWc9jmfWYYuwEiuVjUHEueM9 g7pvWljeVx9y6Vipdp+dx9OwORwfxplVTF9a+6II6YjQfTERGWVYpG6RM04JaggtEykt chFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nOeMTB5/6ChZR/18ZGCbrQXiHpw625FcDhLOCmlCX1o=; b=THiZ3SCjzMIZH98Cvxvz57XzMGiga5MLkJv3Xj8oaSFJR6oXM2jlaK7PRolx/wkW9O XRp8Vz8T4vAfZF1ROGXlN4OBSRpudXGxsxLrTTBmFMzvdbHI4tSTDYbTjq19Oqe0jSGf Dpfrhctm7NcpSLChUvzVPxuPi+bvWioXit0huqwhMqn1uVk8AIL0+iLt4CKXAqRhUPBg XPjwvjkj8B5AiFAjiUd4OmIr5AJkh08KOjxT2XI05fzubpwz/L1s3COycU2gYRQdsB5Z jtmkR3zexsRfT2dj9oWIY5PRcjyi4amIRuaxoDSJrnls5mzYHocSw9EZcTfretg93zo5 F+ww== X-Gm-Message-State: AOAM532f4VIX58k1QjPbNmjNQNh0Ks/2qzG0QjxSxkN2ckuWxIVLUM73 nn3BiDUFpAnqIKBVfBViKe6sMg== X-Received: by 2002:a5d:4b44:: with SMTP id w4mr52893767wrs.275.1594106075180; Tue, 07 Jul 2020 00:14:35 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id h5sm3076543wrc.97.2020.07.07.00.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:28 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id DED0B1FF8F; Tue, 7 Jul 2020 08:09:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 34/41] gitlab: split build-disabled into two phases Date: Tue, 7 Jul 2020 08:08:51 +0100 Message-Id: <20200707070858.6622-35-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x436.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" As we run check-qtest in "SLOW" mode this can timeout so split into two jobs. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-Id: <20200701135652.1366-37-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 13e9531724d2..83e38ea19f3d 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -113,7 +113,7 @@ acceptance-system-fedora-alt: MAKE_CHECK_ARGS: check-acceptance <<: *post_acceptance -build-disabled: +build-system-fedora-disabled: <<: *native_build_job_definition variables: IMAGE: fedora @@ -124,6 +124,17 @@ build-disabled: --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + artifacts: + paths: + - build + +qtest-system-fedora-disabled: + <<: *native_test_job_definition + needs: + - job: build-system-fedora-disabled + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check-qtest SPEED=slow build-tcg-disabled: From patchwork Tue Jul 7 07:08:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234929 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp657660ilg; Tue, 7 Jul 2020 00:19:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwU1NRFa08qjgJ7JDKUoPiPS/whehDz/3Yjs1ByrxG6PxxCnUECp9oQXpcNPC32nZLV4Gh1 X-Received: by 2002:a25:86c7:: with SMTP id y7mr88214645ybm.69.1594106354662; Tue, 07 Jul 2020 00:19:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106354; cv=none; d=google.com; s=arc-20160816; b=yU/2G5k5tsmAFv6L4NY4OyQbyDPJs3M4Ixwr/SbHNpCpMEpjyaJmQdLxQ0ArS2ijkF BgIPfPNy68tamBXIobkT1I5r97z5eBYp68Z/TfK0N+6shvopFikJZqda2eXLGdsu8EpQ MQd4r6lHuIsSBR7Dz5iy3nNbsmrMx2+6IE+VRw2deAIb3KSUwDZ+3YpWlaZgoVNpf1ou Eb5ejYzEZUI2iGBjRk+DujetumIcgvdd1GsKXFTzrP5IIh9TfozCNvyFWhuIvE8iZwuq ZWMN5M+RNHGew0501Hf3BuHyM3G4PeD4jS1KtxOAnDBN6fgGBXfd6DDVfHacMtNTpy0H 0jWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=r0+udXPK/W1B2LpwudPO4KcKX+aKxd6kiztObpKjw3A=; b=CkQXU+gSOSp8/NXopvxqVcvaMKwUPEoA1t4w83WHYpSmlCFR+ni4cJmnsX4FipwPXi BnS017w/PuEGsJ4lWBRzYw/jBpv2s8ALAyaeTcRyMWCPpxDMbfXwmVJ8NT4y69tzmn54 8Z9LvHgO6+TwG4y6lklx2kGkHKLhLbDDaQ6Gr5NQwvfy6BWSXoOTK/JQRkvNwiule47L /TgP10YgiwjEAIdYoXna4CmS3zVxqjUXWvqtEpl1QAbO3PI0iB0ipstecCsdcoDA2CHn LLMKl2zwlx+r10YE42ztBou0YoKQrcKcynQO6LkI2DKcXVPMYwnulUnK335A5kZPB9TH cDmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=TrVD5FCI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v137si22528327ybv.363.2020.07.07.00.19.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:19:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=TrVD5FCI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49548 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshso-0006I0-3d for patch@linaro.org; Tue, 07 Jul 2020 03:19:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43726) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoO-0006wh-2d for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:40 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:44850) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoM-0004sI-C2 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:39 -0400 Received: by mail-wr1-x433.google.com with SMTP id b6so43966027wrs.11 for ; Tue, 07 Jul 2020 00:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r0+udXPK/W1B2LpwudPO4KcKX+aKxd6kiztObpKjw3A=; b=TrVD5FCIn+R7KgPwC2EB7cqF+WuogHvYLcpumTT2l9ORlm+ODXMFLjP9MOaBjuNgUe 6cnHA7Di5ONDcok3NygznaPNrqI2sqsTxSB3ycUrspykZXPHjjBdGcXSxFgGd3+vybRV Y+fkSW272VqUZBAdn/mwdvaZAxkEJxyujfb7Q1Hj6HbrHsGrrtEAGkJ4sQtEYjyoe/y9 /Qf/FhHuKT4eU/LtZSRzWevAaQLd9M7DYq37N968bAMVDTGz+CegADC15XAHKFr1bS7Z tgUaIP0yJXBYRbwV/zUlY0+DVqlqOmTsUV20FnR16I0/pzUkCvEPvCy4ImhlCQcaUxy7 6uGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r0+udXPK/W1B2LpwudPO4KcKX+aKxd6kiztObpKjw3A=; b=YuvauydGdQvm/jCr8ec8UgTCehCr5e8wX0vrdBR/V9xAc1+7d0W/vGhNqNjWI2Mj92 lwpF8Axo/bt7rQxqa9KR6UqPos2VtX2qiyeRL18VvSbmR+h1OkkyKnQpCc+rQUD+ML39 Vm32STI2sfYTa1Q3X7tb2xM2Qa1X0irKY+5CeVhYs5H6QoFJUzhnB3V89okzhSygFZxe RWoK5fG/NDrr0UwNXwOI1skPaJ25WXTAZ7wiCPEIeE0a21VBvrZMWUEtXS7MY5nQeO1E hAXVgVgow8uiktg57W2ty8lVeZcqKn7gnGpUh4ro4tgoq1t0DUBm3gBKz6yheJlJZIwU CZzA== X-Gm-Message-State: AOAM532QnTAACJ8SFAZO6n7mS4dEqrGztk2iRc4XDGC2D2JoTK7bOCh4 g/cDBt38TFo/NH6dCe7xo8/VSw== X-Received: by 2002:a5d:6ac7:: with SMTP id u7mr25600805wrw.25.1594106076960; Tue, 07 Jul 2020 00:14:36 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id y6sm2398452wmy.0.2020.07.07.00.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:28 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 0229D1FFBC; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 35/41] gitlab: limit re-builds of the containers Date: Tue, 7 Jul 2020 08:08:52 +0100 Message-Id: <20200707070858.6622-36-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Most of the time we are just rebuilding the same things. We can skip this although currently there is no mechanism for picking up new distro releases. Rather than try to be too fine grained allow any change to trigger all the images being rebuilt. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index a7621c4204c7..ea350eacff70 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -17,6 +17,12 @@ - docker push "$TAG" after_script: - docker logout + rules: + - changes: + - .gitlab-ci.d/containers.yml + - tests/docker/* + - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH' + - if: '$CI_COMMIT_REF_NAME == "testing/next"' amd64-centos7-container: <<: *container_job_definition From patchwork Tue Jul 7 07:08:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234935 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp658878ilg; Tue, 7 Jul 2020 00:21:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxedsbWNX7o5T53fQb1I55K/DW8f79JNj+8ZgneBgOYFZlWDOg98UhKc+P2+kxsM2j2yGBX X-Received: by 2002:a25:d807:: with SMTP id p7mr32023048ybg.229.1594106492193; Tue, 07 Jul 2020 00:21:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106492; cv=none; d=google.com; s=arc-20160816; b=kLSR5S1xs2Q3qVMRlOn1OGVdP8SVE30t4skvnntGrbKt5VoXPt0hVKk9Z/NqZXLYir mV53/ylU2d18pxCEMqnOq8K+qx116IRD/NtoG8DpWcH8V+/7W5lkt0ZWJ8qNjsK6hWiH lJ1j/8eFRO1b9IiZgj5Xhl0mFSFi/lCzjph5JoTn3FBWZj8z2I6QyGfZCG2RQ08U+6BB 03G2D8TfH+JWriq4V6EdNfrydSLwpNnAfxgyGGEMKx/MA66LJQGTCQvfYZ3WQaZhe9L6 jvW3Rt9kK7TuMTHqmnBYXkELUAWwli7GgAhxuy9WYGcqkk40vxyjcO+BYefD05N7mSfz DAgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ghYwsDv8y9AcQ/2pLkPngqM/jomzSL1uaEsogMN8yjA=; b=OXzy6eqFFyC9PJw5a8XmUuy35dJ8GtL0gRELxpyrOer82b5/jmOzLtlzsAxDRdidVK L+oCdDzcORze/hMret0H/4RVEzTdC9YO+xaJD7SqDR2GI6lzSvf3uhSfsFH6gElOcreQ AQiiqF2epUz0gqDj3MVNv3akSEoxhl/IzwtraEplhAa0iOJLRkez4hLP+K3bQCQFUEIn RZWbkJ8NnA4waLAFpTjq2pYHw5S3O1/ILLDHEEj1vBJA3vSXjl3uYJv8hRAVpLMIOpXh seWRMLNSb8NEAZdZY8xDelm41b3MRU2MylFb6I+M+qBKZZgPUWCpJXaJkm4pF7O1elwE CddQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ZwAaBEp2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h126si11147630ybh.84.2020.07.07.00.21.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:21:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ZwAaBEp2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshv1-0003is-K4 for patch@linaro.org; Tue, 07 Jul 2020 03:21:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43640) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoJ-0006lH-C1 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:35 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]:54862) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoH-0004rU-K3 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:35 -0400 Received: by mail-wm1-x32c.google.com with SMTP id o8so42116085wmh.4 for ; Tue, 07 Jul 2020 00:14:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ghYwsDv8y9AcQ/2pLkPngqM/jomzSL1uaEsogMN8yjA=; b=ZwAaBEp2m32zYdEjsxYfPLNW+Rr4BeBFvs4P9zd7IRvhyAh388q1dePAcoZFUMxnYr v0f5od3tplSIgYsWsQlIFSKTLrXSdBgOJIC4LDLizUik+nJTHMGYx4XWOOTq49hqWR24 Fhe85TwFNBHz3MSy+zDJV7lMX+O+iFJ6ny+KUd+lDvO/XjU7KTphv5yea2Px8tUDdqHy sqFzZgQfNQyA3hepmm4778QSkHkQ1ak9Gu8HPn33rDyZJ+wZt1rHoStZzQtwBdLSlNRr oGEnRfToO2Mtv1d0T/Anlme3EhBPgcr3VEU8BcX6OgcOd9d8mLk2CTI+BwW7XyyYa788 DENQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghYwsDv8y9AcQ/2pLkPngqM/jomzSL1uaEsogMN8yjA=; b=dx45DmGlFpFiRfipm9diaKXy7jSVzAyyPC7jvvTXS8p865MZN8cgVIblV3wNZun2e8 irBQqspXzKxzktTnQ2KKKVvTQk9OEfnOqBAl9mFKsUIMY/ZgQroxNefIppcyOjfNo6bo sEvOp3BScAN54NSsl4USq7qHBMzdjbAqA8Sw5GtOpCGD6oaqDgoQjOW37acW05DdpOfz CJYDJd5Wdf7f/ZzX5stZbY9lyexqo43xPZgAYA+1/vA+TPTPZ6gKFs/BjjNQUehbEUV9 pmITcA5L078OhrpnxLwdLvzUnSZCBnzp+y87woHlg8xhO0uL+lpv1DIIzFs7coE2ZKpI PF8g== X-Gm-Message-State: AOAM531qETlSVItGvCOwwdY0s3u3g2swId5zgmxhLJYO4XmeKNj0UBhn GF8FXomZFk2EpFJYzKbwpIFTCQ== X-Received: by 2002:a1c:1fd1:: with SMTP id f200mr2592718wmf.162.1594106072232; Tue, 07 Jul 2020 00:14:32 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id g13sm28007940wro.84.2020.07.07.00.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:27 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 15B181FFBD; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 36/41] containers.yml: build with docker.py tooling Date: Tue, 7 Jul 2020 08:08:53 +0100 Message-Id: <20200707070858.6622-37-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index ea350eacff70..b1e39cfd9ac1 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -6,14 +6,17 @@ before_script: - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - apk add python3 - docker info - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" script: + - echo "TAG:$TAG" + - echo "COMMON_TAG:$COMMON_TAG" - docker pull "$TAG" || docker pull "$COMMON_TAG" || true - - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker - - DOCKER_BUILDKIT=1 docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" - --build-arg BUILDKIT_INLINE_CACHE=1 - -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - ./tests/docker/docker.py --engine docker build + -t "qemu:$NAME" -f "tests/docker/dockerfiles/$NAME.docker" + -r $CI_REGISTRY_IMAGE + - docker tag "qemu:$NAME" "$TAG" - docker push "$TAG" after_script: - docker logout From patchwork Tue Jul 7 07:08:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234949 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp663918ilg; Tue, 7 Jul 2020 00:30:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyTuki9tpmAkTyWs9J+7WcLnYMl4kURh8X65LCBAjbDdn5Xc0FpdEiuJ2Azum1YwMja+qwb X-Received: by 2002:a25:c805:: with SMTP id y5mr30610498ybf.204.1594107040384; Tue, 07 Jul 2020 00:30:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594107040; cv=none; d=google.com; s=arc-20160816; b=uklS4ufwG1SOS7TaOCReD30vUM+UW3UUoEmSyDBI3W0Fc3gtH4loJ5NXrWbmHLn6Bb KB5xM07hRerWTvHymsLFgvSMxrq3ZGqe9KH/gNu2hwzsDQfmuS/+sKDKCsi0ywkR3svL DBc4DjhNexXhGe2eIG6LiUgUFy1iyjyNuF9zMTRM8Fx0d80eTzfNdyrLG8wBpV2F6JP6 4TkEuTkjmBx+eArDKVuJsdNHIEmPfOKlaYBLxSR8EeBe13z7sb5IFVxznmLwhLiqgnQQ YVbh2AZ2pCYihlBQaFtra0KuhJIBz7CdELs94X354ZkSUtlC67l8yGrYD0w0qK4bPiK3 NnsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9byPofvUmGL/7sYy1T9aOuk75YT7bXG4EnLPH7NVVfk=; b=dIsYVkulFUPThpR9gimpAaTu+V6bB2rdNGK/AoHwfb86qrBl/C4cHmkQ0vHPMFWVq3 IPovWto6s6XP8wk0riC4cJhFCU9PJFt3h3JAZU/oghXXE9MZYFyw4wngiTPuw+SxnE/H erxPxPXdwZLiNf0wPOIADGbHyw6amIal5u1e/15DRkt1DTnq6pecLZsEj8sznTZZ5klj fnYNsOB8ADPck9m0T4x/KSf50MIa6+gAznyr7nZnmeRXClHU1N3NWRvZXkPBIAbXP37C Yimckyhbrlc/94HpQBMj5Psp1h11uQT5kOIy25XUEnbHOdL8PF9Aovarm2XmGgXrzBH0 +0TQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XmVYKRqf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n8si13967867ybb.6.2020.07.07.00.30.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:30:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XmVYKRqf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsi3r-0002Re-RH for patch@linaro.org; Tue, 07 Jul 2020 03:30:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45966) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshxw-0001gq-So for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:32 -0400 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]:37243) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshxv-0006hX-1K for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:24:32 -0400 Received: by mail-wm1-x32e.google.com with SMTP id o2so45353069wmh.2 for ; Tue, 07 Jul 2020 00:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9byPofvUmGL/7sYy1T9aOuk75YT7bXG4EnLPH7NVVfk=; b=XmVYKRqfcRhuzuspcfM/XBV0n8dIDh9HipAQyYLUW1AXX4ATrKaskTzLkU0nQISf6L xHvqxtodYQDnnA7rU4EUXN4smvRkrBgOi6++goUgglCGg/KVovPX9WltL8pEeQxQhRFy CffZWt4rd8mC5jofSAb+EnYHI0jA3wCDQIjVcyiPC9BN6NZ+5dH0Z+sNFJohXbhgP65u DmoVGRQ7vILfLM/gKjgFRlBnblzAdvtHu9lvN1Qt3/o6uPi5FIksPB3nOkSud5jSbqhT 09pR07LPh8lGKrkHPIiKvS93uXaXWhrwkdizNrgczJh4aTEolbQY+rV0mQE5mHrhfhlu V11w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9byPofvUmGL/7sYy1T9aOuk75YT7bXG4EnLPH7NVVfk=; b=Ikzp3JnehjkfWNY52frI9vQlvahlawwWFbaBSTHu/7QWCsMV3mQcoHbuL+7ifLsxUv PZ2xygSUaZUrbOCbZCU4KpSsmLxiTB6U1TPOxpHyJp3wZZmYs7MlRDMYmGl4nAYFODpp lUFGbkFNxnaKNETr0ZIgJdAIOI82XoRfkQ6nH5FNhMEtJgzZ9mtCneka92LzeB1LC9HG hc0eT041bbblTm0Toofu+EE0dPkSy9yrvammzrL2X2arUDs/ivMAKZ9afLvAkgnGUgjq rBzFH2aLSCetYVb/mmzOAJsnmaqCpAcD0XYkyVo4DuR4olLr/x/hMNwAeKJitTQgWcgB eKsA== X-Gm-Message-State: AOAM533N0Y1G1OSFXvMo80v7NUcTANs6nr6qDUF0XhbA1AN4Vjxz1bWB j9TAf0/CmG4q+yHQJcaRVczefQ== X-Received: by 2002:a1c:96ce:: with SMTP id y197mr2649506wmd.86.1594106669604; Tue, 07 Jul 2020 00:24:29 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id q3sm2154570wmq.22.2020.07.07.00.24.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:24:27 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2B77B1FFBE; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 37/41] testing: add check-build target Date: Tue, 7 Jul 2020 08:08:54 +0100 Message-Id: <20200707070858.6622-38-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32e; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe?= =?utf-8?q?_Mathieu-Daud=C3=A9?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" If we want to continue to split build and check phase it seems like a good idea to allow building of the tests during our multi-threaded build phase. Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Cc: Daniel P. Berrangé Message-Id: <20200701135652.1366-40-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 83e38ea19f3d..b0e5417146bf 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -36,7 +36,7 @@ include: - make -j"$JOBS" - if test -n "$MAKE_CHECK_ARGS"; then - make $MAKE_CHECK_ARGS ; + make -j"$JOBS" $MAKE_CHECK_ARGS ; fi .native_test_job_template: &native_test_job_definition @@ -60,6 +60,7 @@ build-system-ubuntu-main: TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu + MAKE_CHECK_ARGS: check-build artifacts: paths: - build @@ -90,6 +91,7 @@ build-system-fedora-alt: TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu + MAKE_CHECK_ARGS: check-build artifacts: paths: - build @@ -124,6 +126,7 @@ build-system-fedora-disabled: --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + MAKE_CHECK_ARGS: check-build artifacts: paths: - build diff --git a/tests/Makefile.include b/tests/Makefile.include index c316e0d66475..94b1cc8302ed 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -22,6 +22,8 @@ endif @echo " $(MAKE) check-venv Creates a Python venv for tests" @echo " $(MAKE) check-clean Clean the tests and related data" @echo + @echo "The following are useful for CI builds" + @echo " $(MAKE) check-build Build most test binaris" @echo " $(MAKE) get-vm-images Downloads all images used by acceptance tests, according to configured targets (~350 MB each, 1.5 GB max)" @echo @echo @@ -649,6 +651,10 @@ $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: %-softmmu/all $(c QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \ QTEST_QEMU_IMG=qemu-img$(EXESUF)) +build-qtest: $(patsubst %, %-softmmu/all, $(QTEST_TARGETS)) $(check-qtest-y) + +build-unit: $(check-unit-y) + check-unit: $(check-unit-y) $(call do_test_human, $^) @@ -680,7 +686,6 @@ check-report.tap: $(patsubst %,check-report-qtest-%.tap, $(QTEST_TARGETS)) check FP_TEST_BIN=$(BUILD_DIR)/tests/fp/fp-test # the build dir is created by configure -.PHONY: $(FP_TEST_BIN) $(FP_TEST_BIN): config-host.h $(test-util-obj-y) $(call quiet-command, \ $(MAKE) $(SUBDIR_MAKEFLAGS) -C $(dir $@) V="$(V)" $(notdir $@), \ @@ -814,9 +819,10 @@ check-softfloat-ops: $(SF_MATH_RULES) .PHONY: check-softfloat ifeq ($(CONFIG_TCG),y) -check-softfloat: check-softfloat-conv check-softfloat-compare check-softfloat-ops +build-softfloat: $(FP_TEST_BIN) +check-softfloat: build-softfloat check-softfloat-conv check-softfloat-compare check-softfloat-ops else -check-softfloat: +build-softfloat check-softfloat: $(call quiet-command, /bin/true, "FLOAT TEST", \ "SKIPPED for non-TCG builds") endif @@ -955,7 +961,8 @@ check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS)) ifeq ($(CONFIG_TOOLS),y) check-block: $(patsubst %,check-%, $(check-block-y)) endif -check: check-block check-qapi-schema check-unit check-softfloat check-qtest check-decodetree +check-build: build-unit build-softfloat build-qtest + check-clean: rm -rf $(check-unit-y) tests/*.o tests/*/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y:%=tests/qtest/%$(EXESUF))) $(check-qtest-generic-y:%=tests/qtest/%$(EXESUF))) @@ -963,6 +970,8 @@ check-clean: rm -f tests/qtest/dbus-vmstate1-gen-timestamp rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) +check: check-block check-qapi-schema check-unit check-softfloat check-qtest check-decodetree + clean: check-clean # Build the help program automatically From patchwork Tue Jul 7 07:08:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234932 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp658198ilg; Tue, 7 Jul 2020 00:20:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwruXSWi+6+dCL61Q4ZVxAkpEYznsf6y39Sr5mWShK1CctWysY9+jHNiqaafTq+HgWiLAJT X-Received: by 2002:a25:f204:: with SMTP id i4mr81262114ybe.486.1594106414137; Tue, 07 Jul 2020 00:20:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106414; cv=none; d=google.com; s=arc-20160816; b=XUahz5aOD6OBGTr8Z1y9BUQyi5XH5Vt9N96UXvG+ZoeOcGdwRN22M7aFsxTBS9HU2P FmU+eG6mtQtK+gd0d/9QvlvH3uVRcd7qtA16F0kkb7CAJMHXxdIln6tF/yxn8u7oztkK Er/HkqFZS70vo0GZ8qO70Xbr3lX1HQQxfrXVBvm4pBHOsOq+WPV2G7K+Gw3RfUimwL6D WOXIOpc7lxuaKFG0z7ijL4i8Nuy7QoRY0FIdUKzS3q3xQr+c2KE7199nJXamDsNaN8Ra kYOk/aE7AuP7GlAgSMV1ccnHMpkdwT1WnovpzaFpH8mLv1yZeXfasEUl51r+zkTgKdZu Y+qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9MdVCtyPDmbjXZc45hJi3iId1MWBt9w+N3ZPMWL/v5c=; b=iZRAcR6eag+UkevjNkRdD8vaCZ64T/nwXInfPXzngL8x1efGtP/G5Z3f2Yh0LZ4aci CxgSZu37FV4GNZCUgJBa7ZSKFP/J0Cty7Tbq9ep34Rjyl2pjUuh2Rjd3sNVPum1kEEI/ /k7pHSOH8rcCvrWBpjQgQGE/e2KehpmkZTp+TbK5s+yZybBCIYn4LPQ0LAEo3OB52HBy C9Yxwe4QX+hK/ZporRfi43vRZ+C+nF7hT7wK1aDhP+AcPc+Ts6L7BphIFM1ceTRyDIiy nw/LWB4jCv1oWv0KtCzLMuralTWYXPY+1RVNAiQfhmKiMyyi2ljGSFRmuQLYBvLzsIZ6 i7fA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ylihklsg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 203si7686216ybc.231.2020.07.07.00.20.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:20:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ylihklsg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57976 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshtl-0001EZ-JM for patch@linaro.org; Tue, 07 Jul 2020 03:20:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43798) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoS-00074k-Og for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:44 -0400 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:40254) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoP-0004tZ-RC for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:44 -0400 Received: by mail-wm1-x332.google.com with SMTP id f139so45346811wmf.5 for ; Tue, 07 Jul 2020 00:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9MdVCtyPDmbjXZc45hJi3iId1MWBt9w+N3ZPMWL/v5c=; b=Ylihklsg8U5au/iLzdzfzydGIxQsLx82Yf2S5BjEBQjGepiGuAxAo0itBYp5bVH1q7 /1bQVey+rEl4KoY6PFVSaekZs+6y87IVSiZVsyWGFrzQvk2gkQaiQjbs5dvpA9bUAc3o fHqgb8EAEWjz2Ux0Yn0Ds/pi+auYCrcSJx7CR0c9Xh2FtcpVMV1BKLbeggLuQ5p1w18G XAXBj4QJTCspomc1iYbk3oxyR7QgwQve+WZTRf8TgPWjFTpjzFj/yVHeNuTO+dVkBs2o nRewPlE1RULPw6qQVZcyu5N4r1AtNuGkw7rsYCvY3YzryqYM+sOYpsOT2Ms2K4CUiUek S11g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9MdVCtyPDmbjXZc45hJi3iId1MWBt9w+N3ZPMWL/v5c=; b=kTVne41FMj9e0Sh41zzbjLs+zmvF3rFB+nB+L15WAckkmNNb3l/U2mnNlfrGvduUJ4 MT3yQW2z7Ng1DGiiDJFrMWO8qSyImTlOpXz19k7EEnoKZly8NePT/yB9nLFKcKOccqgN Jevi4rsQ6BV1EQkfWIzZh14wgyG7VKj2MC/i5sjaE+e/lBvuNoANaoOVe3tmqdgz6nSq beD9+frsJgv21h7vNNsm1NVoUlF0Bd+5SCSA4b2LwJkg22M63AD/k/uR7zDqK+rxt2k6 iDpRxQ7MBpkUHeSuYDvoZOIeh2ICVWDpVNVvJlu4dJlPYndAGFz6v23FInh9Nw4JhjpR Lsvw== X-Gm-Message-State: AOAM530YcZ+WgofFNK9RQMt4ufSKTWauLcC3BEB80Bpol6cEKQsf4OBh 5mcMnEvGrsMKy9smWv8Iz8dvoQ== X-Received: by 2002:a7b:c4d8:: with SMTP id g24mr2550855wmk.127.1594106080423; Tue, 07 Jul 2020 00:14:40 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z25sm2053697wmk.28.2020.07.07.00.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:30 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 418811FFBF; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 38/41] shippable: pull images from registry instead of building Date: Tue, 7 Jul 2020 08:08:55 +0100 Message-Id: <20200707070858.6622-39-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now we have a source for registry images pull from there rather than re-building every time on shippable. Signed-off-by: Alex Bennée Message-Id: <20200701135652.1366-41-alex.bennee@linaro.org> -- 2.20.1 diff --git a/.shippable.yml b/.shippable.yml index 81905727d14c..f6b742432e5f 100644 --- a/.shippable.yml +++ b/.shippable.yml @@ -26,12 +26,10 @@ env: - IMAGE=debian-ppc64el-cross TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user build: - pre_ci: - - make docker-image-${IMAGE} V=1 pre_ci_boot: - image_name: qemu/${IMAGE} + image_name: registry.gitlab.com/qemu-project/qemu/${IMAGE} image_tag: latest - pull: false + pull: true options: "-e HOME=/root" ci: - unset CC From patchwork Tue Jul 7 07:08:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234927 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp656968ilg; Tue, 7 Jul 2020 00:17:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwGjUE4IckNeAiPjl5/h7iWdhi1nkUEg5sRS7fPtf4VQdOrhSdW6WWmqEij9+0YutmzU/8D X-Received: by 2002:a25:cf44:: with SMTP id f65mr83305136ybg.368.1594106278656; Tue, 07 Jul 2020 00:17:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106278; cv=none; d=google.com; s=arc-20160816; b=mgBkGreB/y4WtkIJwQ/ePJb8tRedksD+TCiw8AYTbF3hkuD4RLwS2IhMkkMJ/oNmH7 Nkl50LZJteO7r1rfiRBQxfGcy5bVjemGHiVDUPtFw8TZ+S0/1J8+wrLOaAVr5lT0u7GZ lwDLgpjYuvVciOE0nMEpY1BI4Ijw0WTUt4m/f2mVxpqPe+t6Z4QjbEPeZcBF6mjhMOV9 JQnQxnP6LkusOyGJ4MXl/EVbLsEOORJb/T9BUFWdG+1BRGP1bQrbmBxs65Deyuvl/f7f NtjKd3U0z13Uxz7gihjNs3ORDsNT5fzpThToMit/AVHRT4GkynJy4FkGNppCyGhm+54C l0ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0Du9+TDok3/8t4BiL1lqGOO2m2hHyL++J15tcPz/gDI=; b=Cp+a7fzaZMLRXoAL7YYsmHRBnCSchMUBbdYqmgOLAFEOY71yj5Nbt7d3XAEcND7TWz AJ1TFb05e/KWPrKY+BRx/J4dtlfSHf8dXRT9POLh9lu8NDGs3LL0JSxmT5frtVJmNJiV IPatykAjXp3jr4YZW2mm2+Ky9GOnJFe7PYqKhZpzb1SknVpPKKdD25qOIEUKi6Yb27VU PfaE6kLTKVebwQaBV2hDKMYmOFqACm6H4n1ipjmFN3Xac1qh7yGfapKEOZelTKPT82Rf x95TkDCnispipUYU0K91CZeX8xTZ51VGF0COtQrpWn8XmqwVjZzODnt5pHrVr6eAcIH7 1e7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=eIcbOXwJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k17si21537000ybk.289.2020.07.07.00.17.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:17:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=eIcbOXwJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42518 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshra-0002vT-4m for patch@linaro.org; Tue, 07 Jul 2020 03:17:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoE-0006Yu-EY for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:30 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:40246) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoC-0004qZ-Mo for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:30 -0400 Received: by mail-wm1-x32b.google.com with SMTP id f139so45346276wmf.5 for ; Tue, 07 Jul 2020 00:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0Du9+TDok3/8t4BiL1lqGOO2m2hHyL++J15tcPz/gDI=; b=eIcbOXwJFxzs4KDjejfWky8SJqCIhYrm8YWjqtSH6Td/fYGRBYErOJrPZAQDc875SF DZlZXjfYAX6z3LfpSrZ7f869T5vFUSVFdD+vTAizykmqO02KIfGVb9q+N842pw/1+Nj8 DsE2yozOeFS9JjX53dZvy1I0nLnG+9HOGgMxWsrFEaqJhvMRUar+AihtKoVjrgRRkG4j k59ByDfC6nXz4Xdu/WF2rzTaBsmKSGrkpe5f2eHVkJNzkML8GN/fQcjW14YPjN0285OJ todiyxB/kSgwYSIUSo7G9DGuXZ1KAvQ3zVOGSU864EtemWB0usJUe1l2CZLpw7HqGiwZ 67Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Du9+TDok3/8t4BiL1lqGOO2m2hHyL++J15tcPz/gDI=; b=OVi1e8IQPKLXHgMVk7T7sK4uu1W8Cv5zYn/F78sSLhwzQMYcxBc0lUIGhzwkVHcI8U 3FStUWhhUvLvJYGkCbRfFPb5hcYr8/DW2f3xo3Z3p8I0mbDsTUvHGOALTHOTHJpasHqi Dfq6LbmiSeFBzBBuNUpwZp64zz0VMr4qlTCr1jUuiM383q7oX9svu7nDqjLVefmXGZm2 7BoU0AzRCQu/2JmWSYwAFD4xRhhugZa/YnnVDVqQFty8oJu8/00KJtOJsVoBcMkzAN9/ CPVcKM3TqskExHzuF4mnSS5y8cxi2eKhEE9e4tanKqTJeXQOrVuo356mfErRaZbjaihP YqgQ== X-Gm-Message-State: AOAM532Kfm304UNhELQDB65h15GXbZGXqB6ot/FqO26DpSWE7rj3qdD6 /NaaRJY07La35wnFbgcxGYOH4g== X-Received: by 2002:a05:600c:2209:: with SMTP id z9mr2613163wml.178.1594106066199; Tue, 07 Jul 2020 00:14:26 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id w7sm2283183wmc.32.2020.07.07.00.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:22 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 595E81FFC0; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 39/41] travis.yml: Test also the other targets on s390x Date: Tue, 7 Jul 2020 08:08:56 +0100 Message-Id: <20200707070858.6622-40-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Fam Zheng , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Cornelia Huck , qemu-devel@nongnu.org, "open list:S390 general arch..." , =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Thomas Huth s390x is our only big endian host in our CI, so building and testing QEMU there is quite valuable. Thus let's also test the other targets with additional jobs (also using different sets of pre-installed libraries to get a better coverage of the things that we test). Signed-off-by: Thomas Huth Reviewed-by: Laurent Vivier Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200608114049.4693-1-thuth@redhat.com> -- 2.20.1 diff --git a/.travis.yml b/.travis.yml index c24dfbe377f3..ab429500fc3c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -482,6 +482,45 @@ jobs: $(exit $BUILD_RC); fi + - name: "[s390x] GCC (other-softmmu)" + arch: s390x + dist: bionic + addons: + apt_packages: + - libaio-dev + - libattr1-dev + - libcap-ng-dev + - libgnutls28-dev + - libiscsi-dev + - liblttng-ust-dev + - liblzo2-dev + - libncurses-dev + - libnfs-dev + - libnss3-dev + - libpixman-1-dev + - libsdl2-dev + - libsdl2-image-dev + - libseccomp-dev + - libsnappy-dev + - libzstd-dev + - nettle-dev + - xfslibs-dev + # Tests dependencies + - genisoimage + env: + - CONFIG="--disable-containers --audio-drv-list=sdl --disable-user + --target-list-exclude=${MAIN_SOFTMMU_TARGETS}" + + - name: "[s390x] GCC (user)" + arch: s390x + dist: bionic + addons: + apt_packages: + - libgcrypt20-dev + - libgnutls28-dev + env: + - CONFIG="--disable-containers --disable-system" + - name: "[s390x] Clang (disable-tcg)" arch: s390x dist: bionic From patchwork Tue Jul 7 07:08:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234939 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp660238ilg; Tue, 7 Jul 2020 00:23:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzjK4g2dfsThsbhB6HFwqLHLfvmwAd5C4AQr5upSKZU6R8lMrzJl3wFRIArRqMT95Mjol6o X-Received: by 2002:a25:d843:: with SMTP id p64mr20671950ybg.68.1594106634770; Tue, 07 Jul 2020 00:23:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594106634; cv=none; d=google.com; s=arc-20160816; b=0xhkbpWxo3I/I7XoG+CRnaH9yWbh/zd4uw7PL1+OFyCuz0XAPOQ3OOdbi6yetdkGUm g9/2ISbO6CR5J1Xp13wPy9kXLsFxrXNIeyhkpp+c5zA9hbH+dV0XLFdhqU0FZ6gIIFXC GXzivMua68TxWyiTWhwyWoZKhCvMeBioe8MIClYS0YZy/9H+YNDpbA44WQNbz4Z94pXs EqqAB9oG1BZsl0UCWZEhmoTt46vug17nKL9MLioSaIWvuECj+eWmAM7pbSqFYW92QT1W EIoe7EVKCgzAMysWlnyBa5NTsH6o1cMM+Resf9LHk2tfBN+Y1wetqL0QI6g17BuORGre jqwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WQFoNRo6TyQ2t2BXDRqFhlPss5nm1nW+xtYpQWhCpNM=; b=MkDHKiAdm6doOvvdRqZ2S0V92tkKYxcqhMKL3R/m7TVUzWBtvBQKHkdvj9Z1WSxIC7 FSNshVlCpmB2hm7E7AYBzyv/CTCcrUZUGySXeCa/PH28hL9oncJQQioe66Iu0EKP4PC+ DSV1lmZeybQe69+lz9juSsaEovWdLN4kpVBqre1tV9ZRGGRrYN7zXKtZCp3CaxIdvdS3 L0YwABL2sFisJ1cMKr6lSmH9dYlcAQAJdDhGHeiXYv1noEsBEoB+3PPukzSIlcGyGuD5 JWQMTnlRK7zaC9dNaoFwEk6vhV+tLiKytk7KE6cXQT0FnDCbAydGllZ9nJRzTRe/HoZr 00EA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=w2iIUful; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h10si20450141ybg.116.2020.07.07.00.23.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:23:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=w2iIUful; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshxK-00085t-8t for patch@linaro.org; Tue, 07 Jul 2020 03:23:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43832) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshoU-00076y-Ub for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:48 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]:51260) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshoS-0004vQ-Bo for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:14:46 -0400 Received: by mail-wm1-x329.google.com with SMTP id 22so42135429wmg.1 for ; Tue, 07 Jul 2020 00:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WQFoNRo6TyQ2t2BXDRqFhlPss5nm1nW+xtYpQWhCpNM=; b=w2iIUfulOVmngzyazVWkARRyDYpTUf0QsupyXUuItaDjo6+kBI+6OIhvcW6HpvexsS AUsYlt1NaCbwKrUD2Nzk8kYukkCBjeAmPzpyJG1NWaJLlINIS3j1aZM8v1F3T/Mt3Hii KwTsqBfMuzEP0QsGlpN3+1+4kMB5qJlf3Z9X4V6mM9bY72hKorpCZmF6QeN6owTtz9ft cG7rXtGqZ2zCMMn8M1DsyinXSbXhiuyilbM/pVXGrRX7KDeDrpV+akqiwwfvTXRlEr9F YdSlw5iPhSqMRFE0PMrIb0BpEmJIJsHhnOSQH17VzZg7yAgU0l6x3cE3mwN20+43KYPW /Ljg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WQFoNRo6TyQ2t2BXDRqFhlPss5nm1nW+xtYpQWhCpNM=; b=JvnPMfSWUk+nJZ5fVXb+EwGKEa3r9t5WuFDwT4rfFnjWWUysDwDWCFKej/LuumP2B+ Ux3s16DvvmupZT3Wg56y3B4er5YR5+C1oe2uUY2CO7J4iB2lMFDPJf0TOWb9z0WBp5X6 vySwU1zGsNMUtKvuFQ6ttS/JDFmCoxZjA7EL1HXEmFtOT4ih8q+TNQg9f3Mb19CT5PEW XsQfYDMFQZFgte6J5QYElufQJTZXlbEqtTmQTbBBNOXHlijYZnIe2oL8vxYWKGV98teL m6+ENdzHP93NBa93BT1Lj8hdGysC37Gm/cpN+Lbkylg2JL3ZRzbEE3vyjwEnRPzyzXxK zN5A== X-Gm-Message-State: AOAM533KX0Ix0O+AhLbT4uY9OduF4q2EwR8J9Z6gtozTC/UFTmKXxVJf j9vmyjTUAHirfPLDQAMY9ST4bg== X-Received: by 2002:a1c:1d93:: with SMTP id d141mr2570402wmd.14.1594106082670; Tue, 07 Jul 2020 00:14:42 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 63sm28575200wra.86.2020.07.07.00.14.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:14:38 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 704181FFC1; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 40/41] tests/qht-bench: Adjust testing rate by -1 Date: Tue, 7 Jul 2020 08:08:57 +0100 Message-Id: <20200707070858.6622-41-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x329.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Emilio G . Cota" , Richard Henderson , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Richard Henderson Since the seed must be non-zero, subtracting 1 means puts the rate in 0..UINT64_MAX-1, which allows the 0 and UINT64_MAX thresholds to corrspond to 0% (never) and 100% (always). Suggested-by: Emilio G. Cota Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200626200950.1015121-2-richard.henderson@linaro.org> -- 2.20.1 diff --git a/tests/qht-bench.c b/tests/qht-bench.c index eb88a9013761..ad885d89d054 100644 --- a/tests/qht-bench.c +++ b/tests/qht-bench.c @@ -25,7 +25,13 @@ struct thread_stats { struct thread_info { void (*func)(struct thread_info *); struct thread_stats stats; - uint64_t r; + /* + * Seed is in the range [1..UINT64_MAX], because the RNG requires + * a non-zero seed. To use, subtract 1 and compare against the + * threshold with =. This lets threshold = 0 never match (0% hit), + * and threshold = UINT64_MAX always match (100% hit). + */ + uint64_t seed; bool write_op; /* writes alternate between insertions and removals */ bool resize_down; } QEMU_ALIGNED(64); /* avoid false sharing among threads */ @@ -131,8 +137,9 @@ static uint64_t xorshift64star(uint64_t x) static void do_rz(struct thread_info *info) { struct thread_stats *stats = &info->stats; + uint64_t r = info->seed - 1; - if (info->r < resize_threshold) { + if (r < resize_threshold) { size_t size = info->resize_down ? resize_min : resize_max; bool resized; @@ -151,13 +158,14 @@ static void do_rz(struct thread_info *info) static void do_rw(struct thread_info *info) { struct thread_stats *stats = &info->stats; + uint64_t r = info->seed - 1; uint32_t hash; long *p; - if (info->r >= update_threshold) { + if (r >= update_threshold) { bool read; - p = &keys[info->r & (lookup_range - 1)]; + p = &keys[r & (lookup_range - 1)]; hash = hfunc(*p); read = qht_lookup(&ht, p, hash); if (read) { @@ -166,7 +174,7 @@ static void do_rw(struct thread_info *info) stats->not_rd++; } } else { - p = &keys[info->r & (update_range - 1)]; + p = &keys[r & (update_range - 1)]; hash = hfunc(*p); if (info->write_op) { bool written = false; @@ -208,7 +216,7 @@ static void *thread_func(void *p) rcu_read_lock(); while (!atomic_read(&test_stop)) { - info->r = xorshift64star(info->r); + info->seed = xorshift64star(info->seed); info->func(info); } rcu_read_unlock(); @@ -221,7 +229,7 @@ static void *thread_func(void *p) static void prepare_thread_info(struct thread_info *info, int i) { /* seed for the RNG; each thread should have a different one */ - info->r = (i + 1) ^ time(NULL); + info->seed = (i + 1) ^ time(NULL); /* the first update will be a write */ info->write_op = true; /* the first resize will be down */ From patchwork Tue Jul 7 07:08:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 234916 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp653357ilg; Tue, 7 Jul 2020 00:11:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyCi9uwHFeDH72V4u7Hun8RkcMCWBuBZdawzrSMoZLJf/NgoMf2Itdj3I17c3cl4VpxIiUV X-Received: by 2002:a25:3849:: with SMTP id f70mr85118605yba.135.1594105916877; Tue, 07 Jul 2020 00:11:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594105916; cv=none; d=google.com; s=arc-20160816; b=DuI3dKYRlZP/IFBQ14zK4O2Oy1PFRLqoX//u7YoPQGIxFj6RCMx198/QmumCNWfyiX b4F+qa64vjqtoQ+7sDiu5M2OC/fTSfOpFHeQsDaj3mOrqEr6rzeN6q67umXGFUC3ysYP oglK4xOZgIpxdGoxT/AyqsysWRcpCvNhmbF4G5fQV9Y644cQDAE+kGBaKYmNhN6vGWgJ xdD96k0WVXnjO1QKyKqHolxCI/cy9P9o1fVG7giX7Q+tS6S+kK7OawHwnGWGjLAqGuFm wC92ruM3nSqrfPqHiibz/MYqLtb4peKoRM45V3UB/AWWMGhprmv9RqKLb7/bjzJC/Uoz SoZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nRniERkDA/6Hi4fEHIqm7Z1ve7b/ehRoLpBF7hXj1y4=; b=mogkiU7fW6je771HwgsJLkAV754BwHHFIet29FbPkXFeVFFzTsuuFpQpx2JaQV2ZJa 2vy75hIs6Fp38wJlYIIat3yBJPsBlQthtSrsFi/6Y7yP3FuuMbbUJ0jK7lf5uTUw5biy bVRD5InXcYrvVUKczYoUj/tnhbu3NooIbXbpx5ssX/1ocWAPhu5ir52icHjIveT2YqwK s5ZOErL4ckZ9L/tmyyj3lpTveVFNcuu01nbAYEs1I8pETkW21OqRM6QyEkuJvttOUyqe Wha1l2gkkLfjn2xJOn+kvfwcR8wjUMPIwJ2guwVMlqtCh+t9E3oUVfLmlDvPy8TGCgbB xTOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=W7bBfv30; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s65si23197905ybc.138.2020.07.07.00.11.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 07 Jul 2020 00:11:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=W7bBfv30; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jshlk-0008Vw-B9 for patch@linaro.org; Tue, 07 Jul 2020 03:11:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42708) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jshjJ-000564-G1 for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:25 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:34552) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jshjH-00046y-Lk for qemu-devel@nongnu.org; Tue, 07 Jul 2020 03:09:25 -0400 Received: by mail-wr1-x42c.google.com with SMTP id f7so40960368wrw.1 for ; Tue, 07 Jul 2020 00:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nRniERkDA/6Hi4fEHIqm7Z1ve7b/ehRoLpBF7hXj1y4=; b=W7bBfv30QS5YuvhsI5vWd2Nuyn86nV6VZ2jgWgWG7BA5OwbEU9amvj/s8BFCznBo8r ScOjAa9/Xbk0w/3LpKvE07OD6+eajF8WJees7f8+8aEQXa6CPY7QphQoxq7vWdBW+KGd Ah7RCunCq0OJDRJR/dSCGAdIbvh6XI2xZFgsrqlOL8eeC1nHVNZ65LyKnrrv1mGJ+TCq Ks8vj2L8cnswfUtoLCrSXlj/+875fReZd4eGuEJvtKhubXRQslN9YfAjXq23Jo4RsYWM b3KcfSvb3W6JCflA7A4pXt8lNKGPOjkT5QEnTta1LvlzHrnvhemv8cXJ1/1lNuKYU1hb mdjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nRniERkDA/6Hi4fEHIqm7Z1ve7b/ehRoLpBF7hXj1y4=; b=C0GuVVI5BPaVGH8x6b1KHtN7tH8g1r7uM7ytlVaivH/jn/Dnf8XstwoMG0teNYbTGS fwUSHCQ4cwiI8rSYhqlv4al2gxsBQb3RYEY2jZ1Mut284ZMkJHbp9wNEKVuFb/s66Zr6 3Y1CiXvtEXcJ8iEiQ/4H6DYfBEBtyJrflnKxKJIUdfsKbFUtW4C6DiWJebLsrARezsvf 214o2EV5ZRY1A8HIIoN5akG6WVUnB9S4z68IXLYpeDP8pVU8h1tvJr4ipVQS/tm/uz4v hRi+hPKjbu7jMNBOUN0btqcYgAjW9q8A9z5dnWPqYawC9lujafTXIXaMQ0RLbmpibaw+ lLkg== X-Gm-Message-State: AOAM532lmIOF7lSSwSqzYAZRw08bhzZqj3Y7qx3Ya6lMBf31rhEeGt/v SIjT7X4rFGws0/BX2ter9cYajw== X-Received: by 2002:adf:fa81:: with SMTP id h1mr52813021wrr.266.1594105761843; Tue, 07 Jul 2020 00:09:21 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id j4sm22550758wrp.51.2020.07.07.00.09.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jul 2020 00:09:13 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 87C7D1FFC2; Tue, 7 Jul 2020 08:09:02 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Subject: [PULL 41/41] tests/qht-bench: Adjust threshold computation Date: Tue, 7 Jul 2020 08:08:58 +0100 Message-Id: <20200707070858.6622-42-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200707070858.6622-1-alex.bennee@linaro.org> References: <20200707070858.6622-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Richard Henderson , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Richard Henderson In 06c4cc3660b3, we split the multiplication in two parts to avoid a clang warning. But because double still rounds to 53 bits, this does not provide additional precision beyond multiplication by nextafter(0x1p64, 0), the largest representable value smaller than 2**64. However, since we have eliminated 1.0, mutiplying by 2**64 produces a better distribution of input values to the output values. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Alex Bennée Message-Id: <20200626200950.1015121-3-richard.henderson@linaro.org> -- 2.20.1 diff --git a/tests/qht-bench.c b/tests/qht-bench.c index ad885d89d054..362f03cb0370 100644 --- a/tests/qht-bench.c +++ b/tests/qht-bench.c @@ -289,11 +289,25 @@ static void pr_params(void) static void do_threshold(double rate, uint64_t *threshold) { + /* + * For 0 <= rate <= 1, scale to fit in a uint64_t. + * + * Scale by 2**64, with a special case for 1.0. + * The remainder of the possible values are scattered between 0 + * and 0xfffffffffffff800 (nextafter(0x1p64, 0)). + * + * Note that we cannot simply scale by UINT64_MAX, because that + * value is not representable as an IEEE double value. + * + * If we scale by the next largest value, nextafter(0x1p64, 0), + * then the remainder of the possible values are scattered between + * 0 and 0xfffffffffffff000. Which leaves us with a gap between + * the final two inputs that is twice as large as any other. + */ if (rate == 1.0) { *threshold = UINT64_MAX; } else { - *threshold = (rate * 0xffff000000000000ull) - + (rate * 0x0000ffffffffffffull); + *threshold = rate * 0x1p64; } }