From patchwork Fri May 27 15:35:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 576495 Delivered-To: patch@linaro.org Received: by 2002:a05:7000:8607:0:0:0:0 with SMTP id bb7csp766172mab; Fri, 27 May 2022 09:04:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw0lIh5+NL7mBVE+PN2TCFXPqawMeUApMqSr+TENtjifFYbyD3leoVORuZjw9RODUBa/k+t X-Received: by 2002:ac8:598b:0:b0:2fa:eb18:560f with SMTP id e11-20020ac8598b000000b002faeb18560fmr11854524qte.488.1653667480753; Fri, 27 May 2022 09:04:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1653667480; cv=none; d=google.com; s=arc-20160816; b=YD7myIKKo/dAJ0y5elk4BDyHIe6z7ist9KH9ssza6Dqwh4HJi6kEXFhWsC9CfJyb9k RPugljmXU46icaFySRmSBuVbjOySbdAcbKvSulzhbm0NdrZCNUE1RTjVWsGdB6IT5e4J P+sdaR+1ricEkkN1kZAfsFyCQvA6rpVxHO7lf6H3jVGZYZDq1+E+wlACk9ARLq6PlbPO 0y55Void5S0QhwUpRjuEVZoLAepc74Wu98MpqtAUjzfFBl3IzVR4V+GINR7apvCBskBZ Gh5yWIB71oTOhS6DpxNfosEG35pfA23LtC3hr6yPd1L6dHL7ubEUCqexHt7vX2IURQ3j +d4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=G8GBt5IlQVSVKXeBv2hLy/mbgJLnCe5MD+PYqSSVW+Y=; b=nVnDYZxPxvEzNBcU2GWx7Mf8nhwVB80q97YcKsrMSXKw+awnOMBVyWlj0le0Bw0xWh RJpROYfJhu4E2+cWByscfRazBYJ3C39ZA2xVCfHlda7MxZFaA7Uu7u/DcMynDOxyKUez SVucLNtUdYyifrCY12T1VFUT4wZQ8OJS1pfrzU2c2/j3H/6Oj8ZqjtW8NEWvMo0y5uGB aP7M2Ln6efsn2B0PW2+JZZ0qQUQRnghYoX5E58zAFkB7tmw9wrW5ZvklXV4oYPHGAkTP HsO6+4X1sbL/RCRc0Avyn7PWMXkqzamv5K/FnFhtrbaExbsUwCqpGXxzaf237PCYFJTw xxgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BSTe69nz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p8-20020ad45f48000000b0046235bdc492si2796012qvg.288.2022.05.27.09.04.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 27 May 2022 09:04:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BSTe69nz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nucS7-0001iK-8P for patch@linaro.org; Fri, 27 May 2022 12:04:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nuc8T-0003Hs-54 for qemu-devel@nongnu.org; Fri, 27 May 2022 11:44:21 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]:34028) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nuc8Q-0003tP-Tn for qemu-devel@nongnu.org; Fri, 27 May 2022 11:44:20 -0400 Received: by mail-wm1-x32c.google.com with SMTP id d5-20020a05600c34c500b0039776acee62so2390118wmq.1 for ; Fri, 27 May 2022 08:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G8GBt5IlQVSVKXeBv2hLy/mbgJLnCe5MD+PYqSSVW+Y=; b=BSTe69nzuuDKhU/k6NUTw2Q/Px3Pf20Q8BoaXOSKEU3tCUffcrYMMapsccc8/MGdZO 8Fea9ev2HdPlOxzaQjpjB/gl3SKsPnghzfvOa85Y1XnricNwXTg9ciuVb1PkD7JcpShM 60GU9S+gayr+w5p8SRR6kOsa6Qs0d50onSoqS2hHu/z7vdCYTnOb3lln2ZdcMn+5XtF3 f3IrMLA0TJayQoKNBHwPIbcJx7SXXVlvVU5qvVWJKOysnyu9tjKqsktsSpSawM+Mxiti xi4fnMuB2/L1Vi9xBYV61mhqCdAoti1LXfj9KMbZytsL99kzJ8K7G1OaoJbCxaCvPNkh slNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G8GBt5IlQVSVKXeBv2hLy/mbgJLnCe5MD+PYqSSVW+Y=; b=qcDLqvLPM/yJHrmq7lL8rIR3ijtnInt8Hj3LgHXlsxhAsABVkmZ+RVkPt4uTNlDWFa gj7S7u6dTHl+sqsyOFD6cJ3OAbGb/tASEnvjPWuO/SaML5O2M4qSrcWK0gl7l0bsphUH Gu6biT2PNCReVjqlt9NndJjNFWuQKHu3O8Y/qXxriWmcZNkf2HsJy+pVJvo2EEg/KbPw peVVT11h1OLY3Z8HSQknXJE+QmpHbe7vvRWUqRJZ3TMn5PXBM4HnooLrv4IXgVABcwrK pTCJWBYnG3QOCtzQXmpgd4qKXi3mHXaWa0I2x8DV/DetGozcShj73g4rq3LeBJwGgwK/ g0gw== X-Gm-Message-State: AOAM532wvtauAVAhjlLST1KJ5kZx9heXLeUdLI5XJAvY5w0a+WLU+uqt 7Ofs5B9aR3DK6vY9/3eNUJ7V85xZpYxZlA== X-Received: by 2002:a05:600c:502a:b0:397:44d1:d5b6 with SMTP id n42-20020a05600c502a00b0039744d1d5b6mr7584049wmr.57.1653666257518; Fri, 27 May 2022 08:44:17 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id t22-20020a05600c41d600b003942a244ebesm2544037wmh.3.2022.05.27.08.44.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 May 2022 08:44:06 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 4F4201FFD4; Fri, 27 May 2022 16:36:06 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: fam@euphon.net, berrange@redhat.com, f4bug@amsat.org, aurelien@aurel32.net, pbonzini@redhat.com, stefanha@redhat.com, crosa@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal Subject: [PATCH v1 28/33] gitlab: introduce a common base job template Date: Fri, 27 May 2022 16:35:58 +0100 Message-Id: <20220527153603.887929-29-alex.bennee@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220527153603.887929-1-alex.bennee@linaro.org> References: <20220527153603.887929-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé Currently job rules are spread across the various templates and jobs, making it hard to understand exactly what runs in what scenario. This leads to inconsistency in the rules and increased maint burden. The intent is that we introduce a common '.base_job_template' which will have a general purpose 'rules:' block. No other template or job should define 'rules:', but instead they must rely on the inherited rules. To allow behaviour to be tweaked, rules will be influenced by a number of variables with the naming scheme 'QEMU_JOB_nnnn'. Signed-off-by: Daniel P. Berrangé Message-Id: <20220526110705.59952-2-berrange@redhat.com> Signed-off-by: Alex Bennée Reviewed-by: Thomas Huth --- docs/devel/ci-jobs.rst.inc | 36 ++++++++++++++++++++++++++++++++++- .gitlab-ci.d/base.yml | 28 +++++++++++++++++++++++++++ .gitlab-ci.d/qemu-project.yml | 1 + 3 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 .gitlab-ci.d/base.yml diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc index 92e25872aa..eb6a9e6122 100644 --- a/docs/devel/ci-jobs.rst.inc +++ b/docs/devel/ci-jobs.rst.inc @@ -28,7 +28,35 @@ For further information about how to set these variables, please refer to:: https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd -Here is a list of the most used variables: +Variable naming and grouping +---------------------------- + +The variables used by QEMU's CI configuration are grouped together +in a handful of namespaces + + * QEMU_JOB_nnnn - variables to be defined in individual jobs + or templates, to influence the shared rules defined in the + .base_job_template. + + * QEMU_CI_nnn - variables to be set by contributors in their + repository CI settings, or as git push variables, to influence + which jobs get run in a pipeline + + * nnn - other misc variables not falling into the above + categories, or using different names for historical reasons + and not yet converted. + +Maintainer controlled job variables +----------------------------------- + +The following variables may be set when defining a job in the +CI configuration file. + +Contributor controlled runtime variables +---------------------------------------- + +The following variables may be set by contributors to control +job execution QEMU_CI_AVOCADO_TESTING ~~~~~~~~~~~~~~~~~~~~~~~ @@ -38,6 +66,12 @@ these artifacts are not already cached, downloading them make the jobs reach the timeout limit). Set this variable to have the tests using the Avocado framework run automatically. +Other misc variables +-------------------- + +These variables are primarily to control execution of jobs on +private runners + AARCH64_RUNNER_AVAILABLE ~~~~~~~~~~~~~~~~~~~~~~~~ If you've got access to an aarch64 host that can be used as a gitlab-CI diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml new file mode 100644 index 0000000000..10eb6ab8bc --- /dev/null +++ b/.gitlab-ci.d/base.yml @@ -0,0 +1,28 @@ + +# The order of rules defined here is critically important. +# They are evaluated in order and first match wins. +# +# Thus we group them into a number of stages, ordered from +# most restrictive to least restrictive +# +.base_job_template: + rules: + ############################################################# + # Stage 1: exclude scenarios where we definitely don't + # want jobs to run + ############################################################# + + + ############################################################# + # Stage 2: fine tune execution of jobs in specific scenarios + # where the catch all logic is inapprorpaite + ############################################################# + + + ############################################################# + # Stage 3: catch all logic applying to any job not matching + # an earlier criteria + ############################################################# + + # Jobs can run if any jobs they depend on were successfull + - when: on_success diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml index 871262fe0e..691d9bf5dc 100644 --- a/.gitlab-ci.d/qemu-project.yml +++ b/.gitlab-ci.d/qemu-project.yml @@ -2,6 +2,7 @@ # https://gitlab.com/qemu-project/qemu/-/pipelines include: + - local: '/.gitlab-ci.d/base.yml' - local: '/.gitlab-ci.d/stages.yml' - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml'