From patchwork Fri Jun 26 18:13:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191910 Delivered-To: patch@linaro.org Received: by 2002:a54:3249:0:0:0:0:0 with SMTP id g9csp654908ecs; Fri, 26 Jun 2020 11:29:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyvAb5Au+g9PGRr8DlyLjY0LXuifuxRIUbiCe5dEABFJKkxJ5ZT3vRz0GrRI6E5e5hzC4cj X-Received: by 2002:a25:a226:: with SMTP id b35mr7127758ybi.356.1593196190169; Fri, 26 Jun 2020 11:29:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593196190; cv=none; d=google.com; s=arc-20160816; b=PRsheCkdQuAg2CiNZtqTGe8tk1AlvTS8y9MHcTArFDUOllwk7TPAjwUMxvyQzYj9Vy jhTcPWyXF8DU0iUgiyJucXAxBbiB+rrmCgH6Hee+u3kHuLg7RvogxxVpI/pOt5ldC8v6 ii4acUfrxuyOHdz3CKX9vvAs1IINIroAE3pLU+VXUC/qLc6tfzXsRCozopomqeheqhxa 0XIbpv/WiZgs+rxpT4fvvzvG0zKK4JntXQ9TTyuuA1rPqqkglBgzSbEocQEAzofwwmeX slQvlXOFV37iA+aLoizYe9o6a1Zbnz0C4HjwrUMGz3DDJpyu5ClYvI/U3lP8qqjYw90W idWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=xUZG6tLp/xI8r3bd4dXEs/uYPfQvAur5hQZG/hEKfXs=; b=dDnh37L8gYj3qE8jycCmyW1ZWta5EkKvfRMdwBqzj1xNU8Uq2ClADiLevct47bPmwq KybGg/pXMJU+Csy9rrTroLqxv67PyA3cFuFDcFe+VlPaoN64EdL/m/3tLMf057/d7dAx 5mu0w09aX3JTGstKWKYUDLDVbeB+NMG65vcN2ZChwdAqM7A5OV2ilHhJQ9PKK4GkGN6z 4P0cwGsvPf0gxt9UrWZdD/n3MaZyftwqGk4J/XD3Z0s7XNLtL9Vf1N0LY1TF35SXpbMe PT+vym5zWvGA2KJOTSP6gTzZFIRNNcJextQqlfkPVFELfktj0ouEJt75d3oq6ToP9Swi FnFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=podm8Phc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h1si26004742ybo.141.2020.06.26.11.29.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 26 Jun 2020 11:29:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=podm8Phc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42420 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jot6j-0006et-Fd for patch@linaro.org; Fri, 26 Jun 2020 14:29:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jot0B-0003ds-E8 for qemu-devel@nongnu.org; Fri, 26 Jun 2020 14:23:03 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:44796) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jot09-00055f-Af for qemu-devel@nongnu.org; Fri, 26 Jun 2020 14:23:03 -0400 Received: by mail-wr1-x42f.google.com with SMTP id b6so10356875wrs.11 for ; Fri, 26 Jun 2020 11:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xUZG6tLp/xI8r3bd4dXEs/uYPfQvAur5hQZG/hEKfXs=; b=podm8PhcCmL7vam9aI9CPY+fYS9MtndrEkcOmhw8MQk++72Yuc43FVzU3vk10+0muO OHYf0eclrX9mL3phNozKFbraMeBKc+HpCcAYuNwe7XbHUly4m+NafwlWvaA/PMTrncSo FFUxpSBiu7G24wYTfj63pvyPsUeadOtO+MhHEdy6/7uHLizhi7Ratw/hGI0ViZkfJ6/j 2s7Z5MjeCdCgkGyHV49f36Pd3EfwpoU+Ig7YpiJOo3uJ/dbZzWUy7m6scWTROV3lbrOa UcCnwWrIihHhsfXBKEliydKSpMYSdBelC+ZwjOUCI7AXoLWG9a8kpNGTzC+fRz2nwCWv lXdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xUZG6tLp/xI8r3bd4dXEs/uYPfQvAur5hQZG/hEKfXs=; b=OUvUFaCdyAzR9y6icFNjFZacfeKiDpDtFwKdeKF+srKDDf9Z0DjzGPtnJKTpD9a604 SFO6tAngGYupviGtSJTlqGRcFBJ38SoOwjx07e2F88YuCSld4SHeKqXYm0RIqka9Lru9 Asej6EorQO2Vh/xMtq+mM+BCmCSsQwHwD9S0h4FtugfoE9kZ0ITdhEMlibreDbKogb8b dmUKN7lQ0Rt1YEM/uKkdR7kCSJ6j2paVQjWr6KY8/v5M5DYIao4eOXWRMcjWLu2CWZ7g 76kJoyuLE+VjJPEJR01u8lFULbaOKxlgjNXp6F3tWj5RLqOvGOxfMu8978dbuoYHsuj+ wmLw== X-Gm-Message-State: AOAM532ZJy7XJVca1KUxJn5wVjgvHPqKxwCon7LxW3EuXC98KplJ2HkH gjKVKlxpYyfIi6maKuooI5VHLw== X-Received: by 2002:adf:f2cd:: with SMTP id d13mr4821906wrp.378.1593195779870; Fri, 26 Jun 2020 11:22:59 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z16sm27768267wrr.35.2020.06.26.11.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2020 11:22:56 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id F189A1FFA5; Fri, 26 Jun 2020 19:14:01 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v3 19/30] gitlab: build all container images during CI Date: Fri, 26 Jun 2020 19:13:46 +0100 Message-Id: <20200626181357.26211-20-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200626181357.26211-1-alex.bennee@linaro.org> References: <20200626181357.26211-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée Acked-by: Thomas Huth --- .gitlab-ci.d/containers.yml | 248 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 251 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 00000000000..580439647ef --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,248 @@ + + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu + diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea63..72d688875fd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' .update_apt_template: &before_script_apt before_script: