From patchwork Wed Jun 24 14:04:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 191672 Delivered-To: patch@linaro.org Received: by 2002:a92:1f07:0:0:0:0:0 with SMTP id i7csp635637ile; Wed, 24 Jun 2020 07:17:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxKWvoGW3Bk9XAT0wwEGSKUiBxWcBjnMTy2MMBDIDxPDNsFmNgJR9Yw9fqjAafGWPKnbdmo X-Received: by 2002:a25:60c5:: with SMTP id u188mr43901977ybb.268.1593008269268; Wed, 24 Jun 2020 07:17:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593008269; cv=none; d=google.com; s=arc-20160816; b=0EqhHGxHSpEM2kFn5BZfn60b5O5CbUJLVQs2vzVo6EWE2MEGKt0Gf4KWfcwCC9F2s8 UaBZL8DLMaeU/s13iN0W7p4PwJyEOAdOu0B9VS0rDbkPQOw43GlsjbdAcod31PXOixhk qu1UTC4vpRNfKfBh60laIAf0HE/xibuuz2yjsbC8Ys/oKJSYcsRNrwaE1FT+4YIZsUv7 aUvSi8+VilxJdOMuQhcJMtf/7HUbx9EvVIXPbg2frfmQghFNfjBvQFix+hGMXWXJ9d3/ JweMYLVaiSnzybkve3Zg/h5fmHrpJRTtyyhhlh/qkNRbkgayeg9X4stCRt+0zxguZCHQ TENg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=lg7yVStaFiCphxFtdOAJanK/D+52X1Xo/oiJ0lxSebYQXIPESoZB/hRblqM2idCVVl go0op9B0FRHxX02GBixRimi9wzZuamft80ZDsqt5kZNdOFSrtLsgi6MVQm2NdFv/O+n0 ywUCnq0wUnd9VMJRM9I9yWBUYTJKs/qstgd+7R743Q/pxwJCzH1gKQYiyBXPK6srJLhe mOe4rHg07qI9fgRUU9G4AE5WSm2GpU/9CMly36IDHsp1v/bkKFXZ6Q+PNASYC7OFhtoO aTvI1BDBklnXxtUDCUa9RUUbJLvXXzOEI+pysu8ggYOCW71q8Zm7JeSZP14FSNGh/jX1 R4Pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=k4Tsq37p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a5si18798021ybg.97.2020.06.24.07.17.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Jun 2020 07:17:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=k4Tsq37p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jo6Dk-0004wk-Om for patch@linaro.org; Wed, 24 Jun 2020 10:17:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jo690-0004MG-7c for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:54 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:46103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jo68y-0005yx-2O for qemu-devel@nongnu.org; Wed, 24 Jun 2020 10:12:53 -0400 Received: by mail-wr1-x433.google.com with SMTP id r12so2404322wrj.13 for ; Wed, 24 Jun 2020 07:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=k4Tsq37pNmIh4FQGQktJmIeM+22h8iZzyUuLGzPNnPZSff+zV+vOm2yBPRSk+yj8qq djBgFexMBGJVdnkuiemFkYZjSm9oUsUNexWwdDVeb+s61XxWjQpdw0A+hcZOuF7r50vR Cp0kMz0bgJWGG3CHw1MwadXGfzLi7Rm3tRz1Y/NkeJt/ykShFkBFBceQcAerq1hifHWB jPpzWRq/lP2ffM9LZaeBaOCClymjZEOIz8Bty7LV39FhQ/JNCQtheD/CObAs2HUk0Hfs MldG9SbtxVJKq6Ggcxq+N8tDanl0SEBVzOyrOu8LgIEco8EC8QmCosjviKiNSlIWWTd2 O6yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rqrRYyEjB3eVwRN1aq9rKw8qdZWBco1/JdyE1giTcD0=; b=WZt/lLDU+ikOaxTaQ5/S+0otsBP37tyt6H6gRLoS+IIoCCS4D3a4FNtU21b+tCCaen 1VChqSal+ZwW2OqB16vYgydKy5yki0lgoj/W7BbOokyjoMOsRKExQDu+BomKYGStuxuv EdQruDy40rQXAO3c7/kI9g4esIebp1YQGRm+AtWFVmhvkmGGLKeOFtuCZa4F3MhW2dxs ZwDQkCtHoCtFXL1uKFlFEx7CZRBqQo+eQ37zdgkpcdYEfdAxhwXmO4tAf0586u2PPbok +6IUQ9ouz5RLDoP8oVuAB54eJLmzCZ5g1yT86qgULvSkoA+WJWX5QJjsvqGsp/GwKRlY boMQ== X-Gm-Message-State: AOAM532VasMVUlo71VOSMx5oAfp36km/7TFt5jPZucxG3nX9SszCXzJH Nft/+v98TxojcUj8wpXBm7SuCw== X-Received: by 2002:adf:e50a:: with SMTP id j10mr33329426wrm.71.1593007970435; Wed, 24 Jun 2020 07:12:50 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id z1sm18759179wru.30.2020.06.24.07.12.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 07:12:46 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 2FCFD1FFA6; Wed, 24 Jun 2020 15:04:48 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v2 18/25] gitlab: build all container images during CI Date: Wed, 24 Jun 2020 15:04:39 +0100 Message-Id: <20200624140446.15380-19-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200624140446.15380-1-alex.bennee@linaro.org> References: <20200624140446.15380-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::433; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, Thomas Huth , berrange@redhat.com, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, Wainer dos Santos Moschetta , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?b?w6k=?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Daniel P. Berrangé We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée --- .gitlab-ci.d/containers.yml | 248 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 251 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml -- 2.20.1 diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 00000000000..580439647ef --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,248 @@ + + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu/,FROM $CI_REGISTRY_IMAGE/qemu/," tests/docker/dockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu + diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea63..72d688875fd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' .update_apt_template: &before_script_apt before_script: