From patchwork Fri Jan 6 15:23:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 639692 Delivered-To: patch@linaro.org Received: by 2002:a17:522:f3c4:b0:4b4:3859:abed with SMTP id in4csp834715pvb; Fri, 6 Jan 2023 07:24:46 -0800 (PST) X-Google-Smtp-Source: AMrXdXtkVdkB2YJ9rFBIDJgaJzkyyB495HhHYvkEAEmiMLxJsHQavi3bPPteUc9a4L4wBxYtYE4A X-Received: by 2002:a05:622a:2509:b0:3a9:80bf:83c0 with SMTP id cm9-20020a05622a250900b003a980bf83c0mr80315191qtb.2.1673018685874; Fri, 06 Jan 2023 07:24:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673018685; cv=none; d=google.com; s=arc-20160816; b=07oClVCa/9Ax0Pnif6rsIO7fPJjVxQcgq4ADsWno5vApV+Z1FTXYUqzFq7nG5/0oUN 3e+T91qZoG0gmQ0u2dWqEneTj32gzEChyAjzK+Tycsn+UX/9PXf8f/2vFseaRXS7pZfm 0Czf8wgr6aCF/WkUhspj0rawPMQkLoJjwU5ofcsOmZ6/gXoXYgVXadAEhu1cI2X/FLRh 32plrGtoBd6dkFgOrcdXeM+rf+PuYNcHqpj6OYYFLlj9YB3vyJmZZEMCRfk3BBfhrshS J1br5xvIQT5vNoYAHmZSwiUOQEZwDoOrpIaYtPKygVnumIrWwnztHguZ7z8cr02Lezr+ teEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:cc:to:from:dkim-signature; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=mkBWDB9nx5eCWVk1C1zAz/dpigPUjIb+12A8ls2AX0A6gjH9cnOjdhJ8DHRVZtS0tY iGCLa1IyMuY0RriIlqKeF7BxTlKUuZI8+4VDesjWugh9yGLP4qWHU4Zkip7fGD1o8LwK GJ4x4Oorvf5P9s6ZLS0w+VX5Tt6A/tbk04hq0vCJ7rrkSkIBiN/BIuKUtwJfsWsHxLRh QMugZKKsDCCn3ndJhUM7Eg3TeCS3Sb5tJxY9Ym0KqgTBwn0xBZYFF2w5bf055u9EC11/ Rj77gj5fhsY9eOOizu0rJ0uhP23LiJWCjtZahurIF2qtVl8tfnkKFJWjCLnYBFltbC8S a92g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pcGDOejc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k15-20020ac8474f000000b003a53df740besi845339qtp.632.2023.01.06.07.24.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 06 Jan 2023 07:24:45 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pcGDOejc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pDoZV-0005v9-Gh; Fri, 06 Jan 2023 10:23:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pDoZT-0005ua-U6 for qemu-devel@nongnu.org; Fri, 06 Jan 2023 10:23:51 -0500 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pDoZR-0004ca-RK for qemu-devel@nongnu.org; Fri, 06 Jan 2023 10:23:51 -0500 Received: by mail-wm1-x336.google.com with SMTP id p1-20020a05600c1d8100b003d8c9b191e0so1287635wms.4 for ; Fri, 06 Jan 2023 07:23:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=pcGDOejcgaV77GTz0fD9Ru34t8Gr4g34p5GXGJH+SMxRySbLJm2YkLwKm7zSNHwHHu gNCqeB4dWhOGJN9rI83YYczLFXDxyjUuvoymKfovL3MtsgK8iBOamx4IjhbWCbkxCB7T XYaRJeBz6vvgfEEJcOr2flka/l8zIfgCWf6jIApbloBlj9i/asMkyVavoGLclcsX7YEy L21FTlnedD2J9q0KsTCT9AtoPc2w3FrmlE3oOnjmOb27sz7ILEPL6cxs1hVR72FM8XbW z5iClgAgaBNQXAA72lGZANW1kmcFYvYBl7WZXktnaIqKNJ2+T4zOJMAByqDN9BsaKTPu L5Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bkRhIgXl5AItP6ba03zv87lTI4x9mQkrreMEEEjOaX8=; b=lcFrd98LAaEWPn6HfuSPhafq5wMQslTqRutDkUmWBr2TmWn8KDpx8uPaq37a7PIsBA 0UzekmkDH0MVoUgPefqYb2sE64cWHxihta2cVc3CQTmQ2zpWw6Diias8Iy7yT/7WK+KQ 7sROQo17k8cTD/RIDdtP+QtqvuM4MuY/8QG+HGQ1p87uR2DaWY6FLoVqGUCtBC4KEDY0 mnGnbEPQ2FMJgn2xOW4sAX2GC1arC391ywqZtJsBKaLfJH0bUR5vyXmMwevsxv6FzkMv 9/CEbauOpgj4Qn/DUfrUCL/R8reQw5blmAerRgcvtE6XmM3stpzqDecD2mbgUJrbalWL GHnQ== X-Gm-Message-State: AFqh2kpuBlwo6njMmPoKy9FufAAaFxI+Fa8SqlhTEoNp7t4RHJgWCtYs 3NzsJ3sGn6rO9AX+OL8yEuIswA== X-Received: by 2002:a05:600c:3b29:b0:3cf:d18e:528b with SMTP id m41-20020a05600c3b2900b003cfd18e528bmr40557031wms.39.1673018628054; Fri, 06 Jan 2023 07:23:48 -0800 (PST) Received: from zen.linaroharston ([185.81.254.11]) by smtp.gmail.com with ESMTPSA id m35-20020a05600c3b2300b003d973e939d3sm2378141wms.1.2023.01.06.07.23.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Jan 2023 07:23:47 -0800 (PST) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 20EC31FFB7; Fri, 6 Jan 2023 15:23:47 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Philippe_M?= =?utf-8?q?athieu-Daud=C3=A9?= , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal Subject: [PATCH v2] scripts/ci: update gitlab-runner playbook to use latest runner Date: Fri, 6 Jan 2023 15:23:38 +0000 Message-Id: <20230106152338.2599827-1-alex.bennee@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::336; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org We were using quite and old runner on our machines and running into issues with stalling jobs. Gitlab in the meantime now reliably provide the latest packaged versions of the runner under a stable URL. This update: - creates a per-arch subdir for builds - switches from binary tarballs to deb packages - re-uses the same binary for the secondary runner - updates distro check for second to 22.04 Note this script isn't fully idempotent as we end up accumulating runners especially during testing. However we also want to be able to run twice with different GitLab keys (e.g. project and personal) so I think we just have to be mindful of that during testing. Signed-off-by: Alex Bennée --- v2 - only register aarch32 runner, move service start post both registers - tested on s390x --- scripts/ci/setup/gitlab-runner.yml | 56 +++++++----------------------- scripts/ci/setup/vars.yml.template | 2 -- 2 files changed, 13 insertions(+), 45 deletions(-) diff --git a/scripts/ci/setup/gitlab-runner.yml b/scripts/ci/setup/gitlab-runner.yml index 33128be85d..95d4199c03 100644 --- a/scripts/ci/setup/gitlab-runner.yml +++ b/scripts/ci/setup/gitlab-runner.yml @@ -50,60 +50,30 @@ - name: Download the matching gitlab-runner get_url: - dest: /usr/local/bin/gitlab-runner - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" - owner: gitlab-runner - group: gitlab-runner - mode: u=rwx,g=rwx,o=rx - - - name: Register the gitlab-runner - command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" - - - name: Install the gitlab-runner service using its own functionality - command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner - register: gitlab_runner_install_service_result - failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + dest: "/root/" + url: "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_{{ gitlab_runner_arch }}.deb" - - name: Enable the gitlab-runner service - service: - name: gitlab-runner - state: started - enabled: yes + - name: Install gitlab-runner via package manager + apt: deb="/root/gitlab-runner_{{ gitlab_runner_arch }}.deb" - - name: Download secondary gitlab-runner - get_url: - dest: /usr/local/bin/gitlab-runner-arm - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-arm" - owner: gitlab-runner - group: gitlab-runner - mode: u=rwx,g=rwx,o=rx - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' + - name: Register the gitlab-runner + command: "/usr/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + # The secondary runner will still run under the single gitlab-runner service - name: Register secondary gitlab-runner - command: "/usr/local/bin/gitlab-runner-arm register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + command: "/usr/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" when: - ansible_facts['distribution'] == 'Ubuntu' - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' + - ansible_facts['distribution_version'] == '22.04' - - name: Install the secondary gitlab-runner service using its own functionality - command: /usr/local/bin/gitlab-runner-arm install --user gitlab-runner --working-directory /home/gitlab-runner/arm -n gitlab-runner-arm + - name: Install the gitlab-runner service using its own functionality + command: "/usr/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner" register: gitlab_runner_install_service_result failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' - - name: Enable the secondary gitlab-runner service + - name: Enable the gitlab-runner service service: - name: gitlab-runner-arm + name: gitlab-runner state: started enabled: yes - when: - - ansible_facts['distribution'] == 'Ubuntu' - - ansible_facts['architecture'] == 'aarch64' - - ansible_facts['distribution_version'] == '20.04' diff --git a/scripts/ci/setup/vars.yml.template b/scripts/ci/setup/vars.yml.template index e48089761f..4b355fb80f 100644 --- a/scripts/ci/setup/vars.yml.template +++ b/scripts/ci/setup/vars.yml.template @@ -1,5 +1,3 @@ -# The version of the gitlab-runner to use -gitlab_runner_version: 13.12.0 # The URL of the gitlab server to use, usually https://gitlab.com unless you're # using a private GitLab instance gitlab_runner_server_url: https://gitlab.com