From patchwork Tue May 20 10:54:36 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 30417 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pd0-f200.google.com (mail-pd0-f200.google.com [209.85.192.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2197B20671 for ; Tue, 20 May 2014 11:14:14 +0000 (UTC) Received: by mail-pd0-f200.google.com with SMTP id g10sf820754pdj.11 for ; Tue, 20 May 2014 04:14:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id :mime-version:cc:subject:precedence:list-id:list-unsubscribe :list-post:list-help:list-subscribe:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :list-archive:content-type:content-transfer-encoding; bh=rh4766G1NkHmLIYZF8eTv+E3RygH0uIOyeI/lrJxY9s=; b=g3cCTnnFb8CMWEbrMyJQD0zlUofLtHY2i/goMHB8/fveO3KAEha4EpzQlkgyB9OzxE 8C+3GMzuBlYKbzpm4m1y9fTaI94DuLI7W4vi5HdmddKYXFOtKv2Rvxt1kPCLUQAqDmfd VNRESqPAUpEu0b2mS/5rT+sSLXrSlC7Qp0+BTAtFdJPn3bYkBoKRB30dUUJiVeywyRXj ZzWVp/31mXSuNP9U7PDnq5Mx7iLbVzlK9xESkM/yYnoIZ3a5GFJ9JwzuAV2tp5VKoIhd x7FlyG+MmAprhGvWhlZ8SY18Jn1Ob+KH4RgKAbF7LoZvMYwJbmW3uWN2K6JRND89yRSG ywzg== X-Gm-Message-State: ALoCoQkcr1zvmHETq2d0fHi2OE5KP/W93HUqz1OiU4ZwMIHBZL7CxcqTjeuk5esN+mJGDiAz/BdU X-Received: by 10.66.66.163 with SMTP id g3mr18466810pat.3.1400584454185; Tue, 20 May 2014 04:14:14 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.91.99 with SMTP id y90ls245119qgd.82.gmail; Tue, 20 May 2014 04:14:14 -0700 (PDT) X-Received: by 10.58.39.98 with SMTP id o2mr20233vek.75.1400584454002; Tue, 20 May 2014 04:14:14 -0700 (PDT) Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169]) by mx.google.com with ESMTPS id lu8si481431veb.157.2014.05.20.04.14.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 20 May 2014 04:14:13 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.169 as permitted sender) client-ip=209.85.128.169; Received: by mail-ve0-f169.google.com with SMTP id jx11so375906veb.28 for ; Tue, 20 May 2014 04:14:13 -0700 (PDT) X-Received: by 10.220.69.72 with SMTP id y8mr3089451vci.21.1400584453857; Tue, 20 May 2014 04:14:13 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp21276vcb; Tue, 20 May 2014 04:14:13 -0700 (PDT) X-Received: by 10.140.47.167 with SMTP id m36mr55863089qga.21.1400584453082; Tue, 20 May 2014 04:14:13 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id v6si10327132qas.68.2014.05.20.04.14.12 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 20 May 2014 04:14:13 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Wmhxf-0002sa-TJ; Tue, 20 May 2014 11:11:59 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Wmhxe-0002sV-3w for xen-devel@lists.xen.org; Tue, 20 May 2014 11:11:58 +0000 Received: from [85.158.143.35:30272] by server-1.bemta-4.messagelabs.com id 71/56-09853-D783B735; Tue, 20 May 2014 11:11:57 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-10.tower-21.messagelabs.com!1400584315!6053004!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 25304 invoked from network); 20 May 2014 11:11:56 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-10.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 20 May 2014 11:11:56 -0000 X-IronPort-AV: E=Sophos;i="4.98,873,1392163200"; d="scan'208";a="132852746" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 20 May 2014 11:11:24 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Tue, 20 May 2014 07:11:23 -0400 Received: from kazak.uk.xensource.com ([10.80.2.80] helo=zakaz.uk.xensource.com) by ukmail1.uk.xensource.com with smtp (Exim 4.69) (envelope-from ) id 1Wmhgr-0001YO-2j; Tue, 20 May 2014 11:54:38 +0100 Received: by zakaz.uk.xensource.com (sSMTP sendmail emulation); Tue, 20 May 2014 11:54:37 +0100 From: Ian Campbell To: , Date: Tue, 20 May 2014 11:54:36 +0100 Message-ID: <1400583276-6873-1-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.9.0 MIME-Version: 1.0 X-DLP: MIA1 Cc: Ian Campbell Subject: [Xen-devel] [PATCH OSSTEST] Tollerate outages of git://git.sv.gnu.org/gnulib.git X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.169 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: gnulib is a bit special in that it isn't part of a push gate and therefore doesn't have an output repo on xenbits. This means that if git.sv.gnu.org is down for some reason then any flight which uses it will potentially fail, even ones which are not specifically trying to test gnulib.git. Normally unrelated flights are isolated from 3rd party failures because they use the relevant push gates' outputs. The git-proxy which we use supports passing [fetch=try] as an option which will cause it to try and fetch from upstream and fall back to the cached version if upstream does not respond. This is exactly what we want for gnulib.git. Proxy options in the URL are separate with a space which we encode here as %20 to avoid having to go around adding quotes to all the uses of shell variables which contain a URL. Provide a helper shell function cached_repo which conditionally arranges for the given repo to be cached with the given options only if the osstest config sets GitCacheProxy, so as not to break uses of osstest outside of the production environment where a cache may not be available (of course such uses will then be subject to outages of 3rd party repos) It was also necessary to make repo_tree_rev_fetch_git idempotent WRT the proxy prefix to avoid trying to clone git://proxy/git://proxy/git://some.real.domain/foo.git. Signed-off-by: Ian Campbell --- README.dev | 6 +++++- ap-common | 2 +- cri-common | 17 ++++++++++++++++- 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/README.dev b/README.dev index 68fbe67..fd09766 100644 --- a/README.dev +++ b/README.dev @@ -21,7 +21,11 @@ $ ./mg-allocate \!marilith-n5 Removing machines for servicing/outage ====================================== -mg-blockage +mg-blockage [] +OSSTEST_CONFIG=production-config ./mg-blockage '2014-05-21 0955' '2014-05-21 1400' equiv-marilith + + +Keeps running for the duration, so run it in a screen on the osstest VM. Commisioning a new machine ========================== diff --git a/ap-common b/ap-common index 537f605..b4b3310 100644 --- a/ap-common +++ b/ap-common @@ -37,7 +37,7 @@ : ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git} : ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git} -: ${TREE_GNULIB_LIBVIRT:=git://git.sv.gnu.org/gnulib.git} +: ${TREE_GNULIB_LIBVIRT:=$(cached_repo git://git.sv.gnu.org/gnulib.git \[fetch=try\])} : ${TREE_RUMPUSERXEN:=https://github.com/rumpkernel/rumpuser-xen} : ${TREEVCS_RUMPUSERXEN:=git} diff --git a/cri-common b/cri-common index ea312f2..9cbf0d8 100644 --- a/cri-common +++ b/cri-common @@ -21,14 +21,29 @@ umask 002 +cached_repo () { + local repo=$1 + local options=$2 + local cache=`getconfig GitCacheProxy` + + if [ x$cache != x ] ; then + echo "$cache$repo%20$options" + else + echo "$repo" + fi +} + repo_tree_rev_fetch_git () { local treename=$1 local remoteurl=$2 local remotetag=$3 local localtag=$4 + local proxy=`getconfig GitCacheProxy` case $remoteurl in + $proxy*) + local realurl="$remoteurl" ;; git://*|http://*|https://*) - local realurl="`getconfig GitCacheProxy`$remoteurl" ;; + local realurl="$proxy$remoteurl" ;; *) local realurl="$remoteurl" ;; esac