From patchwork Thu Oct 20 14:52:10 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Larson X-Patchwork-Id: 4748 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id CFB4623E03 for ; Thu, 20 Oct 2011 14:52:16 +0000 (UTC) Received: from mail-bw0-f52.google.com (mail-bw0-f52.google.com [209.85.214.52]) by fiordland.canonical.com (Postfix) with ESMTP id A8825A18154 for ; Thu, 20 Oct 2011 14:52:16 +0000 (UTC) Received: by bkbzs2 with SMTP id zs2so5051617bkb.11 for ; Thu, 20 Oct 2011 07:52:16 -0700 (PDT) Received: by 10.223.4.215 with SMTP id 23mr18702829fas.8.1319122336285; Thu, 20 Oct 2011 07:52:16 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.152.1.71 with SMTP id 7cs125178lak; Thu, 20 Oct 2011 07:52:16 -0700 (PDT) Received: by 10.216.203.69 with SMTP id e47mr8515590weo.60.1319122331190; Thu, 20 Oct 2011 07:52:11 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id w2si1047101wec.49.2011.10.20.07.52.10 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 20 Oct 2011 07:52:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1RGtyc-0000lF-Lb for ; Thu, 20 Oct 2011 14:52:10 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 9AEC8FB1D6 for ; Thu, 20 Oct 2011 14:52:10 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 142 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 142: Fix an issue with a severely long delay before booting the master image Message-Id: <20111020145210.20965.75217.launchpad@ackee.canonical.com> Date: Thu, 20 Oct 2011 14:52:10 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14165"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: ef624853f917650f3a660755d5e66ef31d7a32c4 Merge authors: Paul Larson (pwlars) Related merge proposals: https://code.launchpad.net/~pwlars/lava-dispatcher/in_master_shell_timeout/+merge/79893 proposed by: Paul Larson (pwlars) review: Needs Fixing - Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 142 [merge] committer: Paul Larson branch nick: lava-dispatcher timestamp: Thu 2011-10-20 09:49:29 -0500 message: Fix an issue with a severely long delay before booting the master image to collect results modified: lava_dispatcher/actions/launch_control.py lava_dispatcher/client.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/launch_control.py' --- lava_dispatcher/actions/launch_control.py 2011-10-18 02:38:34 +0000 +++ lava_dispatcher/actions/launch_control.py 2011-10-20 14:48:09 +0000 @@ -106,7 +106,9 @@ """ client = self.client try: - self.in_master_shell() + client.in_master_shell() + except OperationFailed: + client.boot_master_image() except: logging.exception("in_master_shell failed") client.boot_master_image() @@ -154,11 +156,10 @@ while time.time() < now + timeout: try: result_path = download(result_tarball, tarball_dir) - except: + except RuntimeError: if time.time() >= now + timeout: + logging.exception("download failed") raise - else: - logging.exception("download failed, retrying") except: logging.warning(traceback.format_exc()) status = 'fail' === modified file 'lava_dispatcher/client.py' --- lava_dispatcher/client.py 2011-10-20 04:09:15 +0000 +++ lava_dispatcher/client.py 2011-10-20 14:49:29 +0000 @@ -76,11 +76,12 @@ def default_network_interface(self): return self.device_option("default_network_interface") - def in_master_shell(self): + def in_master_shell(self, timeout=10): """ Check that we are in a shell on the master image """ self.proc.sendline("") - id = self.proc.expect([self.master_str, pexpect.TIMEOUT]) + id = self.proc.expect([self.master_str, pexpect.TIMEOUT], + timeout=timeout) if id == 1: raise OperationFailed @@ -98,11 +99,11 @@ self.soft_reboot() try: self.proc.expect("Starting kernel") - self.in_master_shell() + self.in_master_shell(120) except: logging.exception("in_master_shell failed") self.hard_reboot() - self.in_master_shell() + self.in_master_shell(300) self.proc.sendline('export PS1="$PS1 [rc=$(echo \$?)]: "') self.proc.expect(self.master_str) @@ -148,7 +149,7 @@ # XXX Workaround for snowball if self.device_type == "snowball_sd": time.sleep(10) - self.in_master_shell() + self.in_master_shell(300) # Intentionally avoid self.soft_reboot() to prevent looping self.proc.sendline("reboot") self.enter_uboot()