From patchwork Tue Sep 10 16:41:25 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Baker X-Patchwork-Id: 19931 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ye0-f200.google.com (mail-ye0-f200.google.com [209.85.213.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id F2ED925A72 for ; Tue, 10 Sep 2013 16:41:27 +0000 (UTC) Received: by mail-ye0-f200.google.com with SMTP id r3sf8573659yen.7 for ; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:mime-version:to:from:subject :message-id:date:reply-to:sender:errors-to:precedence :x-original-sender:x-original-authentication-results:mailing-list :list-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=hpIF/oUKzDo+e9/Fg/vQJfua7HvxJmolME54zYreuLo=; b=l4TWvwGE3H/gqQFBrIo7ZurSc/t+KG/X7OJmN8kEVGciROjVBkNGukxBfWWnHDP7hC qh+LgZQA4Kl3wL0n3RAMSxR4kmdfjm+x+JD0THlYGBadECM7JkHiI55pfySdYsG+zJ68 T6FiDrlSQQ5MmTDSaRLGcI9W541LqS7Hk1YHzidDukdag3WuhT4vmjTxqQDZkXWnFaGX C0GEzWnyl0rYMrbRj4NPIah4GZyfjl0oURqPeZ4GXdilsAFSRhEtTsB2F721uTCyXhwy F/6U3q+ZuiHu4QdtO1QZxwy7rEbYtcl+vt7VZ+qnsS+cXzs6dgKtYw9F+4xABMi4VpGN 2pmA== X-Received: by 10.236.167.138 with SMTP id i10mr9260386yhl.9.1378831287497; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.2.166 with SMTP id 6ls939264qev.69.gmail; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) X-Received: by 10.52.114.231 with SMTP id jj7mr19894234vdb.2.1378831287220; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) Received: from mail-vc0-f174.google.com (mail-vc0-f174.google.com [209.85.220.174]) by mx.google.com with ESMTPS id tj1si4656794vdc.40.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 10 Sep 2013 09:41:27 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.174; Received: by mail-vc0-f174.google.com with SMTP id gd11so5066744vcb.19 for ; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) X-Gm-Message-State: ALoCoQmEO5iYFcalw57rJFjpqlIFET4LHdfh/uOcJxZkLSETnyiJDALJZn751PWpoI1TR6VXoaIo X-Received: by 10.52.98.7 with SMTP id ee7mr986071vdb.37.1378831287033; Tue, 10 Sep 2013 09:41:27 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp9373vcz; Tue, 10 Sep 2013 09:41:26 -0700 (PDT) X-Received: by 10.180.81.71 with SMTP id y7mr9726310wix.63.1378831285894; Tue, 10 Sep 2013 09:41:25 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id t6si7477066wjq.89.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 10 Sep 2013 09:41:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1VJR0H-00045J-4E for ; Tue, 10 Sep 2013 16:41:25 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 14383F0949 for ; Tue, 10 Sep 2013 16:41:25 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 677 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 677: Consolidate busybox httpd logic Message-Id: <20130910164125.4261.8616.launchpad@ackee.canonical.com> Date: Tue, 10 Sep 2013 16:41:25 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: list X-Generated-By: Launchpad (canonical.com); Revision="16761"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 9296b0f7bb9c831cb49888a8ba7e7cd4b207213a X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: noreply@launchpad.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Merge authors: Tyler Baker (tyler-baker) Related merge proposals: https://code.launchpad.net/~tyler-baker/lava-dispatcher/consolidate-busybox-httpd/+merge/184403 proposed by: Tyler Baker (tyler-baker) review: Approve - Antonio Terceiro (terceiro) ------------------------------------------------------------ revno: 677 [merge] committer: Tyler Baker branch nick: lava-dispatcher timestamp: Tue 2013-09-10 09:41:00 -0700 message: Consolidate busybox httpd logic modified: lava_dispatcher/device/bootloader.py lava_dispatcher/device/ipmi_pxe.py lava_dispatcher/device/target.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/device/bootloader.py' --- lava_dispatcher/device/bootloader.py 2013-08-30 22:15:05 +0000 +++ lava_dispatcher/device/bootloader.py 2013-09-06 22:37:31 +0000 @@ -51,7 +51,6 @@ self._boot_cmds = None self._lava_cmds = None self._uboot_boot = False - self._http_pid = None # This is the offset into the path, used to reference bootfiles self._offset = self.scratch_dir.index('images') @@ -115,11 +114,13 @@ def deploy_linaro(self, hwpack, rfs, bootloadertype): self._uboot_boot = False - super(BootloaderTarget, self).deploy_linaro(hwpack, rfs, bootloadertype) + super(BootloaderTarget, self).deploy_linaro(hwpack, rfs, + bootloadertype) def deploy_linaro_prebuilt(self, image, bootloadertype): self._uboot_boot = False - super(BootloaderTarget, self).deploy_linaro_prebuilt(image, bootloadertype) + super(BootloaderTarget, self).deploy_linaro_prebuilt(image, + bootloadertype) def _inject_boot_cmds(self): if self._is_job_defined_boot_cmds(self.config.boot_cmds): @@ -164,28 +165,6 @@ else: super(BootloaderTarget, self)._boot_linaro_image() - def start_http_server(self, runner, ip): - if self._http_pid is not None: - raise OperationFailed("busybox httpd already running with pid %d" - % self._http_pid) - # busybox produces no output to parse for, - # so run it in the bg and get its pid - runner.run('busybox httpd -f &') - runner.run('echo pid:$!:pid', response="pid:(\d+):pid", timeout=10) - if runner.match_id != 0: - raise OperationFailed("busybox httpd did not start") - else: - self._http_pid = runner.match.group(1) - url_base = "http://%s" % ip - return url_base - - def stop_http_server(self, runner): - if self._http_pid is None: - raise OperationFailed("busybox httpd not running, \ - but stop_http_server called.") - runner.run('kill %s' % self._http_pid) - self._http_pid = None - @contextlib.contextmanager def file_system(self, partition, directory): if self._uboot_boot: @@ -202,7 +181,7 @@ runner.run('cd /tmp') # need to be in same dir as fs.tgz ip = runner.get_target_ip() - url_base = self.start_http_server(runner, ip) + url_base = self._start_busybox_http_server(runner, ip) url = url_base + '/fs.tgz' logging.info("Fetching url: %s" % url) @@ -226,32 +205,10 @@ runner.run('rm -rf %s' % targetdir) self._target_extract(runner, tf, parent_dir) finally: - self.stop_http_server(runner) + self._stop_busybox_http_server(runner) else: with super(BootloaderTarget, self).file_system( partition, directory) as path: yield path - def _target_extract(self, runner, tar_file, dest, timeout=-1): - tmpdir = self.context.config.lava_image_tmpdir - url = self.context.config.lava_image_url - tar_file = tar_file.replace(tmpdir, '') - tar_url = '/'.join(u.strip('/') for u in [url, tar_file]) - self._target_extract_url(runner, tar_url, dest, timeout=timeout) - - def _target_extract_url(self, runner, tar_url, dest, timeout=-1): - decompression_cmd = '' - if tar_url.endswith('.gz') or tar_url.endswith('.tgz'): - decompression_cmd = '| /bin/gzip -dc' - elif tar_url.endswith('.bz2'): - decompression_cmd = '| /bin/bzip2 -dc' - elif tar_url.endswith('.tar'): - decompression_cmd = '' - else: - raise RuntimeError('bad file extension: %s' % tar_url) - - runner.run('wget -O - %s %s | /bin/tar -C %s -xmf -' - % (tar_url, decompression_cmd, dest), - timeout=timeout) - target_class = BootloaderTarget === modified file 'lava_dispatcher/device/ipmi_pxe.py' --- lava_dispatcher/device/ipmi_pxe.py 2013-08-30 22:15:05 +0000 +++ lava_dispatcher/device/ipmi_pxe.py 2013-09-06 22:37:31 +0000 @@ -32,7 +32,6 @@ ) from lava_dispatcher.errors import ( CriticalError, - OperationFailed, ) from lava_dispatcher.downloader import ( download_image, @@ -176,7 +175,8 @@ runner.run('/bin/tar -cmzf /tmp/fs.tgz -C %s %s' % (parent_dir, target_name)) runner.run('cd /tmp') # need to be in same dir as fs.tgz - url_base = runner.start_http_server() + ip = runner.get_target_ip() + url_base = self._start_busybox_http_server(runner, ip) url = url_base + '/fs.tgz' logging.info("Fetching url: %s" % url) @@ -200,38 +200,16 @@ self._target_extract(runner, tf, parent_dir) finally: - runner.stop_http_server() + self._stop_busybox_http_server() runner.run('umount /mnt') - def _target_extract(self, runner, tar_file, dest, timeout=-1): - tmpdir = self.context.config.lava_image_tmpdir - url = self.context.config.lava_image_url - tar_file = tar_file.replace(tmpdir, '') - tar_url = '/'.join(u.strip('/') for u in [url, tar_file]) - self._target_extract_url(runner, tar_url, dest, timeout=timeout) - - def _target_extract_url(self, runner, tar_url, dest, timeout=-1): - decompression_cmd = '' - if tar_url.endswith('.gz') or tar_url.endswith('.tgz'): - decompression_cmd = '| /bin/gzip -dc' - elif tar_url.endswith('.bz2'): - decompression_cmd = '| /bin/bzip2 -dc' - elif tar_url.endswith('.tar'): - decompression_cmd = '' - else: - raise RuntimeError('bad file extension: %s' % tar_url) - - runner.run('wget -O - %s %s | /bin/tar -C %s -xmf -' - % (tar_url, decompression_cmd, dest), - timeout=timeout) - @contextlib.contextmanager def _as_master(self): self.bootcontrol.power_on_boot_master() self.proc.expect("\(initramfs\)") self.proc.sendline('export PS1="%s"' % self.MASTER_PS1) self.proc.expect(self.MASTER_PS1_PATTERN, timeout=180, lava_no_logging=1) - runner = BusyboxHttpdMasterCommandRunner(self) + runner = MasterCommandRunner(self) runner.run(". /scripts/functions") runner.run("DEVICE=%s configure_networking" % @@ -257,32 +235,3 @@ target_class = IpmiPxeTarget - - -class BusyboxHttpdMasterCommandRunner(MasterCommandRunner): - """A CommandRunner to use when the target is booted into the master image. - """ - http_pid = None - - def __init__(self, target): - super(BusyboxHttpdMasterCommandRunner, self).__init__(target) - - def start_http_server(self): - master_ip = self.get_master_ip() - if self.http_pid is not None: - raise OperationFailed("busybox httpd already running with pid %d" % self.http_pid) - # busybox produces no output to parse for, so run it in the bg and get its pid - self.run('busybox httpd -f &') - self.run('echo pid:$!:pid', response="pid:(\d+):pid", timeout=10) - if self.match_id != 0: - raise OperationFailed("busybox httpd did not start") - else: - self.http_pid = self.match.group(1) - url_base = "http://%s" % master_ip - return url_base - - def stop_http_server(self): - if self.http_pid is None: - raise OperationFailed("busybox httpd not running, but stop_http_server called.") - self.run('kill %s' % self.http_pid) - self.http_pid = None === modified file 'lava_dispatcher/device/target.py' --- lava_dispatcher/device/target.py 2013-08-30 22:15:05 +0000 +++ lava_dispatcher/device/target.py 2013-09-10 16:30:04 +0000 @@ -191,13 +191,15 @@ def _customize_bootloader(self, connection, boot_cmds): for line in boot_cmds: - parts = re.match('^(?Psendline|expect)\s*(?P.*)', line) + parts = re.match('^(?Psendline|expect)\s*(?P.*)', + line) if parts: try: action = parts.group('action') command = parts.group('command') except AttributeError as e: - raise Exception("Badly formatted command in boot_cmds %s" % e) + raise Exception("Badly formatted command in \ + boot_cmds %s" % e) if action == "sendline": connection.send(command) connection.sendline('') @@ -205,8 +207,41 @@ command = re.escape(command) connection.expect(command, timeout=300) else: - self._wait_for_prompt(connection, self.config.bootloader_prompt, timeout=300) - connection.sendline(line) + self._wait_for_prompt(connection, + self.config.bootloader_prompt, + timeout=300) + connection.sendline(line) + + def _target_extract(self, runner, tar_file, dest, timeout=-1): + tmpdir = self.context.config.lava_image_tmpdir + url = self.context.config.lava_image_url + tar_file = tar_file.replace(tmpdir, '') + tar_url = '/'.join(u.strip('/') for u in [url, tar_file]) + self._target_extract_url(runner, tar_url, dest, timeout=timeout) + + def _target_extract_url(self, runner, tar_url, dest, timeout=-1): + decompression_cmd = '' + if tar_url.endswith('.gz') or tar_url.endswith('.tgz'): + decompression_cmd = '| /bin/gzip -dc' + elif tar_url.endswith('.bz2'): + decompression_cmd = '| /bin/bzip2 -dc' + elif tar_url.endswith('.tar'): + decompression_cmd = '' + else: + raise RuntimeError('bad file extension: %s' % tar_url) + + runner.run('wget -O - %s %s | /bin/tar -C %s -xmf -' + % (tar_url, decompression_cmd, dest), + timeout=timeout) + + def _start_busybox_http_server(self, runner, ip): + runner.run('busybox httpd -f &') + runner.run('echo $! > /tmp/httpd.pid') + url_base = "http://%s" % ip + return url_base + + def _stop_busybox_http_server(self, runner): + runner.run('kill `cat /tmp/httpd.pid`') def _customize_ubuntu(self, rootdir): self.deployment_data = Target.ubuntu_deployment_data