From patchwork Tue Aug 27 16:33:23 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyler Baker X-Patchwork-Id: 19549 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-gh0-f200.google.com (mail-gh0-f200.google.com [209.85.160.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 3F26A248E8 for ; Tue, 27 Aug 2013 16:33:26 +0000 (UTC) Received: by mail-gh0-f200.google.com with SMTP id 10sf5171842ghy.3 for ; Tue, 27 Aug 2013 09:33:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:delivered-to:mime-version:to:from:subject :message-id:date:reply-to:sender:errors-to:precedence :x-original-sender:x-original-authentication-results:mailing-list :list-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=uDaA+EeQlm7Dlf8bnd3/c8cX7r3Xi0nd3JqfLSfg7uw=; b=TySoFmysjr922d8KpFjo6YfdyhXeIYnbUGvTWw1lsPE7YYHSYD4aNOupSoO4L1FiCz pUCMXapVnhuwqBceq5et/T0N5C+4zWIjZU+Ulg//Nf5yIOjkhAM5R4SamQggQzS+s5q2 qdcMazVVLRtT70E2cHi1zu+/42Dsad56ISoEfsc5IfmZDKe8nhG2U5Kk2s1WformJs2A I/1MGiytvPxhr+mkFpgVYqABq1WjZDlsF6AJQVKBNWlCsYUACsduHhwVHREHf8s1r766 NNP3TLCj3lrlC6cQo2EMiQ5zlT59qIT+6HX3CKh/zOPEGehG+90bkdbrHq2uy25zNGAj NKOQ== X-Received: by 10.236.14.100 with SMTP id c64mr8134056yhc.38.1377621205781; Tue, 27 Aug 2013 09:33:25 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.71.168 with SMTP id w8ls2798728qeu.64.gmail; Tue, 27 Aug 2013 09:33:25 -0700 (PDT) X-Received: by 10.58.165.70 with SMTP id yw6mr12020558veb.19.1377621205660; Tue, 27 Aug 2013 09:33:25 -0700 (PDT) Received: from mail-vb0-f44.google.com (mail-vb0-f44.google.com [209.85.212.44]) by mx.google.com with ESMTPS id o7si5108907vcl.60.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 27 Aug 2013 09:33:25 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.212.44 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.44; Received: by mail-vb0-f44.google.com with SMTP id e13so3315030vbg.31 for ; Tue, 27 Aug 2013 09:33:25 -0700 (PDT) X-Gm-Message-State: ALoCoQmH7JPFVbZ3AGLhQ8c3UV7vHGQDzjsc6MhiEhgLQapQP2CcKTaB6Masr6+2eP7BlduRr/1R X-Received: by 10.221.51.206 with SMTP id vj14mr21068499vcb.17.1377621205334; Tue, 27 Aug 2013 09:33:25 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp305166vcz; Tue, 27 Aug 2013 09:33:24 -0700 (PDT) X-Received: by 10.180.185.146 with SMTP id fc18mr12241505wic.44.1377621203924; Tue, 27 Aug 2013 09:33:23 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id ne11si6382365wic.3.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 27 Aug 2013 09:33:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1VEMCp-0000IX-Aj for ; Tue, 27 Aug 2013 16:33:23 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 3E431E0616 for ; Tue, 27 Aug 2013 16:33:23 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 657 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 657: Implemented deploy_linaro_kernel in qemu class. Refactored dispatcher code for easier reuse. Adde... Message-Id: <20130827163323.3970.62434.launchpad@ackee.canonical.com> Date: Tue, 27 Aug 2013 16:33:23 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: list X-Generated-By: Launchpad (canonical.com); Revision="16738"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 255af916b3f7004e9dea247ec850e24a56b8f97f X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: noreply@launchpad.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.44 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Merge authors: Tyler Baker (tyler-baker) Related merge proposals: https://code.launchpad.net/~tyler-baker/lava-dispatcher/bootloader-refactor/+merge/181701 proposed by: Tyler Baker (tyler-baker) review: Approve - Antonio Terceiro (terceiro) ------------------------------------------------------------ revno: 657 [merge] committer: Tyler Baker branch nick: lava-dispatcher-tip timestamp: Tue 2013-08-27 09:20:42 -0700 message: Implemented deploy_linaro_kernel in qemu class. Refactored dispatcher code for easier reuse. Added lava-test-shell ability to bootloader targets. removed: lava_dispatcher/default-config/lava-dispatcher/device-types/arndale-uefi.conf lava_dispatcher/device/uefi.py modified: lava_dispatcher/actions/boot_control.py lava_dispatcher/actions/deploy.py lava_dispatcher/client/base.py lava_dispatcher/client/targetdevice.py lava_dispatcher/default-config/lava-dispatcher/device-types/arndale.conf lava_dispatcher/device/bootloader.py lava_dispatcher/device/fastmodel.py lava_dispatcher/device/master.py lava_dispatcher/device/qemu.py lava_dispatcher/device/target.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/boot_control.py' --- lava_dispatcher/actions/boot_control.py 2013-08-09 19:44:35 +0000 +++ lava_dispatcher/actions/boot_control.py 2013-08-27 15:49:57 +0000 @@ -49,12 +49,17 @@ parameters_schema['properties']['wait_for_home_screen'] = { 'default': False, 'optional': True } + parameters_schema['properties']['interactive_boot_cmds'] = { + 'default': False, 'optional': True + } - def run(self, options=None, adb_check=False, wait_for_home_screen=True): - if not options: - options = [] + def run(self, options=[], adb_check=False, + wait_for_home_screen=True, interactive_boot_cmds=False): client = self.client - client.target_device.boot_options = options + if interactive_boot_cmds: + client.config.boot_cmds = options + else: + client.target_device.boot_options = options client.config.android_wait_for_home_screen = wait_for_home_screen try: client.boot_linaro_android_image( === modified file 'lava_dispatcher/actions/deploy.py' --- lava_dispatcher/actions/deploy.py 2013-08-21 02:55:19 +0000 +++ lava_dispatcher/actions/deploy.py 2013-08-23 03:48:42 +0000 @@ -100,8 +100,11 @@ 'ramdisk': {'type': 'string', 'optional': True}, 'dtb': {'type': 'string', 'optional': True}, 'rootfs': {'type': 'string', 'optional': True}, + 'bootloader': {'type': 'string', 'optional': True}, + 'firmware': {'type': 'string', 'optional': True}, 'rootfstype': {'type': 'string', 'optional': True}, - 'bootloader': {'type': 'string', 'optional': True, 'default': 'u_boot'}, + 'bootloadertype': {'type': 'string', 'optional': True, 'default': 'u_boot'}, + 'role': {'type': 'string', 'optional': True}, }, 'additionalProperties': False, } @@ -112,11 +115,12 @@ if 'kernel' not in parameters: raise ValueError('must specify a kernel') - def run(self, kernel=None, ramdisk=None, dtb=None, rootfs=None, rootfstype='ext4', - bootloader='u_boot'): + def run(self, kernel=None, ramdisk=None, dtb=None, rootfs=None, bootloader=None, + firmware=None, rootfstype='ext4', bootloadertype='u_boot'): self.client.deploy_linaro_kernel( - kernel=kernel, ramdisk=ramdisk, dtb=dtb, rootfs=rootfs, - rootfstype=rootfstype, bootloader=bootloader) + kernel=kernel, ramdisk=ramdisk, dtb=dtb, rootfs=rootfs, + bootloader=bootloader, firmware=firmware, rootfstype=rootfstype, + bootloadertype=bootloadertype) class cmd_dummy_deploy(BaseAction): === modified file 'lava_dispatcher/client/base.py' --- lava_dispatcher/client/base.py 2013-07-24 16:56:18 +0000 +++ lava_dispatcher/client/base.py 2013-08-20 17:19:11 +0000 @@ -149,6 +149,29 @@ prompt_str_includes_rc=prompt_str_includes_rc) self._client = client + def get_target_ip(self): + logging.info("Waiting for network to come up") + try: + self.wait_network_up(timeout=20) + except NetworkError: + logging.exception("Unable to reach LAVA server") + raise + + pattern1 = "<(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)>" + cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" + "awk '{print \"<\" $1 \">\"}'" % + self._client.config.default_network_interface) + self.run( + cmd, [pattern1, pexpect.EOF, pexpect.TIMEOUT], timeout=5) + if self.match_id != 0: + msg = "Unable to determine target image IP address" + logging.error(msg) + raise CriticalError(msg) + + ip = self.match.group(1) + logging.debug("Target image IP is %s" % ip) + return ip + def _check_network_up(self): """Internal function for checking network once.""" lava_server_ip = self._client.context.config.lava_server_ip === modified file 'lava_dispatcher/client/targetdevice.py' --- lava_dispatcher/client/targetdevice.py 2013-08-08 00:15:59 +0000 +++ lava_dispatcher/client/targetdevice.py 2013-08-20 03:10:37 +0000 @@ -65,9 +65,11 @@ self.target_device.deploy_linaro_prebuilt(image) def deploy_linaro_kernel(self, kernel, ramdisk=None, dtb=None, rootfs=None, - rootfstype='ext4', bootloader='u_boot'): + bootloader=None, firmware=None, rootfstype='ext4', + bootloadertype='u_boot'): self.target_device.deploy_linaro_kernel(kernel, ramdisk, dtb, rootfs, - bootloader) + bootloader, firmware, rootfstype, + bootloadertype) def _boot_linaro_image(self): if self.proc: === removed file 'lava_dispatcher/default-config/lava-dispatcher/device-types/arndale-uefi.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/arndale-uefi.conf 2013-08-14 21:48:44 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/arndale-uefi.conf 1970-01-01 00:00:00 +0000 @@ -1,54 +0,0 @@ -boot_part = 2 -root_part = 3 - -boot_cmds = sendline a, - expect Choice:, - sendline 1, - expect Select the Boot Device:, - sendline 2, - expect File path of the EFI Application or the kernel:, - sendline uImage, - expect Boot Type: [a] ATAGS, [g] Global FDT or [l] Local FDT? [a/g/l], - sendline g, - expect Add an initrd: [y/n], - sendline y, - expect File path of the initrd:, - sendline uInitrd, - expect Arguments to pass to the binary:, - sendline "root=/dev/mmcblk1p6 rw rootwait console=ttySAC2,115200n8 init --no-log", - expect Description for this new Entry:, - sendline Test Image, - expect Choice:, - sendline 5, - expect Start:, - sendline 2 - - -client_type = uefi - -boot_cmds_android = 3 - -interrupt_boot_prompt = The default boot selection will start in - -bootloader_prompt = Start: - -lmc_dev_arg = arndale - -# Original linaro-android-media-create generated Android system SD card layout -boot_part_android_org = 2 -sys_part_android_org = 3 -cache_part_android_org = 5 -data_part_android_org = 6 -sdcard_part_android_org = 7 - -# Android LAVA test image SD card layout -sys_part_android = 6 -sdcard_part_android = 7 -data_part_android = 7 - -boot_options = - boot_cmds - -[boot_cmds] -default = boot_cmds - === modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/arndale.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/arndale.conf 2013-08-16 18:43:12 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/arndale.conf 2013-08-21 23:19:48 +0000 @@ -21,7 +21,8 @@ sdcard_part_android = 7 data_part_android = 7 -image_boot_msg = Starting kernel +android_orig_block_device = mmcblk1 +android_lava_block_device = mmcblk1 boot_cmds_android = mmc init, mmc part 0, === modified file 'lava_dispatcher/device/bootloader.py' --- lava_dispatcher/device/bootloader.py 2013-08-12 02:44:22 +0000 +++ lava_dispatcher/device/bootloader.py 2013-08-27 15:49:57 +0000 @@ -22,19 +22,25 @@ import contextlib import time import os -import pexpect from lava_dispatcher.device.master import ( MasterImageTarget ) +from lava_dispatcher.client.base import ( + NetworkCommandRunner, +) from lava_dispatcher.utils import ( - string_to_list + string_to_list, + mk_targz, + rmtree, ) from lava_dispatcher.errors import ( - CriticalError + CriticalError, + OperationFailed, ) from lava_dispatcher.downloader import ( - download_image + download_image, + download_with_retry, ) class BootloaderTarget(MasterImageTarget): @@ -45,50 +51,67 @@ self._boot_cmds = None self._lava_cmds = None self._uboot_boot = False + self._http_pid = None # This is the offset into the path, used to reference bootfiles self._offset = self.scratch_dir.index('images') - def power_off(self, proc): - if self._uboot_boot: - if self.config.power_off_cmd: - self.context.run_command(self.config.power_off_cmd) + def deploy_linaro_kernel(self, kernel, ramdisk, dtb, rootfs, bootloader, + firmware, rootfstype, bootloadertype): + if bootloadertype == "u_boot": + # We assume we will be controlling u-boot + if kernel is not None: + # We have been passed kernel image, setup TFTP boot + self._uboot_boot = True + # We are not booted yet + self._booted = False + # We specify OE deployment data, vanilla as possible + self.deployment_data = self.target_map['oe'] + # Set the TFTP server IP (Dispatcher) + self._lava_cmds = "lava_server_ip=" + \ + self.context.config.lava_server_ip + "," + kernel = download_image(kernel, self.context, + self.scratch_dir, decompress=False) + self._lava_cmds += "lava_kernel=" + \ + kernel[self._offset::] + "," + if ramdisk is not None: + # We have been passed a ramdisk + ramdisk = download_image(ramdisk, self.context, + self.scratch_dir, + decompress=False) + self._lava_cmds += "lava_ramdisk=" + \ + ramdisk[self._offset::] + "," + if dtb is not None: + # We have been passed a device tree blob + dtb = download_image(dtb, self.context, + self.scratch_dir, decompress=False) + self._lava_cmds += "lava_dtb=" + dtb[self._offset::] + "," + if rootfs is not None: + # We have been passed a rootfs + rootfs = download_image(rootfs, self.context, + self.scratch_dir, decompress=False) + self._lava_cmds += "lava_rootfs=" + \ + rootfs[self._offset::] + "," + if bootloader is not None: + # We have been passed a bootloader + bootloader = download_image(bootloader, self.context, + self.scratch_dir, + decompress=False) + self._lava_cmds += "lava_bootloader=" + \ + bootloader[self._offset::] + "," + if firmware is not None: + # We have been passed firmware + firmware = download_image(firmware, self.context, + self.scratch_dir, + decompress=False) + self._lava_cmds += "lava_firmware=" + \ + firmware[self._offset::] + "," + else: + # This *should* never happen + raise CriticalError("No kernel images to boot") else: - super(BootloaderTarget, self).power_off(proc) - - def deploy_linaro_kernel(self, kernel, ramdisk, dtb, rootfs, bootloader): - if bootloader == "u_boot": - # We assume we will be controlling u-boot - if kernel is not None: - # We have been passed kernel image, setup TFTP boot - self._uboot_boot = True - # Set the TFTP server IP (Dispatcher) - self._lava_cmds = "lava_server_ip=" + self.context.config.lava_server_ip + "," - kernel = download_image(kernel, self.context, self.scratch_dir, decompress=False) - # Set the TFTP bootfile path for the kernel - self._lava_cmds += "lava_kernel=" + kernel[self._offset::] + "," - if ramdisk is not None: - # We have been passed a ramdisk - ramdisk = download_image(ramdisk, self.context, self.scratch_dir, decompress=False) - # Set the TFTP bootfile path for the ramdisk - self._lava_cmds += "lava_ramdisk=" + ramdisk[self._offset::] + "," - if dtb is not None: - # We have been passed a device tree blob - dtb = download_image(dtb, self.context, self.scratch_dir, decompress=False) - # Set the bootfile path for the ramdisk - self._lava_cmds += "lava_dtb=" + dtb[self._offset::] + "," - if rootfs is not None: - # We have been passed a rootfs - rootfs = download_image(rootfs, self.context, self.scratch_dir, decompress=True) - self._lava_cmds += "lava_rootfs=" + dtb[self._offset::] + "," - else: - # TODO: Faking the deployment data - Ubuntu - self.deployment_data = self.target_map['ubuntu'] - else: - # This *should* never happen - raise CriticalError("No kernel images to boot") - else: - # Define other "types" of bootloaders here. UEFI? Grub? - raise CriticalError("U-Boot is the only supported bootloader at this time") + # Define other "types" of bootloaders here. UEFI? Grub? + raise CriticalError("U-Boot is the only supported bootloader \ + at this time") def deploy_linaro(self, hwpack, rfs, bootloader): self._uboot_boot = False @@ -99,32 +122,136 @@ super(BootloaderTarget, self).deploy_linaro_prebuilt(image) def _inject_boot_cmds(self): - if isinstance(self.config.boot_cmds, basestring): + if self._is_job_defined_boot_cmds(self.config.boot_cmds): + logging.info('Overriding boot_cmds from job file') + self._boot_cmds = string_to_list(self._lava_cmds.encode('ascii')) \ + + self.config.boot_cmds + else: if self.config.boot_cmds_tftp is None: raise CriticalError("No TFTP boot commands defined") else: + logging.info('Loading boot_cmds from device configuration') self._boot_cmds = self._lava_cmds + self.config.boot_cmds_tftp - self._boot_cmds = string_to_list(self._boot_cmds.encode('ascii')) - else: - self._boot_cmds = string_to_list(self._lava_cmds.encode('ascii')) + self.config.boot_cmds + self._boot_cmds = string_to_list( + self._boot_cmds.encode('ascii')) def _run_boot(self): self._enter_bootloader(self.proc) self._inject_boot_cmds() self._customize_bootloader(self.proc, self._boot_cmds) - self._wait_for_prompt(self.proc, ['\(initramfs\)', self.config.master_str], - self.config.boot_linaro_timeout) + self.proc.expect(self.config.image_boot_msg, timeout=300) + self._wait_for_prompt(self.proc, ['\(initramfs\)', + self.config.master_str], + self.config.boot_linaro_timeout) def _boot_linaro_image(self): - if self._uboot_boot: - if self.config.hard_reset_command: - self._hard_reboot() - else: - raise CriticalError("No hard reset command defined") - self._run_boot() - self.proc.sendline('export PS1="%s"' % self.deployment_data['TESTER_PS1']) + if self._uboot_boot and not self._booted: + try: + if self.config.hard_reset_command: + self._hard_reboot() + self._run_boot() + else: + self._soft_reboot() + self._run_boot() + except: + raise OperationFailed("_run_boot failed") + self.proc.sendline('export PS1="%s"' + % self.deployment_data['TESTER_PS1']) self._booted = True + elif self._uboot_boot and self._booted: + self.proc.sendline('export PS1="%s"' + % self.deployment_data['TESTER_PS1']) else: super(BootloaderTarget, self)._boot_linaro_image() + def start_http_server(self, runner, ip): + if self._http_pid is not None: + raise OperationFailed("busybox httpd already running with pid %d" + % self._http_pid) + # busybox produces no output to parse for, + # so run it in the bg and get its pid + runner.run('busybox httpd -f &') + runner.run('echo pid:$!:pid', response="pid:(\d+):pid", timeout=10) + if runner.match_id != 0: + raise OperationFailed("busybox httpd did not start") + else: + self._http_pid = runner.match.group(1) + url_base = "http://%s" % ip + return url_base + + def stop_http_server(self, runner): + if self._http_pid is None: + raise OperationFailed("busybox httpd not running, \ + but stop_http_server called.") + runner.run('kill %s' % self._http_pid) + self._http_pid = None + + @contextlib.contextmanager + def file_system(self, partition, directory): + if self._uboot_boot: + try: + pat = self.deployment_data['TESTER_PS1_PATTERN'] + incrc = self.deployment_data['TESTER_PS1_INCLUDES_RC'] + runner = NetworkCommandRunner(self, pat, incrc) + + targetdir = '/%s' % directory + runner.run('mkdir -p %s' % targetdir) + parent_dir, target_name = os.path.split(targetdir) + runner.run('/bin/tar -cmzf /tmp/fs.tgz -C %s %s' + % (parent_dir, target_name)) + runner.run('cd /tmp') # need to be in same dir as fs.tgz + + ip = runner.get_target_ip() + url_base = self.start_http_server(runner, ip) + + url = url_base + '/fs.tgz' + logging.info("Fetching url: %s" % url) + tf = download_with_retry(self.context, self.scratch_dir, + url, False) + + tfdir = os.path.join(self.scratch_dir, str(time.time())) + + try: + os.mkdir(tfdir) + self.context.run_command('/bin/tar -C %s -xzf %s' + % (tfdir, tf)) + yield os.path.join(tfdir, target_name) + finally: + tf = os.path.join(self.scratch_dir, 'fs.tgz') + mk_targz(tf, tfdir) + rmtree(tfdir) + + # get the last 2 parts of tf, ie "scratchdir/tf.tgz" + tf = '/'.join(tf.split('/')[-2:]) + runner.run('rm -rf %s' % targetdir) + self._target_extract(runner, tf, parent_dir) + finally: + self.stop_http_server(runner) + else: + with super(BootloaderTarget, self).file_system( + partition, directory) as path: + yield path + + def _target_extract(self, runner, tar_file, dest, timeout=-1): + tmpdir = self.context.config.lava_image_tmpdir + url = self.context.config.lava_image_url + tar_file = tar_file.replace(tmpdir, '') + tar_url = '/'.join(u.strip('/') for u in [url, tar_file]) + self._target_extract_url(runner, tar_url, dest, timeout=timeout) + + def _target_extract_url(self, runner, tar_url, dest, timeout=-1): + decompression_cmd = '' + if tar_url.endswith('.gz') or tar_url.endswith('.tgz'): + decompression_cmd = '| /bin/gzip -dc' + elif tar_url.endswith('.bz2'): + decompression_cmd = '| /bin/bzip2 -dc' + elif tar_url.endswith('.tar'): + decompression_cmd = '' + else: + raise RuntimeError('bad file extension: %s' % tar_url) + + runner.run('wget -O - %s %s | /bin/tar -C %s -xmf -' + % (tar_url, decompression_cmd, dest), + timeout=timeout) + target_class = BootloaderTarget === modified file 'lava_dispatcher/device/fastmodel.py' --- lava_dispatcher/device/fastmodel.py 2013-08-12 02:44:22 +0000 +++ lava_dispatcher/device/fastmodel.py 2013-08-21 07:46:10 +0000 @@ -301,10 +301,10 @@ if self._uefi: self._enter_bootloader(self.proc) - if isinstance(self.config.boot_cmds, basestring): + if self._is_job_defined_boot_cmds(self.config.boot_cmds): + boot_cmds = self.config.boot_cmds + else: boot_cmds = string_to_list(self.config.boot_cmds.encode('ascii')) - else: - boot_cmds = self.config.boot_cmds self._customize_bootloader(self.proc, boot_cmds) return self.proc === modified file 'lava_dispatcher/device/master.py' --- lava_dispatcher/device/master.py 2013-08-12 02:44:22 +0000 +++ lava_dispatcher/device/master.py 2013-08-22 22:08:15 +0000 @@ -99,8 +99,6 @@ return self.device_version def power_on(self): - if self.config.power_on_cmd: - self.context.run_command(self.config.power_on_cmd) self._boot_linaro_image() return self.proc @@ -390,7 +388,7 @@ def _wait_for_master_boot(self): self.proc.expect(self.config.image_boot_msg, timeout=300) - self.proc.expect(self.config.master_str, timeout=300) + self._wait_for_prompt(self.proc, self.config.master_str, timeout=300) def boot_master_image(self): """ @@ -428,7 +426,7 @@ runner = MasterCommandRunner(self) try: - self.master_ip = runner.get_master_ip() + self.master_ip = runner.get_target_ip() self.device_version = runner.get_device_version() except NetworkError as e: msg = "Failed to get network up: " % e @@ -485,22 +483,31 @@ self.proc.empty_buffer() def _boot_linaro_image(self): + boot_cmds_job_file = False + boot_cmds_boot_options = False boot_cmds = self.deployment_data['boot_cmds'] options = boot_options.as_dict(self, defaults={'boot_cmds': boot_cmds}) + boot_cmds_job_file = self._is_job_defined_boot_cmds(self.config.boot_cmds) + + if 'boot_cmds' in options: + if options['boot_cmds'].value != 'boot_cmds': + boot_cmds_boot_options = True + # Interactive boot_cmds from the job file are a list. # We check for them first, if they are present, we use # them and ignore the other cases. - if not isinstance(self.config.boot_cmds, basestring): + if boot_cmds_job_file: logging.info('Overriding boot_cmds from job file') boot_cmds_override = True boot_cmds = self.config.boot_cmds # If there were no interactive boot_cmds, next we check # for boot_option overrides. If one exists, we use them # and ignore all other cases. - elif options['boot_cmds'].value != 'boot_cmds': + elif boot_cmds_boot_options: logging.info('Overriding boot_cmds from boot_options') boot_cmds = options['boot_cmds'].value + logging.info('boot_option=%s' % boot_cmds) boot_cmds = self.config.cp.get('__main__', boot_cmds) boot_cmds = string_to_list(boot_cmds.encode('ascii')) # No interactive or boot_option overrides are present, @@ -541,29 +548,6 @@ super(MasterCommandRunner, self).__init__( target, target.MASTER_PS1_PATTERN, prompt_str_includes_rc=True) - def get_master_ip(self): - logging.info("Waiting for network to come up") - try: - self.wait_network_up(timeout=20) - except NetworkError: - logging.exception("Unable to reach LAVA server") - raise - - pattern1 = "<(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)>" - cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" - "awk '{print \"<\" $1 \">\"}'" % - self._client.config.default_network_interface) - self.run( - cmd, [pattern1, pexpect.EOF, pexpect.TIMEOUT], timeout=5) - if self.match_id != 0: - msg = "Unable to determine master image IP address" - logging.error(msg) - raise CriticalError(msg) - - ip = self.match.group(1) - logging.debug("Master image IP is %s" % ip) - return ip - def get_device_version(self): pattern = 'device_version=(\d+-\d+/\d+-\d+)' self.run("echo \"device_version=" === modified file 'lava_dispatcher/device/qemu.py' --- lava_dispatcher/device/qemu.py 2013-07-24 14:04:25 +0000 +++ lava_dispatcher/device/qemu.py 2013-08-23 03:26:23 +0000 @@ -38,22 +38,58 @@ extract_targz, finalize_process, ) +from lava_dispatcher.errors import ( + CriticalError +) class QEMUTarget(Target): def __init__(self, context, config): super(QEMUTarget, self).__init__(context, config) + self._qemu_options = None self._sd_image = None + def deploy_linaro_kernel(self, kernel, ramdisk, dtb, rootfs, bootloader, + firmware, rootfstype, bootloadertype): + if rootfs is not None: + self._sd_image = download_image(rootfs, self.context) + self._customize_linux(self._sd_image) + self.append_qemu_options(self.config.qemu_options.format( + DISK_IMAGE=self._sd_image)) + kernel_args = 'root=/dev/sda1' + else: + raise CriticalError("You must specify a QEMU file system image") + + if kernel is not None: + kernel = download_image(kernel, self.context) + self.append_qemu_options(' -kernel %s' % kernel) + kernel_args += ' console=ttyS0,115200' + if ramdisk is not None: + ramdisk = download_image(ramdisk, self.context) + self.append_qemu_options(' -initrd %s' % ramdisk) + if dtb is not None: + dtb = download_image(dtb, self.context) + self.append_qemu_options(' -dtb %s' % ramdisk) + if firmware is not None: + firmware = download_image(firmware, self.context) + self.append_qemu_options(' -bios %s' % firmware) + self.append_qemu_options(' -append "%s"' % kernel_args) + else: + raise CriticalError("No kernel images to boot") + def deploy_linaro(self, hwpack=None, rootfs=None, bootloader='u_boot'): odir = self.scratch_dir self._sd_image = generate_image(self, hwpack, rootfs, odir, bootloader) self._customize_linux(self._sd_image) + self.append_qemu_options(self.config.qemu_options.format( + DISK_IMAGE=self._sd_image)) def deploy_linaro_prebuilt(self, image): self._sd_image = download_image(image, self.context) self._customize_linux(self._sd_image) + self.append_qemu_options(self.config.qemu_options.format( + DISK_IMAGE=self._sd_image)) @contextlib.contextmanager def file_system(self, partition, directory): @@ -70,9 +106,7 @@ extract_targz(tb, '%s/%s' % (mntdir, directory)) def power_on(self): - qemu_options = self.config.qemu_options.format( - DISK_IMAGE=self._sd_image) - qemu_cmd = '%s %s' % (self.config.qemu_binary, qemu_options) + qemu_cmd = '%s %s' % (self.config.qemu_binary, self._qemu_options) logging.info('launching qemu with command %r' % qemu_cmd) proc = self.context.spawn(qemu_cmd, timeout=1200) return proc @@ -89,4 +123,10 @@ except subprocess.CalledProcessError: return "unknown" + def append_qemu_options(self, parameter): + if self._qemu_options is None: + self._qemu_options = parameter + else: + self._qemu_options += parameter + target_class = QEMUTarget === modified file 'lava_dispatcher/device/target.py' --- lava_dispatcher/device/target.py 2013-08-12 02:44:22 +0000 +++ lava_dispatcher/device/target.py 2013-08-21 05:26:08 +0000 @@ -140,7 +140,7 @@ yield runner finally: if proc and runner: - self.power_off(proc) + pass def _get_runner(self, proc): from lava_dispatcher.client.base import CommandRunner @@ -178,6 +178,12 @@ def _wait_for_prompt(self, connection, prompt_pattern, timeout): wait_for_prompt(connection, prompt_pattern, timeout) + def _is_job_defined_boot_cmds(self, boot_cmds): + if isinstance(self.config.boot_cmds, basestring): + return False + else: + return True + def _enter_bootloader(self, connection): if connection.expect(self.config.interrupt_boot_prompt) != 0: raise Exception("Failed to enter bootloader") === removed file 'lava_dispatcher/device/uefi.py' --- lava_dispatcher/device/uefi.py 2013-07-16 16:06:08 +0000 +++ lava_dispatcher/device/uefi.py 1970-01-01 00:00:00 +0000 @@ -1,67 +0,0 @@ -# Copyright (C) 2013 Linaro Limited -# -# Author: Dave Pigott -# -# This file is part of LAVA Dispatcher. -# -# LAVA Dispatcher is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# LAVA Dispatcher is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along -# with this program; if not, see . - -import logging -import re -from lava_dispatcher.device.master import ( - MasterImageTarget -) - - -class UEFITarget(MasterImageTarget): - - def __init__(self, context, config): - super(UEFITarget, self).__init__(context, config) - - def _boot(self, boot_cmds): - """ - - :param boot_cmds: - :raise: - """ - try: - self._soft_reboot() - self._enter_uefi() - except: - logging.exception("enter uefi failed") - self._hard_reboot() - self._enter_uefi() - self.proc.expect(self.config.bootloader_prompt, timeout=300) - for line in range(0, len(boot_cmds)): - parts = re.match('^(?Psendline|expect)\s*(?P.*)', line) - try: - action = parts.group('action') - command = re.escape(parts.group('command')) - except AttributeError as e: - raise Exception("Badly formatted command in boot_cmds %s" % e) - if action == "sendline": - self.proc.send(command) - elif action == "expect": - self.proc.expect(command, timeout=300) - else: - raise Exception("Unrecognised action in boot_cmds") - - def _enter_uefi(self): - if self.proc.expect(self.config.interrupt_boot_prompt) != 0: - raise Exception("Failed to enter bootloader") - self.proc.send(self.config.interrupt_boot_command) - - -target_class = UEFITarget