From patchwork Wed Nov 30 02:42:20 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 5390 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 5161B23E04 for ; Wed, 30 Nov 2011 02:42:26 +0000 (UTC) Received: from mail-bw0-f52.google.com (mail-bw0-f52.google.com [209.85.214.52]) by fiordland.canonical.com (Postfix) with ESMTP id 12159A1858C for ; Wed, 30 Nov 2011 02:42:26 +0000 (UTC) Received: by bkbzv3 with SMTP id zv3so271335bkb.11 for ; Tue, 29 Nov 2011 18:42:25 -0800 (PST) Received: by 10.204.10.81 with SMTP id o17mr60760bko.65.1322620944049; Tue, 29 Nov 2011 18:42:24 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.223.96.199 with SMTP id i7cs28414fan; Tue, 29 Nov 2011 18:42:23 -0800 (PST) Received: by 10.216.229.220 with SMTP id h70mr46306weq.75.1322620941465; Tue, 29 Nov 2011 18:42:21 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id ej20si125164wbb.142.2011.11.29.18.42.21 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 29 Nov 2011 18:42:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1RVa7o-0005Th-Vd for ; Wed, 30 Nov 2011 02:42:20 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id E4986E003A for ; Wed, 30 Nov 2011 02:42:20 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 169 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 169: Major refactoring of how the dispatcher deploys images and communicates with Message-Id: <20111130024220.28635.81277.launchpad@ackee.canonical.com> Date: Wed, 30 Nov 2011 02:42:20 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14404"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: bd43cc67475f810b252092ccc3df906543362574 Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-dispatcher/image-client-cleanup/+merge/83243 proposed by: Michael Hudson-Doyle (mwhudson) review: Needs Fixing - Le Chi Thu (le-chi-thu) https://code.launchpad.net/~mwhudson/lava-dispatcher/qemu-image-client/+merge/83107 proposed by: Michael Hudson-Doyle (mwhudson) https://code.launchpad.net/~mwhudson/lava-dispatcher/master-image-client/+merge/83106 proposed by: Michael Hudson-Doyle (mwhudson) https://code.launchpad.net/~mwhudson/lava-dispatcher/more-transient-connections/+merge/83104 proposed by: Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 169 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Wed 2011-11-30 15:33:16 +1300 message: Major refactoring of how the dispatcher deploys images and communicates with the device being tested. * One major change is the commands are executed in 'sessions', for example: with client.tester_session() as session: session.run('ls') This will ensure the system is booted into the test partition and run() wraps up the "run a shell command, optionally looking for a particular response and wait for the next prompt" functionality we use in a number of places. * Another major change is to move the details of deployment to client methods. Nothing outside of the LavaMasterImageClient implementation of thse methods assumes the existence of a master image now. * Finally, add an experimental client subclass that builds an image and tests it for qemu. The QEMU support is probably a bit fragile and specific to beagle currently, but that should be easy to change. The support for testing on master image based boards should work as before. added: lava_dispatcher/client/ lava_dispatcher/client/__init__.py lava_dispatcher/client/lmc_utils.py lava_dispatcher/client/master.py lava_dispatcher/client/qemu.py renamed: lava_dispatcher/client.py => lava_dispatcher/client/base.py modified: lava_dispatcher/actions/android_deploy.py lava_dispatcher/actions/boot_control.py lava_dispatcher/actions/deploy.py lava_dispatcher/actions/launch_control.py lava_dispatcher/actions/lava-android-test.py lava_dispatcher/actions/lava-test.py lava_dispatcher/context.py lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf lava_dispatcher/job.py lava_dispatcher/utils.py lava_dispatcher/client/base.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/android_deploy.py' --- lava_dispatcher/actions/android_deploy.py 2011-11-23 02:38:04 +0000 +++ lava_dispatcher/actions/android_deploy.py 2011-11-24 03:00:54 +0000 @@ -20,220 +20,8 @@ # along with this program; if not, see . from lava_dispatcher.actions import BaseAction -import os -import shutil -import traceback -import logging -from tempfile import mkdtemp -from lava_dispatcher.utils import download, download_with_cache -from lava_dispatcher.client import CriticalError class cmd_deploy_linaro_android_image(BaseAction): def run(self, boot, system, data, pkg=None, use_cache=True): - LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir - LAVA_IMAGE_URL = self.context.lava_image_url - client = self.client - logging.info("Deploying Android on %s" % client.hostname) - logging.info(" boot: %s" % boot) - logging.info(" system: %s" % system) - logging.info(" data: %s" % data) - logging.info("Boot master image") - client.boot_master_image() - - logging.info("Waiting for network to come up...") - try: - client.wait_network_up() - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Unable to reach LAVA server, check network") - - try: - boot_tbz2, system_tbz2, data_tbz2, pkg_tbz2 = \ - self.download_tarballs(boot, system, data, pkg, use_cache) - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Unable to download artifacts for deployment") - - boot_tarball = boot_tbz2.replace(LAVA_IMAGE_TMPDIR, '') - system_tarball = system_tbz2.replace(LAVA_IMAGE_TMPDIR, '') - data_tarball = data_tbz2.replace(LAVA_IMAGE_TMPDIR, '') - - boot_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, boot_tarball]) - system_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, system_tarball]) - data_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, data_tarball]) - if pkg_tbz2: - pkg_tarball = pkg_tbz2.replace(LAVA_IMAGE_TMPDIR, '') - pkg_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, pkg_tarball]) - - try: - if pkg_tbz2: - self.deploy_linaro_android_testboot(boot_url, pkg_url) - else: - self.deploy_linaro_android_testboot(boot_url) - self.deploy_linaro_android_testrootfs(system_url) - self.purge_linaro_android_sdcard() - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Android deployment failed") - finally: - shutil.rmtree(self.tarball_dir) - logging.info("Android image deployment exiting") - - def download_tarballs(self, boot_url, system_url, data_url, pkg_url=None, - use_cache=True): - """Download tarballs from a boot, system and data tarball url - - :param boot_url: url of the Linaro Android boot tarball to download - :param system_url: url of the Linaro Android system tarball to download - :param data_url: url of the Linaro Android data tarball to download - :param pkg_url: url of the custom kernel tarball to download - :param use_cache: whether or not to use the cached copy (if it exists) - """ - lava_cachedir = self.context.lava_cachedir - LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir - self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) - tarball_dir = self.tarball_dir - os.chmod(tarball_dir, 0755) - logging.info("Downloading the image files") - - if use_cache: - boot_path = download_with_cache(boot_url, tarball_dir, lava_cachedir) - system_path = download_with_cache(system_url, tarball_dir, lava_cachedir) - data_path = download_with_cache(data_url, tarball_dir, lava_cachedir) - if pkg_url: - pkg_path = download_with_cache(pkg_url, tarball_dir) - else: - pkg_path = None - else: - boot_path = download(boot_url, tarball_dir) - system_path = download(system_url, tarball_dir) - data_path = download(data_url, tarball_dir) - if pkg_url: - pkg_path = download(pkg_url, tarball_dir) - else: - pkg_path = None - logging.info("Downloaded the image files") - return boot_path, system_path, data_path, pkg_path - - def deploy_linaro_android_testboot(self, boottbz2, pkgbz2=None): - logging.info("Deploying test boot filesystem") - client = self.client - client.run_cmd_master('umount /dev/disk/by-label/testboot') - client.run_cmd_master('mkfs.vfat /dev/disk/by-label/testboot ' - '-n testboot') - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/lava/boot') - client.run_cmd_master('mount /dev/disk/by-label/testboot ' - '/mnt/lava/boot') - client.run_cmd_master('wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % boottbz2) - if pkgbz2: - client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' - % pkgbz2) - - self.recreate_uInitrd() - - client.run_cmd_master('umount /mnt/lava/boot') - - def recreate_uInitrd(self): - logging.info("Recreate uInitrd") - client = self.client - # Original android sdcard partition layout by l-a-m-c - sys_part_org = client.device_option("sys_part_android_org") - cache_part_org = client.device_option("cache_part_android_org") - data_part_org = client.device_option("data_part_android_org") - # Sdcard layout in Lava image - sys_part_lava = client.device_option("sys_part_android") - - client.run_cmd_master('mkdir -p ~/tmp/') - client.run_cmd_master('mv /mnt/lava/boot/uInitrd ~/tmp') - client.run_cmd_master('cd ~/tmp/') - - client.run_cmd_master('dd if=uInitrd of=uInitrd.data ibs=64 skip=1') - client.run_cmd_master('mv uInitrd.data ramdisk.cpio.gz') - client.run_cmd_master( - 'gzip -d ramdisk.cpio.gz; cpio -i -F ramdisk.cpio') - client.run_cmd_master( - 'sed -i "/mount ext4 \/dev\/block\/mmcblk0p%s/d" init.rc' - % cache_part_org) - client.run_cmd_master( - 'sed -i "/mount ext4 \/dev\/block\/mmcblk0p%s/d" init.rc' - % data_part_org) - client.run_cmd_master('sed -i "s/mmcblk0p%s/mmcblk0p%s/g" init.rc' - % (sys_part_org, sys_part_lava)) - client.run_cmd_master( - 'sed -i "/export PATH/a \ \ \ \ export PS1 root@linaro: " init.rc') - - client.run_cmd_master( - 'cpio -i -t -F ramdisk.cpio | cpio -o -H newc | \ - gzip > ramdisk_new.cpio.gz') - - client.run_cmd_master( - 'mkimage -A arm -O linux -T ramdisk -n "Android Ramdisk Image" \ - -d ramdisk_new.cpio.gz uInitrd') - - client.run_cmd_master('cd -') - client.run_cmd_master('mv ~/tmp/uInitrd /mnt/lava/boot/uInitrd') - client.run_cmd_master('rm -rf ~/tmp') - - def deploy_linaro_android_testrootfs(self, systemtbz2): - logging.info("Deploying the test root filesystem") - client = self.client - sdcard_part_lava = client.device_option("sdcard_part_android") - - client.run_cmd_master('umount /dev/disk/by-label/testrootfs') - client.run_cmd_master( - 'mkfs.ext4 -q /dev/disk/by-label/testrootfs -L testrootfs') - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/lava/system') - client.run_cmd_master( - 'mount /dev/disk/by-label/testrootfs /mnt/lava/system') - client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % systemtbz2, - 600) - - sed_cmd = "/dev_mount sdcard \/mnt\/sdcard/c dev_mount sdcard /mnt/sdcard %s " \ - "/devices/platform/omap/omap_hsmmc.0/mmc_host/mmc0" %sdcard_part_lava - client.run_cmd_master( - 'sed -i "%s" /mnt/lava/system/etc/vold.fstab' % sed_cmd) - #Change the prompt if it exists in mkshrc also - client.run_cmd_master('sed -i "s/^PS1=.*$/PS1=\'root@linaro: \'/" /mnt/lava/system/etc/mkshrc') - client.run_cmd_master('umount /mnt/lava/system') - - def purge_linaro_android_sdcard(self): - logging.info("Reformatting Linaro Android sdcard filesystem") - client = self.client - client.run_cmd_master('mkfs.vfat /dev/disk/by-label/sdcard -n sdcard') - client.run_cmd_master('udevadm trigger') - - def deploy_linaro_android_system(self, systemtbz2): - logging.info("Deploying the Android system") - client = self.client - client.run_cmd_master('mkfs.ext4 -q /dev/disk/by-label/system -L system') - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/lava/system') - client.run_cmd_master('mount /dev/disk/by-label/system /mnt/lava/system') - client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % systemtbz2, - 600) - client.run_cmd_master('umount /mnt/lava/system') - - def deploy_linaro_android_data(self, datatbz2): - logging.info("Deploying the Android data") - client = self.client - client.run_cmd_master('mkfs.ext4 -q /dev/disk/by-label/data -L data') - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/lava/data') - client.run_cmd_master('mount /dev/disk/by-label/data /mnt/lava/data') - client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % datatbz2, - 600) - client.run_cmd_master('umount /mnt/lava/data') + self.client.deploy_linaro_android(boot, system, data, pkg, use_cache) === modified file 'lava_dispatcher/actions/boot_control.py' --- lava_dispatcher/actions/boot_control.py 2011-10-27 09:11:47 +0000 +++ lava_dispatcher/actions/boot_control.py 2011-11-24 03:00:54 +0000 @@ -23,15 +23,13 @@ import logging from lava_dispatcher.actions import BaseAction -from lava_dispatcher.client import CriticalError +from lava_dispatcher.client.base import CriticalError class cmd_boot_linaro_android_image(BaseAction): """ Call client code to boot to the master image """ def run(self): - #Workaround for commands coming too quickly at this point client = self.client - client.proc.sendline("") try: client.boot_linaro_android_image() except Exception as e: @@ -43,8 +41,6 @@ """ def run(self): client = self.client - #Workaround for commands coming too quickly at this point - client.proc.sendline("") status = 'pass' try: logging.info("Boot Linaro image") @@ -56,6 +52,7 @@ finally: self.context.test_data.add_result("boot_image", status) + class cmd_boot_master_image(BaseAction): """ Call client code to boot to the master image """ === modified file 'lava_dispatcher/actions/deploy.py' --- lava_dispatcher/actions/deploy.py 2011-11-15 08:08:40 +0000 +++ lava_dispatcher/actions/deploy.py 2011-11-21 04:21:06 +0000 @@ -17,260 +17,10 @@ # You should have received a copy of the GNU General Public License # along with this program; if not, see . -from commands import getoutput, getstatusoutput -import os -import re -import shutil -import traceback -from tempfile import mkdtemp -import logging - from lava_dispatcher.actions import BaseAction -from lava_dispatcher.utils import download, download_with_cache -from lava_dispatcher.client import CriticalError, OperationFailed class cmd_deploy_linaro_image(BaseAction): def run(self, hwpack, rootfs, kernel_matrix=None, use_cache=True): - LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir - LAVA_IMAGE_URL = self.context.lava_image_url - client = self.client - logging.info("deploying on %s" % client.hostname) - logging.info(" hwpack: %s" % hwpack) - logging.info(" rootfs: %s" % rootfs) - if kernel_matrix: - logging.info(" package: %s" % kernel_matrix[0]) - logging.info("Booting master image") - client.boot_master_image() - self._format_testpartition() - - logging.info("Waiting for network to come up") - try: - client.wait_network_up() - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Unable to reach LAVA server, check network") - - if kernel_matrix: - hwpack = self.refresh_hwpack(kernel_matrix, hwpack, use_cache) - #make new hwpack downloadable - hwpack = hwpack.replace(LAVA_IMAGE_TMPDIR, '') - hwpack = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, hwpack]) - logging.info(" hwpack with new kernel: %s" % hwpack) - - logging.info("About to handle with the build") - try: - boot_tgz, root_tgz = self.generate_tarballs(hwpack, rootfs, - use_cache) - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Deployment tarballs preparation failed") - boot_tarball = boot_tgz.replace(LAVA_IMAGE_TMPDIR, '') - root_tarball = root_tgz.replace(LAVA_IMAGE_TMPDIR, '') - boot_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, boot_tarball]) - root_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, root_tarball]) - try: - self.deploy_linaro_rootfs(root_url) - self.deploy_linaro_bootfs(boot_url) - except: - tb = traceback.format_exc() - client.sio.write(tb) - raise CriticalError("Deployment failed") - finally: - shutil.rmtree(self.tarball_dir) - - def _format_testpartition(self): - client = self.client - logging.info("Format testboot and testrootfs partitions") - client.run_cmd_master('umount /dev/disk/by-label/testrootfs') - client.run_cmd_master( - 'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs') - client.run_cmd_master('umount /dev/disk/by-label/testboot') - client.run_cmd_master( - 'mkfs.vfat /dev/disk/by-label/testboot -n testboot') - - def _get_partition_offset(self, image, partno): - cmd = 'parted %s -m -s unit b print' % image - part_data = getoutput(cmd) - pattern = re.compile('%d:([0-9]+)B:' % partno) - for line in part_data.splitlines(): - found = re.match(pattern, line) - if found: - return found.group(1) - return None - - def _extract_partition(self, image, offset, tarfile): - """Mount a partition and produce a tarball of it - - :param image: The image to mount - :param offset: offset of the partition, as a string - :param tarfile: path and filename of the tgz to output - """ - error_msg = None - mntdir = mkdtemp() - cmd = "sudo mount -o loop,offset=%s %s %s" % (offset, image, mntdir) - rc, output = getstatusoutput(cmd) - if rc: - os.rmdir(mntdir) - raise RuntimeError("Unable to mount image %s at offset %s" % ( - image, offset)) - cmd = "sudo tar -C %s -czf %s ." % (mntdir, tarfile) - rc, output = getstatusoutput(cmd) - if rc: - error_msg = "Failed to create tarball: %s" % tarfile - cmd = "sudo umount %s" % mntdir - rc, output = getstatusoutput(cmd) - os.rmdir(mntdir) - if error_msg: - raise RuntimeError(error_msg) - - def generate_tarballs(self, hwpack_url, rootfs_url, use_cache=True): - """Generate tarballs from a hwpack and rootfs url - - :param hwpack_url: url of the Linaro hwpack to download - :param rootfs_url: url of the Linaro image to download - """ - lava_cachedir = self.context.lava_cachedir - LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir - client = self.client - self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) - tarball_dir = self.tarball_dir - os.chmod(tarball_dir, 0755) - #fix me: if url is not http-prefix, copy it to tarball_dir - if use_cache: - logging.info("Downloading the %s file using cache" % hwpack_url) - hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir) - - logging.info("Downloading the %s file using cache" % rootfs_url) - rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir) - else: - logging.info("Downloading the %s file" % hwpack_url) - hwpack_path = download(hwpack_url, tarball_dir) - - logging.info("Downloading the %s file" % rootfs_url) - rootfs_path = download(rootfs_url, tarball_dir) - - logging.info("linaro-media-create version information") - cmd = "sudo linaro-media-create -v" - rc, output = getstatusoutput(cmd) - metadata = self.context.test_data.get_metadata() - metadata['target.linaro-media-create-version'] = output - self.context.test_data.add_metadata(metadata) - - image_file = os.path.join(tarball_dir, "lava.img") - #XXX Hack for removing startupfiles from snowball hwpacks - if client.device_type == "snowball_sd": - cmd = "sudo linaro-hwpack-replace -r startupfiles-v3 -t %s -i" % hwpack_path - rc, output = getstatusoutput(cmd) - if rc: - raise RuntimeError("linaro-hwpack-replace failed: %s" % output) - - cmd = ("sudo flock /var/lock/lava-lmc.lck linaro-media-create --hwpack-force-yes --dev %s " - "--image-file %s --binary %s --hwpack %s --image-size 3G" % - (client.lmc_dev_arg, image_file, rootfs_path, hwpack_path)) - logging.info("Executing the linaro-media-create command") - logging.info(cmd) - rc, output = getstatusoutput(cmd) - if rc: - shutil.rmtree(tarball_dir) - tb = traceback.format_exc() - client.sio.write(tb) - raise RuntimeError("linaro-media-create failed: %s" % output) - boot_offset = self._get_partition_offset(image_file, client.boot_part) - root_offset = self._get_partition_offset(image_file, client.root_part) - boot_tgz = os.path.join(tarball_dir, "boot.tgz") - root_tgz = os.path.join(tarball_dir, "root.tgz") - try: - self._extract_partition(image_file, boot_offset, boot_tgz) - self._extract_partition(image_file, root_offset, root_tgz) - except: - shutil.rmtree(tarball_dir) - tb = traceback.format_exc() - client.sio.write(tb) - raise - return boot_tgz, root_tgz - - def deploy_linaro_rootfs(self, rootfs): - client = self.client - logging.info("Deploying linaro image") - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/root') - client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root') - rc = client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/root -xzf -' % rootfs, - timeout=3600) - if rc != 0: - msg = "Deploy test rootfs partition: failed to download tarball." - raise OperationFailed(msg) - - client.run_cmd_master('echo linaro > /mnt/root/etc/hostname') - #DO NOT REMOVE - diverting flash-kernel and linking it to /bin/true - #prevents a serious problem where packages getting installed that - #call flash-kernel can update the kernel on the master image - client.run_cmd_master( - 'chroot /mnt/root dpkg-divert --local /usr/sbin/flash-kernel') - client.run_cmd_master( - 'chroot /mnt/root ln -sf /bin/true /usr/sbin/flash-kernel') - client.run_cmd_master('umount /mnt/root') - - def deploy_linaro_bootfs(self, bootfs): - client = self.client - logging.info("Deploying linaro bootfs") - client.run_cmd_master('udevadm trigger') - client.run_cmd_master('mkdir -p /mnt/boot') - client.run_cmd_master('mount /dev/disk/by-label/testboot /mnt/boot') - rc = client.run_cmd_master( - 'wget -qO- %s |tar --numeric-owner -C /mnt/boot -xzf -' % bootfs) - if rc != 0: - msg = "Deploy test boot partition: failed to download tarball." - raise OperationFailed(msg) - client.run_cmd_master('umount /mnt/boot') - - def refresh_hwpack(self, kernel_matrix, hwpack, use_cache=True): - client = self.client - lava_cachedir = self.context.lava_cachedir - LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir - logging.info("Deploying new kernel") - new_kernel = kernel_matrix[0] - deb_prefix = kernel_matrix[1] - filesuffix = new_kernel.split(".")[-1] - - if filesuffix != "deb": - raise CriticalError("New kernel only support deb kernel package!") - - # download package to local - tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) - os.chmod(tarball_dir, 0755) - if use_cache: - kernel_path = download_with_cache(new_kernel, tarball_dir, lava_cachedir) - hwpack_path = download_with_cache(hwpack, tarball_dir, lava_cachedir) - else: - kernel_path = download(new_kernel, tarball_dir) - hwpack_path = download(hwpack, tarball_dir) - - cmd = ("sudo linaro-hwpack-replace -t %s -p %s -r %s" - % (hwpack_path, kernel_path, deb_prefix)) - - rc, output = getstatusoutput(cmd) - if rc: - shutil.rmtree(tarball_dir) - tb = traceback.format_exc() - client.sio.write(tb) - raise RuntimeError("linaro-hwpack-replace failed: %s" % output) - - #fix it:l-h-r doesn't make a output option to specify the output hwpack, - #so it needs to do manually here - - #remove old hwpack and leave only new hwpack in tarball_dir - os.remove(hwpack_path) - hwpack_list = os.listdir(tarball_dir) - for hp in hwpack_list: - if hp.split(".")[-1] == "gz": - new_hwpack_path = os.path.join(tarball_dir, hp) - return new_hwpack_path - + self.client.deploy_linaro( + hwpack, rootfs, kernel_matrix=None, use_cache=True) === modified file 'lava_dispatcher/actions/launch_control.py' --- lava_dispatcher/actions/launch_control.py 2011-10-27 02:28:52 +0000 +++ lava_dispatcher/actions/launch_control.py 2011-11-24 03:00:54 +0000 @@ -26,10 +26,7 @@ import logging from lava_dispatcher.actions import BaseAction -from lava_dispatcher.client import OperationFailed -from lava_dispatcher.utils import download -from tempfile import mkdtemp -import time +from lava_dispatcher.client.base import OperationFailed import xmlrpclib import traceback @@ -98,76 +95,17 @@ if status == 'fail': raise OperationFailed(err_msg) + class cmd_submit_results(SubmitResultAction): + def run(self, server, stream, result_disk="testrootfs"): """Submit test results to a lava-dashboard server :param server: URL of the lava-dashboard server RPC endpoint :param stream: Stream on the lava-dashboard server to save the result to """ - client = self.client - try: - client.in_master_shell() - except OperationFailed: - client.boot_master_image() - except: - logging.exception("in_master_shell failed") - client.boot_master_image() - - client.run_cmd_master('mkdir -p /mnt/root') - client.run_cmd_master( - 'mount /dev/disk/by-label/%s /mnt/root' % result_disk) - # Clean results directory on master image - client.run_cmd_master( - 'rm -rf /tmp/lava_results.tgz /tmp/%s' % self.context.lava_result_dir) - client.run_cmd_master('mkdir -p /tmp/%s' % self.context.lava_result_dir) - client.run_cmd_master( - 'cp /mnt/root/%s/*.bundle /tmp/%s' % (self.context.lava_result_dir, - self.context.lava_result_dir)) - # Clean result bundle on test image - client.run_cmd_master( - 'rm -f /mnt/root/%s/*.bundle' % (self.context.lava_result_dir)) - client.run_cmd_master('umount /mnt/root') - - # Create tarball of all results - logging.info("Creating lava results tarball") - client.run_cmd_master('cd /tmp') - client.run_cmd_master( - 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir) - - # start gather_result job, status - status = 'pass' - err_msg = '' - master_ip = client.get_master_ip() - if master_ip: - # Set 80 as server port - client.run_cmd_master('python -m SimpleHTTPServer 80 &> /dev/null &') - time.sleep(3) - - result_tarball = "http://%s/lava_results.tgz" % master_ip - tarball_dir = mkdtemp(dir=self.context.lava_image_tmpdir) - os.chmod(tarball_dir, 0755) - - # download test result with a retry mechanism - # set retry timeout to 2mins - logging.info("About to download the result tarball to host") - now = time.time() - timeout = 120 - try: - while time.time() < now + timeout: - try: - result_path = download(result_tarball, tarball_dir) - except RuntimeError: - if time.time() >= now + timeout: - logging.exception("download failed") - raise - except: - logging.warning(traceback.format_exc()) - status = 'fail' - err_msg = err_msg + " Can't retrieve test case results." - logging.warning(err_msg) - - client.run_cmd_master('kill %1') - + + status, err_msg, result_path = self.client.retrieve_results(result_disk) + if result_path is not None: try: tar = tarfile.open(result_path) for tarinfo in tar: @@ -183,15 +121,11 @@ err_msg = err_msg + " Some test case result appending failed." logging.warning(err_msg) finally: - shutil.rmtree(tarball_dir) - else: - status = 'fail' - err_msg = err_msg + "Getting master image IP address failed, \ -no test case result retrived." - logging.warning(err_msg) - - #flush the serial log - client.run_shell_command("") + shutil.rmtree(os.path.dirname(result_path)) + + if err_msg is None: + err_msg = '' + self.submit_combine_bundles(status, err_msg, server, stream) if status == 'fail': raise OperationFailed(err_msg) === modified file 'lava_dispatcher/actions/lava-android-test.py' --- lava_dispatcher/actions/lava-android-test.py 2011-11-18 02:37:31 +0000 +++ lava_dispatcher/actions/lava-android-test.py 2011-11-24 03:00:54 +0000 @@ -19,48 +19,20 @@ # You should have received a copy of the GNU General Public License # along with this program; if not, see . -import sys -import pexpect -import time +import os import logging from datetime import datetime from lava_dispatcher.actions import BaseAction -from lava_dispatcher.client import OperationFailed, NetworkError +from lava_dispatcher.client.base import OperationFailed + class AndroidTestAction(BaseAction): - def wait_devices_attached(self, dev_name): - for count in range(3): - if self.check_device_state(dev_name): - return - time.sleep(1) - - raise NetworkError("The android device(%s) isn't attached" % self.client.hostname) - - def check_device_state(self, dev_name): - (output, rc) = pexpect.run('adb devices', timeout=None, logfile=sys.stdout, withexitstatus=True) - if rc != 0: - return False - expect_line = '%s\tdevice' % dev_name - for line in output.splitlines(): - if line.strip() == expect_line: - return True - return False - def check_lava_android_test_installed(self): - rc = pexpect.run('which lava-android-test', timeout=None, logfile=sys.stdout, withexitstatus=True)[1] + rc = os.system('which lava-android-test') if rc != 0: raise OperationFailed('lava-android-test has not been installed') - def is_ready_for_test(self): - self.check_lava_android_test_installed() - dev_name = self.client.android_adb_connect_over_default_nic_ip() - if dev_name is None: - raise NetworkError("The android device(%s) isn't attached over tcpip" % self.client.hostname) - - self.wait_devices_attached(dev_name) - self.client.wait_home_screen() - return dev_name class cmd_lava_android_test_run(AndroidTestAction): @@ -70,28 +42,36 @@ def run(self, test_name, timeout=-1): #Make sure in test image now - dev_name = self.is_ready_for_test() - bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S") - cmd = 'lava-android-test run %s -s %s -o %s/%s.bundle' % ( - test_name, dev_name, self.context.host_result_dir, bundle_name) - - logging.info("Execute command on host: %s" % cmd) - rc = pexpect.run(cmd, timeout=None, logfile=sys.stdout, withexitstatus=True)[1] - if rc != 0: - raise OperationFailed("Failed to run test case(%s) on device(%s) with return value: %s" % (test_name, dev_name, rc)) + self.check_lava_android_test_installed() + with self.client.android_tester_session() as session: + bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S") + cmd = 'lava-android-test run %s -s %s -o %s/%s.bundle' % ( + test_name, session.dev_name, self.context.host_result_dir, + bundle_name) + + logging.info("Execute command on host: %s" % cmd) + rc = os.system(cmd) + if rc != 0: + raise OperationFailed( + "Failed to run test case(%s) on device(%s) with return " + "value: %s" % (test_name, session.dev_name, rc)) + class cmd_lava_android_test_install(AndroidTestAction): """ lava-test deployment to test image rootfs by chroot """ def run(self, tests, option=None, timeout=2400): - dev_name = self.is_ready_for_test() - for test in tests: - cmd = 'lava-android-test install %s -s %s' % (test, dev_name) - if option is not None: - cmd += ' -o ' + option - logging.info("Execute command on host: %s" % cmd) - rc = pexpect.run(cmd, timeout=None, logfile=sys.stdout, withexitstatus=True)[1] - if rc != 0: - raise OperationFailed("Failed to install test case(%s) on device(%s) with return value: %s" % (test, dev_name, rc)) - + self.check_lava_android_test_installed() + with self.client.android_tester_session() as session: + for test in tests: + cmd = 'lava-android-test install %s -s %s' % ( + test, session.dev_name) + if option is not None: + cmd += ' -o ' + option + logging.info("Execute command on host: %s" % cmd) + rc = os.system(cmd) + if rc != 0: + raise OperationFailed( + "Failed to install test case(%s) on device(%s) with " + "return value: %s" % (test, session.dev_name, rc)) === modified file 'lava_dispatcher/actions/lava-test.py' --- lava_dispatcher/actions/lava-test.py 2011-11-16 16:50:44 +0000 +++ lava_dispatcher/actions/lava-test.py 2011-11-24 03:00:54 +0000 @@ -25,57 +25,27 @@ import traceback from lava_dispatcher.actions import BaseAction -from lava_dispatcher.client import OperationFailed, CriticalError +from lava_dispatcher.client.base import OperationFailed, CriticalError from lava_dispatcher.config import get_config -def _setup_testrootfs(client): - #Make sure in master image - #, or exception can be caught and do boot_master_image() - try: - client.in_master_shell() - except: - logging.exception("in_master_shell failed") - client.boot_master_image() - - client.run_cmd_master('mkdir -p /mnt/root') - client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root') - client.run_cmd_master( - 'cp -f /mnt/root/etc/resolv.conf /mnt/root/etc/resolv.conf.bak') - client.run_cmd_master('cp -L /etc/resolv.conf /mnt/root/etc') - #eliminate warning: Can not write log, openpty() failed - # (/dev/pts not mounted?), does not work - client.run_cmd_master('mount --rbind /dev /mnt/root/dev') - - -def _teardown_testrootfs(client): - client.run_cmd_master( - 'cp -f /mnt/root/etc/resolv.conf.bak /mnt/root/etc/resolv.conf') - cmd = ('cat /proc/mounts | awk \'{print $2}\' | grep "^/mnt/root/dev"' - '| sort -r | xargs umount') - client.run_cmd_master(cmd) - client.run_cmd_master('umount /mnt/root') - - -def _install_lava_test(client): + +def _install_lava_test(client, session): #install bazaar in tester image - client.run_cmd_master( - 'chroot /mnt/root apt-get update') + session.run('apt-get update') #Install necessary packages for build lava-test - cmd = ('chroot /mnt/root apt-get -y install bzr usbutils python-apt ' - 'python-setuptools python-simplejson lsb-release') - client.run_cmd_master(cmd, timeout=2400) - client.run_cmd_master("chroot /mnt/root apt-get -y install python-pip") + cmd = ('apt-get -y install ' + 'bzr usbutils python-apt python-setuptools python-simplejson lsb-release') + session.run(cmd, timeout=2400) + session.run("apt-get -y install python-pip") dispatcher_config = get_config("lava-dispatcher") lava_test_url = dispatcher_config.get("LAVA_TEST_URL") logging.debug("Installing %s with pip" % lava_test_url) - client.run_cmd_master('chroot /mnt/root pip install -e ' + lava_test_url) + session.run('pip install -e ' + lava_test_url) #Test if lava-test installed try: - client.run_shell_command( - 'chroot /mnt/root lava-test help', - response="list-test", timeout=60) + session.run('lava-test help', response="list-test", timeout=60) except: tb = traceback.format_exc() client.sio.write(tb) @@ -89,36 +59,31 @@ def run(self, test_name, test_options = "", timeout=-1): logging.info("Executing lava_test_run %s command" % test_name) - #Make sure in test image now - client = self.client - try: - client.in_test_shell() - except: - client.boot_linaro_image() - client.run_cmd_tester('mkdir -p %s' % self.context.lava_result_dir) - client.export_display() - bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S") - - if test_options != "": - test_options = "-t '%s'" % test_options - - cmd = ('lava-test run %s %s -o %s/%s.bundle' % ( - test_name, test_options, self.context.lava_result_dir, bundle_name)) - try: - rc = client.run_cmd_tester(cmd, timeout=timeout) - except: - logging.exception("run_cmd_tester failed") - client.proc.sendcontrol('c') + with self.client.tester_session() as session: + session.run('mkdir -p %s' % self.context.lava_result_dir) + session.export_display() + bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S") + + if test_options != "": + test_options = "-t '%s'" % test_options + + cmd = ('lava-test run %s %s -o %s/%s.bundle' % ( + test_name, test_options, self.context.lava_result_dir, bundle_name)) try: - client.run_cmd_tester('true', timeout=20) + rc = session.run(cmd, timeout=timeout) except: - logging.exception("run_cmd_tester true failed, rebooting") - client.boot_linaro_image() - raise - if rc is None: - raise OperationFailed("test case getting return value failed") - elif rc != 0: - raise OperationFailed("test case failed with return value: %s" % rc) + logging.exception("session.run failed") + self.client.proc.sendcontrol('c') + try: + session.run('true', timeout=20) + except: + logging.exception("killing test failed, rebooting") + self.client.boot_linaro_image() + raise + if rc is None: + raise OperationFailed("test case getting return value failed") + elif rc != 0: + raise OperationFailed("test case failed with return value: %s" % rc) class cmd_lava_test_install(BaseAction): """ @@ -126,42 +91,36 @@ """ def run(self, tests, install_python = None, register = None, timeout=2400): logging.info("Executing lava_test_install (%s) command" % ",".join(tests)) - client = self.client - - _setup_testrootfs(client) - _install_lava_test(client) - - if install_python: - for module in install_python: - client.run_cmd_master("chroot /mnt/root pip install -e " + module) - - if register: - for test_def_url in register: - client.run_cmd_master('chroot /mnt/root lava-test register-test ' + test_def_url) - - for test in tests: - client.run_cmd_master( - 'chroot /mnt/root lava-test install %s' % test) - - client.run_cmd_master('rm -rf /mnt/root/lava-test') - - _teardown_testrootfs(client) + + with self.client.reliable_session() as session: + + _install_lava_test(self.client, session) + + if install_python: + for module in install_python: + session.run("pip install -e " + module) + + if register: + for test_def_url in register: + session.run('lava-test register-test ' + test_def_url) + + for test in tests: + session.run('lava-test install %s' % test) + + session.run('rm -rf lava-test') class cmd_add_apt_repository(BaseAction): """ add apt repository to test image rootfs by chroot - arg could be 'deb uri distribution [component1] [component2][...]' or ppm: + arg could be 'deb uri distribution [component1] [component2][...]' or ppa: """ def run(self, arg): - client = self.client - _setup_testrootfs(client) - - #install add-apt-repository - client.run_cmd_master('chroot /mnt/root apt-get -y install python-software-properties') - - #add ppa - client.run_cmd_master('chroot /mnt/root add-apt-repository %s < /dev/null' % arg[0]) - client.run_cmd_master('chroot /mnt/root apt-get update') - - _teardown_testrootfs(client) + with self.client.reliable_session() as session: + + #install add-apt-repository + session.run('apt-get -y install python-software-properties') + + #add ppa + session.run('add-apt-repository %s < /dev/null' % arg[0]) + session.run('apt-get update') === added directory 'lava_dispatcher/client' === added file 'lava_dispatcher/client/__init__.py' --- lava_dispatcher/client/__init__.py 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/client/__init__.py 2011-11-24 03:00:54 +0000 @@ -0,0 +1,19 @@ +# Copyright (C) 2011 Linaro Limited +# +# Author: Michael Hudson-Doyle +# +# This file is part of LAVA Dispatcher. +# +# LAVA Dispatcher is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# LAVA Dispatcher is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along +# with this program; if not, see . === renamed file 'lava_dispatcher/client.py' => 'lava_dispatcher/client/base.py' --- lava_dispatcher/client.py 2011-11-21 23:06:03 +0000 +++ lava_dispatcher/client/base.py 2011-11-30 01:17:02 +0000 @@ -1,5 +1,6 @@ # Copyright (C) 2011 Linaro Limited # +# Author: Michael Hudson-Doyle # Author: Paul Larson # # This file is part of LAVA Dispatcher. @@ -18,34 +19,258 @@ # along # with this program; if not, see . +import commands +import contextlib import pexpect import sys import time from cStringIO import StringIO import traceback -from utils import string_to_list +from lava_dispatcher.utils import string_to_list import logging -from lava_dispatcher.connection import ( - LavaConmuxConnection, - ) + +class CommandRunner(object): + """A convenient way to run a shell command and wait for a shell prompt. + + The main interface is run(). Subclasses exist to (a) be more conveniently + constructed in some situations and (b) define higher level functions that + involve executing multiple commands. + """ + + def __init__(self, connection, prompt_str, wait_for_rc=True): + """ + + :param connection: A pexpect.spawn-like object. + :param prompt_str: The shell prompt to wait for. + :param wait_for_rc: Whether to wait for a rc=$? indication of the + command's return value after prompt_str. + """ + self._connection = connection + self._prompt_str = prompt_str + self._wait_for_rc = wait_for_rc + self.match_id = None + self.match = None + + def _empty_pexpect_buffer(self): + """Make sure there is nothing in the pexpect buffer.""" + # Do we really need this? It wastes at least 1 second per command + # invocation, if nothing else. + index = 0 + while index == 0: + index = self._connection.expect( + ['.+', pexpect.EOF, pexpect.TIMEOUT], timeout=1) + + def run(self, cmd, response=None, timeout=-1): + """Run `cmd` and wait for a shell response. + + :param cmd: The command to execute. + :param response: A pattern or sequences of patterns to pass to + .expect(). + :param timeout: How long to wait for 'response' (if specified) and the + shell prompt, defaulting to forever. + :return: The exit value of the command, if wait_for_rc not explicitly + set to False during construction. + """ + self._empty_pexpect_buffer() + self._connection.sendline(cmd) + start = time.time() + if response is not None: + self.match_id = self._connection.expect(response, timeout=timeout) + self.match = self._connection.match + # If a non-trivial timeout was specified, it is held to apply to + # the whole invocation, so now reduce the time we'll wait for the + # shell prompt. + if timeout > 0: + timeout -= time.time() - start + # But not too much; give at least a little time for the shell + # prompt to appear. + if timeout < 1: + timeout = 1 + else: + self.match_id = None + self.match = None + self._connection.expect(self._prompt_str, timeout=timeout) + if self._wait_for_rc: + match_id = self._connection.expect( + ['rc=(\d+\d?\d?)', pexpect.EOF, pexpect.TIMEOUT], timeout=2) + if match_id == 0: + rc = int(self._connection.match.groups()[0]) + else: + rc = None + else: + rc = None + return rc + + +class NetworkCommandRunner(CommandRunner): + """A CommandRunner with some networking utility methods.""" + + def __init__(self, client, prompt_str, wait_for_rc=True): + CommandRunner.__init__( + self, client.proc, prompt_str, wait_for_rc=wait_for_rc) + self._client = client + + def _check_network_up(self): + """Internal function for checking network once.""" + lava_server_ip = self._client.context.lava_server_ip + self.run( + "LC_ALL=C ping -W4 -c1 %s" % lava_server_ip, + ["1 received", "0 received", "Network is unreachable"], timeout=5) + if self.match_id == 0: + return True + else: + return False + + def wait_network_up(self, timeout=300): + """Wait until the networking is working.""" + now = time.time() + while time.time() < now+timeout: + if self._check_network_up(): + return + raise NetworkError + + +class TesterCommandRunner(CommandRunner): + """A CommandRunner to use when the board is booted into the test image. + + See `LavaClient.tester_session`. + """ + + def __init__(self, client, wait_for_rc=True): + CommandRunner.__init__( + self, client.proc, client.tester_str, wait_for_rc) + + def export_display(self): + self.run("su - linaro -c 'DISPLAY=:0 xhost local:'") + self.run("export DISPLAY=:0") + + +class AndroidTesterCommandRunner(NetworkCommandRunner): + """A CommandRunner to use when the board is booted into the android image. + + See `LavaClient.android_tester_session`. + """ + + def __init__(self, client): + super(AndroidTesterCommandRunner, self).__init__( + client, client.tester_str, wait_for_rc=False) + self.dev_name = None + + # adb cound be connected through network + def android_adb_connect(self, dev_ip): + pattern1 = "connected to (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d{1,5})" + pattern2 = "already connected to (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d{1,5})" + pattern3 = "unable to connect to" + + cmd = "adb connect %s" % dev_ip + logging.info("Execute adb command on host: %s" % cmd) + adb_proc = pexpect.spawn(cmd, timeout=300, logfile=sys.stdout) + match_id = adb_proc.expect([pattern1, pattern2, pattern3, pexpect.EOF]) + if match_id in [0, 1]: + self.dev_name = adb_proc.match.groups()[0] + + def android_adb_disconnect(self, dev_ip): + cmd = "adb disconnect %s" % dev_ip + logging.info("Execute adb command on host: %s" % cmd) + pexpect.run(cmd, timeout=300, logfile=sys.stdout) + + def get_default_nic_ip(self): + # XXX: IP could be assigned in other way in the validation farm + network_interface = self._client.default_network_interface + ip = None + try: + ip = self._get_default_nic_ip_by_ifconfig(network_interface) + except: + logging.exception("_get_default_nic_ip_by_ifconfig failed") + pass + + if ip is None: + self.get_ip_via_dhcp(network_interface) + ip = self._get_default_nic_ip_by_ifconfig(network_interface) + return ip + + def _get_default_nic_ip_by_ifconfig(self, nic_name): + # Check network ip and setup adb connection + try: + self.wait_network_up() + except: + logging.warning(traceback.format_exc()) + return None + ip_pattern = "%s: ip (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) mask" % nic_name + try: + self.run( + "ifconfig %s" % nic_name, [ip_pattern, pexpect.EOF], timeout=60) + except Exception as e: + raise NetworkError("ifconfig can not match ip pattern for %s:%s" % (nic_name, e)) + + if self.match_id == 0: + match_group = self.match.groups() + if len(match_group) > 0: + return match_group[0] + return None + + def get_ip_via_dhcp(self, nic): + try: + self.run('netcfg %s dhcp' % nic, timeout=60) + except: + logging.exception("netcfg %s dhcp failed" % nic) + raise NetworkError("netcfg %s dhcp exception" % nic) + + def wait_until_attached(self): + for count in range(3): + if self.check_device_state(): + return + time.sleep(1) + + raise NetworkError( + "The android device(%s) isn't attached" % self._client.hostname) + + def wait_home_screen(self): + cmd = 'getprop init.svc.bootanim' + for count in range(100): + self.run(cmd, response='stopped') + if self.match_id == 0: + return True + time.sleep(1) + raise GeneralError('The home screen has not displayed') + + def check_device_state(self): + (rc, output) = commands.getstatusoutput('adb devices') + if rc != 0: + return False + expect_line = '%s\tdevice' % self.dev_name + for line in output.splitlines(): + if line.strip() == expect_line: + return True + return False + + def retrieve_results(self, result_disk): + raise NotImplementedError(self.retrieve_results) class LavaClient(object): """ - LavaClient manipulates the target board, bootup, reset, power off the board, - sends commands to board to execute + LavaClient manipulates the target board, bootup, reset, power off the + board, sends commands to board to execute. + + The main interfaces to execute commands on the board are the *_session() + methods. These should be used as context managers, for example:: + + with client.tester_session() as session: + session.run('ls') + + Each method makes sure the board is booted into the appropriate state + (tester image, chrooted into a partition, etc) and additionally + android_tester_session connects to the board via adb while in the 'with' + block. """ + def __init__(self, context, config): self.context = context self.config = config self.sio = SerialIO(sys.stdout) - if config.get('client_type') == 'conmux': - self.proc = LavaConmuxConnection(config, self.sio) - else: - raise RuntimeError( - "this version of lava-dispatcher only supports conmux " - "clients, not %r" % config.get('client_type')) + self.proc = None def device_option(self, option_name): return self.config.get(option_name) @@ -62,10 +287,6 @@ return self.device_option("TESTER_STR") @property - def master_str(self): - return self.device_option("MASTER_STR") - - @property def boot_cmds(self): uboot_str = self.device_option("boot_cmds") return string_to_list(uboot_str) @@ -90,21 +311,54 @@ def lmc_dev_arg(self): return self.device_option("lmc_dev_arg") - def in_master_shell(self, timeout=10): - """ - Check that we are in a shell on the master image - """ - self.proc.sendline("") - id = self.proc.expect([self.master_str, pexpect.TIMEOUT], - timeout=timeout) - if id == 1: - raise OperationFailed - logging.info("System is in master image now") + @contextlib.contextmanager + def tester_session(self): + """A session that can be used to run commands booted into the test + image.""" + try: + self.in_test_shell() + except OperationFailed: + self.boot_linaro_image() + yield TesterCommandRunner(self) + + @contextlib.contextmanager + def android_tester_session(self): + """A session that can be used to run commands booted into the android + test image. + + Additionally, adb is connected while in the with block using this + manager. + """ + try: + self.in_test_shell() + except OperationFailed: + self.boot_linaro_android_image() + session = AndroidTesterCommandRunner(self) + logging.info("adb connect over default network interface") + dev_ip = session.get_default_nic_ip() + if dev_ip is None: + raise OperationFailed("failed to get board ip address") + session.android_adb_connect(dev_ip) + session.wait_until_attached() + session.wait_home_screen() + try: + yield session + finally: + session.android_adb_disconnect(dev_ip) + + def reliable_session(self): + """ + Return a session rooted in the rootfs to be tested where networking is + guaranteed to work. + """ + raise NotImplementedError(self.reliable_session) def in_test_shell(self, timeout=10): """ Check that we are in a shell on the test image """ + if self.proc is None: + raise OperationFailed self.proc.sendline("") match_id = self.proc.expect([self.tester_str, pexpect.TIMEOUT], timeout=timeout) @@ -112,20 +366,11 @@ raise OperationFailed logging.info("System is in test image now") + def deploy_linaro(self, hwpack, rootfs, kernel_matrix=None, use_cache=True): + raise NotImplementedError(self.deploy_linaro) + def boot_master_image(self): - """ - reboot the system, and check that we are in a master shell - """ - self.proc.soft_reboot() - try: - self.proc.expect("Starting kernel") - self.in_master_shell(300) - except: - logging.exception("in_master_shell failed") - self.proc.hard_reboot() - self.in_master_shell(300) - self.proc.sendline('export PS1="$PS1 [rc=$(echo \$?)]: "') - self.proc.expect(self.master_str, timeout=10) + raise NotImplementedError(self.boot_master_image) def boot_linaro_image(self): """ @@ -140,94 +385,9 @@ self.proc.sendline('export PS1="$PS1 [rc=$(echo \$?)]: "') self.proc.expect(self.tester_str, timeout=10) - def run_shell_command(self, cmd, response=None, timeout=-1): - self.empty_pexpect_buffer() - # return return-code if captured, else return None - self.proc.sendline(cmd) - start_time = time.time() - if response: - self.proc.expect(response, timeout=timeout) - elapsed_time = int(time.time()-start_time) - # if reponse is master/tester string, make rc expect timeout to be - # 2 sec, else make it consume remained timeout - if response in [self.master_str, self.tester_str]: - timeout = 2 - else: - timeout = int(timeout-elapsed_time) - #verify return value of last command, match one number at least - #PS1 setting is in boot_linaro_image or boot_master_image - pattern1 = "rc=(\d+\d?\d?)" - id = self.proc.expect([pattern1, pexpect.EOF, pexpect.TIMEOUT], - timeout=timeout) - if id == 0: - rc = int(self.proc.match.groups()[0]) - else: - rc = None - return rc - - def run_cmd_master(self, cmd, timeout=-1): - return self.run_shell_command(cmd, self.master_str, timeout) - - def run_cmd_tester(self, cmd, timeout=-1): - return self.run_shell_command(cmd, self.tester_str, timeout) - - def _check_network_up(self): - """ - Internal function for checking network one time - """ - lava_server_ip = self.context.lava_server_ip - self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % lava_server_ip) - id = self.proc.expect(["1 received", "0 received", - "Network is unreachable"], timeout=5) - if id == 0: - return True - else: - return False - - def wait_network_up(self, timeout=300): - now = time.time() - while time.time() < now+timeout: - if self._check_network_up(): - return - raise NetworkError - - def get_master_ip(self): - #get master image ip address - try: - self.wait_network_up() - except: - logging.warning(traceback.format_exc()) - return None - #tty device uses minimal match, see pexpect wiki - #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" - pattern1 = "(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)" - cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" - "awk '{print $1}'" % self.default_network_interface) - self.proc.sendline(cmd) - #if running from ipython, it needs another Enter, don't know why: - #self.proc.sendline("") - id = self.proc.expect([pattern1, pexpect.EOF, - pexpect.TIMEOUT], timeout=5) - logging.info("\nmatching pattern is %s" % id) - if id == 0: - ip = self.proc.match.groups()[0] - logging.info("Master IP is %s" % ip) - return ip - return None - - def export_display(self): - #export the display, ignore errors on non-graphical images - self.run_cmd_tester("su - linaro -c 'DISPLAY=:0 xhost local:'") - self.run_cmd_tester("export DISPLAY=:0") - def get_seriallog(self): return self.sio.getvalue() - def empty_pexpect_buffer(self): - index = 0 - while (index == 0): - index = self.proc.expect (['.+', pexpect.EOF, pexpect.TIMEOUT], timeout=1) - # Android stuff def boot_linaro_android_image(self): @@ -236,104 +396,15 @@ self.in_test_shell(timeout=900) self.proc.sendline("export PS1=\"root@linaro: \"") - self.enable_adb_over_tcpip() - self.android_adb_disconnect_over_default_nic_ip() - - # adb cound be connected through network - def android_adb_connect(self, dev_ip): - pattern1 = "connected to (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d{1,5})" - pattern2 = "already connected to (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d{1,5})" - pattern3 = "unable to connect to" - - cmd = "adb connect %s" % dev_ip - logging.info("Execute adb command on host: %s" % cmd) - adb_proc = pexpect.spawn(cmd, timeout=300, logfile=sys.stdout) - match_id = adb_proc.expect([pattern1, pattern2, pattern3, pexpect.EOF]) - if match_id in [0, 1]: - dev_name = adb_proc.match.groups()[0] - return dev_name - else: - return None - - def android_adb_disconnect(self, dev_ip): - cmd = "adb disconnect %s" % dev_ip - logging.info("Execute adb command on host: %s" % cmd) - pexpect.run(cmd, timeout=300, logfile=sys.stdout) - - def get_default_nic_ip(self): - # XXX: IP could be assigned in other way in the validation farm - network_interface = self.default_network_interface - ip = None - try: - ip = self._get_default_nic_ip_by_ifconfig(network_interface) - except: - logging.exception("_get_default_nic_ip_by_ifconfig failed") - pass - - if ip is None: - self.get_ip_via_dhcp(network_interface) - ip = self._get_default_nic_ip_by_ifconfig(network_interface) - return ip - - def _get_default_nic_ip_by_ifconfig(self, nic_name): - # Check network ip and setup adb connection - try: - self.wait_network_up() - except: - logging.warning(traceback.format_exc()) - return None - ip_pattern = "%s: ip (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) mask" % nic_name - cmd = "ifconfig %s" % nic_name - self.proc.sendline('') - self.proc.sendline(cmd) - match_id = 0 - try: - match_id = self.proc.expect([ip_pattern, pexpect.EOF], timeout=60) - except Exception as e: - raise NetworkError("ifconfig can not match ip pattern for %s:%s" % (nic_name, e)) - - if match_id == 0: - match_group = self.proc.match.groups() - if len(match_group) > 0: - return match_group[0] - return None - - def get_ip_via_dhcp(self, nic): - try: - self.run_cmd_tester('netcfg %s dhcp' % nic, timeout=60) - except: - logging.exception("netcfg %s dhcp failed" % nic) - raise NetworkError("netcfg %s dhcp exception" % nic) - - - def android_adb_connect_over_default_nic_ip(self): - logging.info("adb connect over default network interface") - dev_ip = self.get_default_nic_ip() - if dev_ip is not None: - return self.android_adb_connect(dev_ip) - - def android_adb_disconnect_over_default_nic_ip(self): - logging.info("adb disconnect over default network interface") - dev_ip = self.get_default_nic_ip() - if dev_ip is not None: - self.android_adb_disconnect(dev_ip) - - def enable_adb_over_tcpip(self): + self._enable_adb_over_tcpip() + + def _enable_adb_over_tcpip(self): logging.info("Enable adb over TCPIP") - self.proc.sendline('echo 0>/sys/class/android_usb/android0/enable') - self.proc.sendline('setprop service.adb.tcp.port 5555') - self.proc.sendline('stop adbd') - self.proc.sendline('start adbd') - - def wait_home_screen(self): - cmd = 'getprop init.svc.bootanim' - for count in range(100): - self.proc.sendline(cmd) - match_id = self.proc.expect('stopped') - if match_id == 0: - return True - time.sleep(1) - raise GeneralError('The home screen does not displayed') + session = TesterCommandRunner(self, wait_for_rc=False) + session.run('echo 0>/sys/class/android_usb/android0/enable') + session.run('setprop service.adb.tcp.port 5555') + session.run('stop adbd') + session.run('start adbd') class SerialIO(file): === added file 'lava_dispatcher/client/lmc_utils.py' --- lava_dispatcher/client/lmc_utils.py 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/client/lmc_utils.py 2011-11-24 04:42:13 +0000 @@ -0,0 +1,145 @@ +from commands import getoutput, getstatusoutput +import contextlib +import logging +import re +import os +import shutil +from tempfile import mkdtemp + +from lava_dispatcher.client.base import CriticalError +from lava_dispatcher.utils import ( + download, + download_with_cache, + logging_system, + ) + +def refresh_hwpack(client, kernel_matrix, hwpack, use_cache=True): + lava_cachedir = client.context.lava_cachedir + LAVA_IMAGE_TMPDIR = client.context.lava_image_tmpdir + logging.info("Deploying new kernel") + new_kernel = kernel_matrix[0] + deb_prefix = kernel_matrix[1] + filesuffix = new_kernel.split(".")[-1] + + if filesuffix != "deb": + raise CriticalError("New kernel only support deb kernel package!") + + # download package to local + tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) + os.chmod(tarball_dir, 0755) + if use_cache: + kernel_path = download_with_cache(new_kernel, tarball_dir, lava_cachedir) + hwpack_path = download_with_cache(hwpack, tarball_dir, lava_cachedir) + else: + kernel_path = download(new_kernel, tarball_dir) + hwpack_path = download(hwpack, tarball_dir) + + cmd = ("sudo linaro-hwpack-replace -t %s -p %s -r %s" + % (hwpack_path, kernel_path, deb_prefix)) + + rc, output = getstatusoutput(cmd) + if rc: + shutil.rmtree(tarball_dir) + raise RuntimeError("linaro-hwpack-replace failed: %s" % output) + + #fix it:l-h-r doesn't make a output option to specify the output hwpack, + #so it needs to do manually here + + #remove old hwpack and leave only new hwpack in tarball_dir + os.remove(hwpack_path) + hwpack_list = os.listdir(tarball_dir) + for hp in hwpack_list: + if hp.split(".")[-1] == "gz": + new_hwpack_path = os.path.join(tarball_dir, hp) + return new_hwpack_path + + +def generate_image(client, hwpack_url, rootfs_url, kernel_matrix, use_cache=True): + """Generate image from a hwpack and rootfs url + + :param hwpack_url: url of the Linaro hwpack to download + :param rootfs_url: url of the Linaro image to download + """ + lava_cachedir = client.context.lava_cachedir + LAVA_IMAGE_TMPDIR = client.context.lava_image_tmpdir + LAVA_IMAGE_URL = client.context.lava_image_url + logging.info("preparing to deploy on %s" % client.hostname) + logging.info(" hwpack: %s" % hwpack_url) + logging.info(" rootfs: %s" % rootfs_url) + if kernel_matrix: + logging.info(" package: %s" % kernel_matrix[0]) + hwpack_url = refresh_hwpack(kernel_matrix, hwpack_url, use_cache) + #make new hwpack downloadable + hwpack_url = hwpack_url.replace(LAVA_IMAGE_TMPDIR, '') + hwpack_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, hwpack_url]) + logging.info(" hwpack with new kernel: %s" % hwpack_url) + tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) + os.chmod(tarball_dir, 0755) + #fix me: if url is not http-prefix, copy it to tarball_dir + if use_cache: + logging.info("Downloading the %s file using cache" % hwpack_url) + hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir) + + logging.info("Downloading the %s file using cache" % rootfs_url) + rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir) + else: + logging.info("Downloading the %s file" % hwpack_url) + hwpack_path = download(hwpack_url, tarball_dir) + + logging.info("Downloading the %s file" % rootfs_url) + rootfs_path = download(rootfs_url, tarball_dir) + + logging.info("linaro-media-create version information") + cmd = "sudo linaro-media-create -v" + rc, output = getstatusoutput(cmd) + metadata = client.context.test_data.get_metadata() + metadata['target.linaro-media-create-version'] = output + client.context.test_data.add_metadata(metadata) + + image_file = os.path.join(tarball_dir, "lava.img") + #XXX Hack for removing startupfiles from snowball hwpacks + if client.device_type == "snowball_sd": + cmd = "sudo linaro-hwpack-replace -r startupfiles-v3 -t %s -i" % hwpack_path + rc, output = getstatusoutput(cmd) + if rc: + raise RuntimeError("linaro-hwpack-replace failed: %s" % output) + + cmd = ("sudo flock /var/lock/lava-lmc.lck linaro-media-create --hwpack-force-yes --dev %s " + "--image-file %s --binary %s --hwpack %s --image-size 3G" % + (client.lmc_dev_arg, image_file, rootfs_path, hwpack_path)) + logging.info("Executing the linaro-media-create command") + logging.info(cmd) + rc, output = getstatusoutput(cmd) + if rc: + client.rmtree(tarball_dir) + raise RuntimeError("linaro-media-create failed: %s" % output) + return image_file + +def get_partition_offset(image, partno): + cmd = 'parted %s -m -s unit b print' % image + part_data = getoutput(cmd) + pattern = re.compile('%d:([0-9]+)B:' % partno) + for line in part_data.splitlines(): + found = re.match(pattern, line) + if found: + return found.group(1) + return None + + +@contextlib.contextmanager +def image_partition_mounted(image_file, partno): + mntdir = mkdtemp() + image = image_file + offset = get_partition_offset(image, partno) + mount_cmd = "sudo mount -o loop,offset=%s %s %s" % (offset, image, mntdir) + rc = logging_system(mount_cmd) + if rc != 0: + os.rmdir(mntdir) + raise RuntimeError("Unable to mount image %s at offset %s" % ( + image, offset)) + try: + yield mntdir + finally: + logging_system('sudo umount ' + mntdir) + logging_system('rm -rf ' + mntdir) === added file 'lava_dispatcher/client/master.py' --- lava_dispatcher/client/master.py 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/client/master.py 2011-11-29 03:28:23 +0000 @@ -0,0 +1,548 @@ +# Copyright (C) 2011 Linaro Limited +# +# Author: Michael Hudson-Doyle +# Author: Paul Larson +# +# This file is part of LAVA Dispatcher. +# +# LAVA Dispatcher is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# LAVA Dispatcher is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along +# with this program; if not, see . + +import contextlib +import os +import pexpect +import shutil +import traceback +from tempfile import mkdtemp +import logging +import time + +from lava_dispatcher.utils import ( + download, + download_with_cache, + logging_system, + ) +from lava_dispatcher.client.base import ( + CommandRunner, + CriticalError, + LavaClient, + NetworkCommandRunner, + OperationFailed, + ) +from lava_dispatcher.client.lmc_utils import ( + generate_image, + image_partition_mounted, + ) +from lava_dispatcher.connection import ( + LavaConmuxConnection, + ) + + +def _extract_partition(image, partno, tarfile): + """Mount a partition and produce a tarball of it + + :param image: The image to mount + :param partno: The index of the partition in the image + :param tarfile: path and filename of the tgz to output + """ + with image_partition_mounted(image, partno) as mntdir: + cmd = "sudo tar -C %s -czf %s ." % (mntdir, tarfile) + rc = logging_system(cmd) + if rc: + raise RuntimeError("Failed to create tarball: %s" % tarfile) + + +def _deploy_linaro_rootfs(session, rootfs): + logging.info("Deploying linaro image") + session.run('udevadm trigger') + session.run('mkdir -p /mnt/root') + session.run('mount /dev/disk/by-label/testrootfs /mnt/root') + rc = session.run( + 'wget -qO- %s |tar --numeric-owner -C /mnt/root -xzf -' % rootfs, + timeout=3600) + if rc != 0: + msg = "Deploy test rootfs partition: failed to download tarball." + raise OperationFailed(msg) + + session.run('echo linaro > /mnt/root/etc/hostname') + #DO NOT REMOVE - diverting flash-kernel and linking it to /bin/true + #prevents a serious problem where packages getting installed that + #call flash-kernel can update the kernel on the master image + session.run( + 'chroot /mnt/root dpkg-divert --local /usr/sbin/flash-kernel') + session.run( + 'chroot /mnt/root ln -sf /bin/true /usr/sbin/flash-kernel') + session.run('umount /mnt/root') + +def _deploy_linaro_bootfs(session, bootfs): + logging.info("Deploying linaro bootfs") + session.run('udevadm trigger') + session.run('mkdir -p /mnt/boot') + session.run('mount /dev/disk/by-label/testboot /mnt/boot') + rc = session.run( + 'wget -qO- %s |tar --numeric-owner -C /mnt/boot -xzf -' % bootfs) + if rc != 0: + msg = "Deploy test boot partition: failed to download tarball." + raise OperationFailed(msg) + session.run('umount /mnt/boot') + +def _deploy_linaro_android_testboot(session, boottbz2, pkgbz2=None): + logging.info("Deploying test boot filesystem") + session.run('umount /dev/disk/by-label/testboot') + session.run('mkfs.vfat /dev/disk/by-label/testboot ' + '-n testboot') + session.run('udevadm trigger') + session.run('mkdir -p /mnt/lava/boot') + session.run('mount /dev/disk/by-label/testboot ' + '/mnt/lava/boot') + session.run('wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % boottbz2) + if pkgbz2: + session.run( + 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' + % pkgbz2) + + _recreate_uInitrd(session) + + session.run('umount /mnt/lava/boot') + +def _recreate_uInitrd(session): + logging.info("Recreate uInitrd") + # Original android sdcard partition layout by l-a-m-c + sys_part_org = session._client.device_option("sys_part_android_org") + cache_part_org = session._client.device_option("cache_part_android_org") + data_part_org = session._client.device_option("data_part_android_org") + # Sdcard layout in Lava image + sys_part_lava = session._client.device_option("sys_part_android") + + session.run('mkdir -p ~/tmp/') + session.run('mv /mnt/lava/boot/uInitrd ~/tmp') + session.run('cd ~/tmp/') + + session.run('dd if=uInitrd of=uInitrd.data ibs=64 skip=1') + session.run('mv uInitrd.data ramdisk.cpio.gz') + session.run( + 'gzip -d ramdisk.cpio.gz; cpio -i -F ramdisk.cpio') + session.run( + 'sed -i "/mount ext4 \/dev\/block\/mmcblk0p%s/d" init.rc' + % cache_part_org) + session.run( + 'sed -i "/mount ext4 \/dev\/block\/mmcblk0p%s/d" init.rc' + % data_part_org) + session.run('sed -i "s/mmcblk0p%s/mmcblk0p%s/g" init.rc' + % (sys_part_org, sys_part_lava)) + session.run( + 'sed -i "/export PATH/a \ \ \ \ export PS1 root@linaro: " init.rc') + + session.run( + 'cpio -i -t -F ramdisk.cpio | cpio -o -H newc | \ + gzip > ramdisk_new.cpio.gz') + + session.run( + 'mkimage -A arm -O linux -T ramdisk -n "Android Ramdisk Image" \ + -d ramdisk_new.cpio.gz uInitrd') + + session.run('cd -') + session.run('mv ~/tmp/uInitrd /mnt/lava/boot/uInitrd') + session.run('rm -rf ~/tmp') + +def _deploy_linaro_android_testrootfs(session, systemtbz2): + logging.info("Deploying the test root filesystem") + sdcard_part_lava = session._client.device_option("sdcard_part_android") + + session.run('umount /dev/disk/by-label/testrootfs') + session.run( + 'mkfs.ext4 -q /dev/disk/by-label/testrootfs -L testrootfs') + session.run('udevadm trigger') + session.run('mkdir -p /mnt/lava/system') + session.run( + 'mount /dev/disk/by-label/testrootfs /mnt/lava/system') + session.run( + 'wget -qO- %s |tar --numeric-owner -C /mnt/lava -xjf -' % systemtbz2, + timeout=600) + + sed_cmd = "/dev_mount sdcard \/mnt\/sdcard/c dev_mount sdcard /mnt/sdcard %s " \ + "/devices/platform/omap/omap_hsmmc.0/mmc_host/mmc0" %sdcard_part_lava + session.run( + 'sed -i "%s" /mnt/lava/system/etc/vold.fstab' % sed_cmd) + session.run('sed -i "s/^PS1=.*$/PS1=\'root@linaro: \'/" /mnt/lava/system/etc/mkshrc') + session.run('umount /mnt/lava/system') + +def _purge_linaro_android_sdcard(session): + logging.info("Reformatting Linaro Android sdcard filesystem") + session.run('mkfs.vfat /dev/disk/by-label/sdcard -n sdcard') + session.run('udevadm trigger') + + +class PrefixCommandRunner(CommandRunner): + """A CommandRunner that prefixes every command run with a given string. + + The motivating use case is to prefix every command with 'chroot + $LOCATION'. + """ + + def __init__(self, prefix, connection, prompt_str): + super(PrefixCommandRunner, self).__init__(connection, prompt_str) + if not prefix.endswith(' '): + prefix += ' ' + self._prefix = prefix + + def run(self, cmd, response=None, timeout=-1): + return super(PrefixCommandRunner, self).run(self._prefix + cmd) + + +class MasterCommandRunner(NetworkCommandRunner): + """A CommandRunner to use when the board is booted into the master image. + + See `LavaClient.master_session`. + """ + + def __init__(self, client): + super(MasterCommandRunner, self).__init__(client, client.master_str) + + def get_master_ip(self): + #get master image ip address + try: + self.wait_network_up() + except: + logging.warning(traceback.format_exc()) + return None + #tty device uses minimal match, see pexpect wiki + #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" + pattern1 = "(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)" + cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" + "awk '{print $1}'" % self._client.default_network_interface) + self.run( + cmd, [pattern1, pexpect.EOF, pexpect.TIMEOUT], timeout=5) + if self.match_id == 0: + logging.info("\nmatching pattern is %s" % self.match_id) + ip = self.match.groups()[0] + logging.info("Master IP is %s" % ip) + return ip + return None + + +class LavaMasterImageClient(LavaClient): + + def __init__(self, context, config): + super(LavaMasterImageClient, self).__init__(context, config) + self.proc = LavaConmuxConnection(config, self.sio) + + @property + def master_str(self): + return self.device_option("MASTER_STR") + + def deploy_linaro(self, hwpack, rootfs, kernel_matrix=None, use_cache=True): + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir + LAVA_IMAGE_URL = self.context.lava_image_url + try: + boot_tgz, root_tgz = self._generate_tarballs( + hwpack, rootfs, kernel_matrix, use_cache) + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Deployment tarballs preparation failed") + logging.info("Booting master image") + self.boot_master_image() + try: + boot_tarball = boot_tgz.replace(LAVA_IMAGE_TMPDIR, '') + root_tarball = root_tgz.replace(LAVA_IMAGE_TMPDIR, '') + boot_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, boot_tarball]) + root_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, root_tarball]) + with self._master_session() as session: + self._format_testpartition(session) + + logging.info("Waiting for network to come up") + try: + session.wait_network_up() + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Unable to reach LAVA server, check network") + + try: + _deploy_linaro_rootfs(session, root_url) + _deploy_linaro_bootfs(session, boot_url) + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Deployment failed") + finally: + shutil.rmtree(os.path.dirname(boot_tgz)) + + def deploy_linaro_android(self, boot, system, data, pkg=None, use_cache=True): + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir + LAVA_IMAGE_URL = self.context.lava_image_url + logging.info("Deploying Android on %s" % self.hostname) + logging.info(" boot: %s" % boot) + logging.info(" system: %s" % system) + logging.info(" data: %s" % data) + logging.info("Boot master image") + self.boot_master_image() + + with self._master_session() as session: + logging.info("Waiting for network to come up...") + try: + session.wait_network_up() + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Unable to reach LAVA server, check network") + + try: + boot_tbz2, system_tbz2, data_tbz2, pkg_tbz2 = \ + self._download_tarballs(boot, system, data, pkg, use_cache) + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Unable to download artifacts for deployment") + + boot_tarball = boot_tbz2.replace(LAVA_IMAGE_TMPDIR, '') + system_tarball = system_tbz2.replace(LAVA_IMAGE_TMPDIR, '') + #data_tarball = data_tbz2.replace(LAVA_IMAGE_TMPDIR, '') + + boot_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, boot_tarball]) + system_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, system_tarball]) + #data_url = '/'.join(u.strip('/') for u in [ + # LAVA_IMAGE_URL, data_tarball]) + if pkg_tbz2: + pkg_tarball = pkg_tbz2.replace(LAVA_IMAGE_TMPDIR, '') + pkg_url = '/'.join(u.strip('/') for u in [ + LAVA_IMAGE_URL, pkg_tarball]) + else: + pkg_url = None + + try: + _deploy_linaro_android_testboot(session, boot_url, pkg_url) + _deploy_linaro_android_testrootfs(session, system_url) + _purge_linaro_android_sdcard(session) + except: + tb = traceback.format_exc() + self.sio.write(tb) + raise CriticalError("Android deployment failed") + finally: + shutil.rmtree(self.tarball_dir) + logging.info("Android image deployment exiting") + + def _download_tarballs(self, boot_url, system_url, data_url, pkg_url=None, + use_cache=True): + """Download tarballs from a boot, system and data tarball url + + :param boot_url: url of the Linaro Android boot tarball to download + :param system_url: url of the Linaro Android system tarball to download + :param data_url: url of the Linaro Android data tarball to download + :param pkg_url: url of the custom kernel tarball to download + :param use_cache: whether or not to use the cached copy (if it exists) + """ + lava_cachedir = self.context.lava_cachedir + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir + self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) + tarball_dir = self.tarball_dir + os.chmod(tarball_dir, 0755) + logging.info("Downloading the image files") + + if use_cache: + boot_path = download_with_cache(boot_url, tarball_dir, lava_cachedir) + system_path = download_with_cache(system_url, tarball_dir, lava_cachedir) + data_path = download_with_cache(data_url, tarball_dir, lava_cachedir) + if pkg_url: + pkg_path = download_with_cache(pkg_url, tarball_dir) + else: + pkg_path = None + else: + boot_path = download(boot_url, tarball_dir) + system_path = download(system_url, tarball_dir) + data_path = download(data_url, tarball_dir) + if pkg_url: + pkg_path = download(pkg_url, tarball_dir) + else: + pkg_path = None + logging.info("Downloaded the image files") + return boot_path, system_path, data_path, pkg_path + + def boot_master_image(self): + """ + reboot the system, and check that we are in a master shell + """ + self.proc.soft_reboot() + try: + self.proc.expect("Starting kernel") + self._in_master_shell(300) + except: + logging.exception("in_master_shell failed") + self.proc.hard_reboot() + self._in_master_shell(300) + self.proc.sendline('export PS1="$PS1 [rc=$(echo \$?)]: "') + self.proc.expect(self.master_str, timeout=10) + + def _format_testpartition(self, session): + logging.info("Format testboot and testrootfs partitions") + session.run('umount /dev/disk/by-label/testrootfs') + session.run( + 'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs') + session.run('umount /dev/disk/by-label/testboot') + session.run('mkfs.vfat /dev/disk/by-label/testboot -n testboot') + + def _generate_tarballs(self, hwpack_url, rootfs_url, kernel_matrix, use_cache=True): + """Generate tarballs from a hwpack and rootfs url + + :param hwpack_url: url of the Linaro hwpack to download + :param rootfs_url: url of the Linaro image to download + """ + image_file = generate_image(self, hwpack_url, rootfs_url, kernel_matrix, use_cache) + tarball_dir = os.path.dirname(image_file) + boot_tgz = os.path.join(tarball_dir, "boot.tgz") + root_tgz = os.path.join(tarball_dir, "root.tgz") + try: + _extract_partition(image_file, self.boot_part, boot_tgz) + _extract_partition(image_file, self.root_part, root_tgz) + except: + shutil.rmtree(tarball_dir) + tb = traceback.format_exc() + self.sio.write(tb) + raise + return boot_tgz, root_tgz + + def reliable_session(self): + return self._partition_session('testrootfs') + + def retrieve_results(self, result_disk): + with self._master_session() as session: + + session.run('mkdir -p /mnt/root') + session.run( + 'mount /dev/disk/by-label/%s /mnt/root' % result_disk) + # Clean results directory on master image + session.run( + 'rm -rf /tmp/lava_results.tgz /tmp/%s' % self.context.lava_result_dir) + session.run('mkdir -p /tmp/%s' % self.context.lava_result_dir) + session.run( + 'cp /mnt/root/%s/*.bundle /tmp/%s' % (self.context.lava_result_dir, + self.context.lava_result_dir)) + # Clean result bundle on test image + session.run( + 'rm -f /mnt/root/%s/*.bundle' % (self.context.lava_result_dir)) + session.run('umount /mnt/root') + + # Create tarball of all results + logging.info("Creating lava results tarball") + session.run('cd /tmp') + session.run( + 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir) + + # start gather_result job, status + err_msg = '' + master_ip = session.get_master_ip() + if not master_ip: + err_msg = (err_msg + "Getting master image IP address failed, " + "no test case result retrived.") + logging.warning(err_msg) + return 'fail', err_msg, None + # Set 80 as server port + session.run('python -m SimpleHTTPServer 80 &> /dev/null &') + try: + time.sleep(3) + + result_tarball = "http://%s/lava_results.tgz" % master_ip + tarball_dir = mkdtemp(dir=self.context.lava_image_tmpdir) + os.chmod(tarball_dir, 0755) + + # download test result with a retry mechanism + # set retry timeout to 2mins + logging.info("About to download the result tarball to host") + now = time.time() + timeout = 120 + tries = 0 + try: + while time.time() < now + timeout: + try: + result_path = download( + result_tarball, tarball_dir, + verbose_failure=tries==0) + except RuntimeError: + tries += 1 + if time.time() >= now + timeout: + logging.exception("download failed") + raise + except: + logging.warning(traceback.format_exc()) + err_msg = err_msg + " Can't retrieve test case results." + logging.warning(err_msg) + return 'fail', err_msg, None + + return 'pass', None, result_path + finally: + session.run('kill %1') + session.run('') + + + @contextlib.contextmanager + def _partition_session(self, partition): + """A session that can be used to run commands in a given test + partition. + + Anything that uses this will have to be done differently for images + that are not deployed via a master image (e.g. using a JTAG to blow + the image onto the card or testing under QEMU). + """ + with self._master_session() as master_session: + directory = '/mnt/' + partition + master_session.run('mkdir -p %s' % directory) + master_session.run('mount /dev/disk/by-label/%s %s' % (partition, directory)) + master_session.run( + 'cp -f %s/etc/resolv.conf %s/etc/resolv.conf.bak' % ( + directory, directory)) + master_session.run('cp -L /etc/resolv.conf %s/etc' % directory) + #eliminate warning: Can not write log, openpty() failed + # (/dev/pts not mounted?), does not work + master_session.run('mount --rbind /dev %s/dev' % directory) + try: + yield PrefixCommandRunner( + 'chroot ' + directory, self.proc, self.master_str) + finally: + master_session.run( + 'cp -f %s/etc/resolv.conf.bak %s/etc/resolv.conf' % ( + directory, directory)) + cmd = ('cat /proc/mounts | awk \'{print $2}\' | grep "^%s/dev"' + '| sort -r | xargs umount' % directory) + master_session.run(cmd) + master_session.run('umount ' + directory) + + def _in_master_shell(self, timeout=10): + """ + Check that we are in a shell on the master image + """ + self.proc.sendline("") + match_id = self.proc.expect( + [self.master_str, pexpect.TIMEOUT], timeout=timeout) + if match_id == 1: + raise OperationFailed + logging.info("System is in master image now") + + @contextlib.contextmanager + def _master_session(self): + """A session that can be used to run commands in the master image. + + Anything that uses this will have to be done differently for images + that are not deployed via a master image (e.g. using a JTAG to blow + the image onto the card or testing under QEMU). + """ + try: + self._in_master_shell() + except OperationFailed: + self.boot_master_image() + yield MasterCommandRunner(self) === added file 'lava_dispatcher/client/qemu.py' --- lava_dispatcher/client/qemu.py 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/client/qemu.py 2011-11-30 02:06:29 +0000 @@ -0,0 +1,124 @@ +# Copyright (C) 2011 Linaro Limited +# +# Author: Michael Hudson-Doyle +# +# This file is part of LAVA Dispatcher. +# +# LAVA Dispatcher is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# LAVA Dispatcher is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along +# with this program; if not, see . + +import contextlib +import logging +import os +import pexpect +from tempfile import mkdtemp + +from lava_dispatcher.client.base import ( + CommandRunner, + LavaClient, + ) +from lava_dispatcher.client.lmc_utils import ( + generate_image, + image_partition_mounted, + ) +from lava_dispatcher.utils import ( + logging_system, + ) + + + +class LavaQEMUClient(LavaClient): + + def __init__(self, context, config): + super(LavaQEMUClient, self).__init__(context, config) + self._lava_image = None + + def deploy_linaro(self, hwpack, rootfs, kernel_matrix=None, use_cache=True): + image_file = generate_image(self, hwpack, rootfs, kernel_matrix, use_cache) + self._lava_image = image_file + with image_partition_mounted(self._lava_image, self.root_part) as mntdir: + logging_system('echo linaro > %s/etc/hostname' % mntdir) + + @contextlib.contextmanager + def _mnt_prepared_for_qemu(self, mntdir): + logging_system('sudo cp %s/etc/resolv.conf %s/etc/resolv.conf.bak' % (mntdir, mntdir)) + logging_system('sudo cp %s/etc/hosts %s/etc/hosts.bak' % (mntdir, mntdir)) + logging_system('sudo cp /etc/hosts %s/etc/hosts' % (mntdir,)) + logging_system('sudo cp /etc/resolv.conf %s/etc/resolv.conf' % (mntdir,)) + logging_system('sudo cp /usr/bin/qemu-arm-static %s/usr/bin/' % (mntdir,)) + try: + yield + finally: + logging_system('sudo mv %s/etc/resolv.conf.bak %s/etc/resolv.conf' % (mntdir, mntdir)) + logging_system('sudo mv %s/etc/hosts.bak %s/etc/hosts' % (mntdir, mntdir)) + logging_system('sudo rm %s/usr/bin/qemu-arm-static' % (mntdir,)) + + @contextlib.contextmanager + def _chroot_into_rootfs_session(self): + with image_partition_mounted(self._lava_image, self.root_part) as mntdir: + with self._mnt_prepared_for_qemu(mntdir): + cmd = pexpect.spawn('chroot ' + mntdir, logfile=self.sio, timeout=None) + try: + cmd.sendline('export PS1="root@host-mount:# [rc=$(echo \$?)] "') + cmd.expect('root@host-mount:#') + yield CommandRunner(cmd, 'root@host-mount:#') + finally: + cmd.sendline('exit') + cmd.close() + + def reliable_session(self): + # We could use _chroot_into_rootfs_session instead, but in my testing + # as of 2011-11-30, the network works better in a tested image than + # qemu-arm-static works to run the complicated commands of test + # installation. + return self.tester_session() + + def boot_master_image(self): + raise RuntimeError("QEMU devices do not have a master image to boot.") + + def boot_linaro_image(self): + """ + Boot the system to the test image + """ + if self.proc is not None: + self.proc.sendline('sync') + self.proc.expect([self.tester_str, pexpect.TIMEOUT], timeout=10) + self.proc.close() + qemu_cmd = ('%s -M %s -drive if=sd,cache=writeback,file=%s ' + '-clock unix -device usb-kbd -device usb-mouse -usb ' + '-device usb-net,netdev=mynet -netdev user,id=mynet ' + '-nographic') % ( + self.context.config.get('default_qemu_binary'), + self.device_option('qemu_machine_type'), + self._lava_image) + logging.info('launching qemu with command %r' % qemu_cmd) + self.proc = pexpect.spawn(qemu_cmd, logfile=self.sio, timeout=None) + self.proc.expect(self.tester_str, timeout=300) + # set PS1 to include return value of last command + self.proc.sendline('export PS1="$PS1 [rc=$(echo \$?)]: "') + self.proc.expect(self.tester_str, timeout=10) + + def retrieve_results(self, result_disk): + if self.proc is not None: + self.proc.sendline('sync') + self.proc.expect([self.tester_str, pexpect.TIMEOUT], timeout=10) + self.proc.close() + tardir = mkdtemp() + tarfile = os.path.join(tardir, "lava_results.tgz") + with image_partition_mounted(self._lava_image, self.root_part) as mntdir: + logging_system( + 'tar czf %s -C %s%s .' % ( + tarfile, mntdir, self.context.lava_result_dir)) + logging_system('rm %s%s/*.bundle' % (mntdir, self.context.lava_result_dir)) + return 'pass', None, tarfile === modified file 'lava_dispatcher/context.py' --- lava_dispatcher/context.py 2011-11-28 20:12:01 +0000 +++ lava_dispatcher/context.py 2011-11-30 02:31:12 +0000 @@ -20,8 +20,9 @@ import tempfile -from lava_dispatcher.client import LavaClient from lava_dispatcher.config import get_device_config +from lava_dispatcher.client.master import LavaMasterImageClient +from lava_dispatcher.client.qemu import LavaQEMUClient from lava_dispatcher.test_data import LavaTestData @@ -31,7 +32,15 @@ self.job_data = job_data device_config = get_device_config( target, dispatcher_config.config_dir) - self._client = LavaClient(self, device_config) + client_type = device_config.get('client_type') + if client_type == 'master': + self._client = LavaMasterImageClient(self, device_config) + elif client_type == 'qemu': + self._client = LavaQEMUClient(self, device_config) + else: + raise RuntimeError( + "this version of lava-dispatcher only supports master & qemu " + "clients, not %r" % device_config.get('client_type')) self.test_data = LavaTestData() self.oob_file = oob_file self._host_result_dir = None === modified file 'lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf 2011-11-15 07:17:34 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf 2011-11-30 01:12:17 +0000 @@ -5,10 +5,12 @@ # (device-types/${TYPE}.conf) or the specific device file # (devices/${DEVICE}.conf). -# The client_type. Only 'conmux' (meaning we communicate with the -# device over a serial line via conmux) is supported today but 'qemu' -# and 'ssh' are coming. -client_type = conmux +# The client_type. This determines how we connect, deploy to and +# control the booting of the device. 'master', the default, means a +# board that boots into a known good image by default but can be +# manipulated to boot from different boot and rootfs filesystems. +# 'qemu' is the other possible value at this time. +client_type = master # The bootloader commands to boot the device into the test image (we # assume that the device boots into the master image without bootloader @@ -82,3 +84,6 @@ # Defaults to device_type because that's what was used before this # option was introduced. lmc_dev_arg = %(device_type)s + +# The value to pass to qemu-system-arm's -M option. +qemu_machine_type = %(device_type)s \ No newline at end of file === modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf 2011-09-07 03:46:15 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf 2011-11-30 01:12:17 +0000 @@ -18,3 +18,5 @@ vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 init=/init androidboot.console=ttyO2'", boot + +qemu_machine_type = beaglexm \ No newline at end of file === modified file 'lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf' --- lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 2011-09-23 03:26:35 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 2011-11-30 01:12:17 +0000 @@ -27,3 +27,7 @@ # The url point to the version of lava-test to be install with pip LAVA_TEST_URL = bzr+http://bazaar.launchpad.net/~linaro-validation/lava-test/trunk/#egg=lava-test + +# The qemu command to use. Called 'default_qemu_binary' because we +# want to allow testing custom qemu binaries soon. +default_qemu_binary = qemu \ No newline at end of file === modified file 'lava_dispatcher/job.py' --- lava_dispatcher/job.py 2011-11-28 20:13:34 +0000 +++ lava_dispatcher/job.py 2011-11-30 02:31:12 +0000 @@ -26,7 +26,7 @@ from linaro_json.schema import Schema, Validator from lava_dispatcher.actions import get_all_cmds -from lava_dispatcher.client import CriticalError, GeneralError +from lava_dispatcher.client.base import CriticalError, GeneralError from lava_dispatcher.config import get_config from lava_dispatcher.context import LavaContext === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2011-10-18 03:02:02 +0000 +++ lava_dispatcher/utils.py 2011-11-24 04:07:12 +0000 @@ -25,7 +25,7 @@ import urlparse from shlex import shlex -def download(url, path=""): +def download(url, path="", verbose_failure=1): urlpath = urlparse.urlsplit(url).path filename = os.path.basename(urlpath) if path: @@ -38,7 +38,8 @@ fd.close() response.close() except: - logging.exception("download failed") + if verbose_failure: + logging.exception("download failed") raise RuntimeError("Could not retrieve %s" % url) return filename @@ -82,3 +83,8 @@ newlines_to_spaces = lambda x: x.replace('\n', ' ') strip_newlines = lambda x: newlines_to_spaces(x).strip(' ') return map(strip_newlines, list(splitter)) + +def logging_system(cmd): + logging.info('executing %r'%cmd) + return os.system(cmd) +