From patchwork Thu Sep 20 21:35:14 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 11613 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 144E623E57 for ; Thu, 20 Sep 2012 21:35:19 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 1E0D9A186BA for ; Thu, 20 Sep 2012 21:35:18 +0000 (UTC) Received: by ieje10 with SMTP id e10so3673022iej.11 for ; Thu, 20 Sep 2012 14:35:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=wX09wxEzblyN5LUzSIjNBfhX0ymek4UOBLfSIONYq4k=; b=PglMC4iY2oyrfU4WySvwnAsFZh5OpvRqXEltgOyDTOqinsEkSrxmqNUQUP/KBolAbz U1pvQCesgrM51hPAXthl7y4WOu9A2reDXSs5jZ7vZA8WxqMvEIml/zf/ascTOL7nuLIs FO4rSJbBc2ZPYHuG6oozIZYQ1UJDGzE84lHwmk3GFPzGRpfmSn5EK+nYHueJ6Te6297Y CbP58UeJSRTSFs0u9YlhUfo9O08ErjrMUA4000GBs/abUyKqNFtCNL9O+doiJarfUSra L/P8+G0CIO77qRjotWpMsiybK6mbFjU4W8XytA4YSZaGqJ0x7i0zyJpirWda0CH1ZmpY QwFQ== Received: by 10.50.217.227 with SMTP id pb3mr3718347igc.28.1348176917527; Thu, 20 Sep 2012 14:35:17 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.232 with SMTP id ex8csp98796igc; Thu, 20 Sep 2012 14:35:16 -0700 (PDT) Received: by 10.180.100.131 with SMTP id ey3mr6848412wib.15.1348176914954; Thu, 20 Sep 2012 14:35:14 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id d6si9018749wec.106.2012.09.20.14.35.14 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 20 Sep 2012 14:35:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TEoOw-000841-7L for ; Thu, 20 Sep 2012 21:35:14 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 26C54E18F7 for ; Thu, 20 Sep 2012 21:35:14 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 386 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 386: remove never-used features that claimed to support testing ubuntu and android images with custom ... Message-Id: <20120920213514.23103.97982.launchpad@ackee.canonical.com> Date: Thu, 20 Sep 2012 21:35:14 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15985"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 4b94fdb32fb2bc02af2395559412ce42437e7035 X-Gm-Message-State: ALoCoQlRP2T18pVX0vKkg3+ipuTzkpdhemZjQcNadUJovMG5oNGFGEcNxY43UFz+yfJa9OP7Qr1d Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-dispatcher/kill-kernel_matrix/+merge/123177 proposed by: Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 386 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Fri 2012-09-21 09:34:24 +1200 message: remove never-used features that claimed to support testing ubuntu and android images with custom kernels removed: doc/examples/jobs/android-new-kernel.json doc/examples/jobs/lava-new-kernel.json modified: doc/jobfile.rst lava_dispatcher/actions/android_deploy.py lava_dispatcher/actions/deploy.py lava_dispatcher/client/fastmodel.py lava_dispatcher/client/lmc_utils.py lava_dispatcher/client/master.py lava_dispatcher/client/qemu.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === removed file 'doc/examples/jobs/android-new-kernel.json' --- doc/examples/jobs/android-new-kernel.json 2012-03-09 01:18:19 +0000 +++ doc/examples/jobs/android-new-kernel.json 1970-01-01 00:00:00 +0000 @@ -1,47 +0,0 @@ -{ - "job_name": "android_new_kernel", - "target": "panda01", - "timeout": 18000, - "actions": [ - { - "command": "deploy_linaro_android_image", - "parameters": - { - "boot": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/boot.tar.bz2", - "system": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/system.tar.bz2", - "data": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/userdata.tar.bz2", - "pkg": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/171/artifact/build/out/target/product/pandaboard/boot.tar.bz2" - }, - "metadata": - { - "rootfs.type": "android", - "rootfs.build": "61" - } - }, - { - "command": "boot_linaro_android_image" - }, - { - "command": "lava_android_test_install", - "parameters": - { - "tests": ["0xbench"] - } - }, - { - "command": "lava_android_test_run", - "parameters": - { - "test_name": "0xbench" - } - }, - { - "command": "submit_results_on_host", - "parameters": - { - "server": "http://validation.linaro.org/lava-server/RPC2/", - "stream": "/anonymous/android-panda01-basic/" - } - } - ] -} === removed file 'doc/examples/jobs/lava-new-kernel.json' --- doc/examples/jobs/lava-new-kernel.json 2011-12-22 06:11:18 +0000 +++ doc/examples/jobs/lava-new-kernel.json 1970-01-01 00:00:00 +0000 @@ -1,30 +0,0 @@ -{ - "job_name": "new-kernel", - "target": "mx53loco01", - "timeout": 6000, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": - { - "rootfs": -"http://snapshots.linaro.org/11.05-daily/linaro-nano/20110612/0/images/tar/nano-n-tar-20110612-0.tar.gz", - "hwpack": -"http://snapshots.linaro.org/11.05-daily/linaro-hwpacks/lt-mx53loco/20110609/0/images/hwpack/hwpack_linaro-lt-mx53loco_20110609-0_armel_supported.tar.gz", - "kernel_matrix":[ -"http://pkgserver/original/linux-image-2.6.38-1000-linaro-lt-mx5_2.6.38-1000.7_armel.deb", "linux-image-2.6.38"] - } - }, - { - "command": "boot_linaro_image" - }, - { - "command": "submit_results", - "parameters": - { - "server": "http://validation.linaro.org/lava-server/RPC2/", - "stream": "/anonymous/testresult/" - } - } - ] -} === modified file 'doc/jobfile.rst' --- doc/jobfile.rst 2012-03-09 01:18:19 +0000 +++ doc/jobfile.rst 2012-09-06 20:41:35 +0000 @@ -49,88 +49,6 @@ ] } -Linaro Ubuntu Images with new kernel -==================================== -Here's an example showing how to test a new kernel, new kernel is specified by "kernel_matrix". First parameter is the deb package which contains a kernel, and second one is the prefix of the deb package which will be removed from existing rootfs before installing the first deb:: - - { - "job_name": "fpp", - "target": "mx53loco01", - "timeout": 6000, - "actions": [ - { - "command": "deploy_linaro_image", - "parameters": - { - "rootfs": "http://snapshots.linaro.org/11.05-daily/linaro-nano/20110612/0/images/tar/nano-n-tar-20110612-0.tar.gz", - "hwpack": "http://snapshots.linaro.org/11.05-daily/linaro-hwpacks/lt-mx53loco/20110609/0/images/hwpack/hwpack_linaro-lt-mx53loco_20110609-0_armel_supported.tar.gz", - "kernel_matrix":["http://pkgserver/original/linux-image-2.6.38-1000-linaro-lt-mx5_2.6.38-1000.7_armel.deb", "linux-image-2.6.38"] - } - }, - { - "command": "boot_linaro_image" - }, - { - "command": "submit_results", - "parameters": - { - "server": "http://validation.linaro.org/lava-server/RPC2/", - "stream": "/anonymous/testresult/" - } - } - ] - } - -And an example for Android images, it use a boot partition tarball specified by "pkg" directly, it will replace the files in tarball to boot partition:: - - { - "job_name": "android_new_kernel", - "target": "panda01", - "timeout": 18000, - "actions": [ - { - "command": "deploy_linaro_android_image", - "parameters": - { - "boot": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/boot.tar.bz2", - "system": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/system.tar.bz2", - "data": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/61/artifact/build/out/target/product/pandaboard/userdata.tar.bz2", - "pkg": "https://android-build.linaro.org/jenkins/job/linaro-android_leb-panda/171/artifact/build/out/target/product/pandaboard/boot.tar.bz2" - }, - "metadata": - { - "rootfs.type": "android", - "rootfs.build": "61" - } - }, - { - "command": "boot_linaro_android_image" - }, - { - "command": "lava_android_test_install", - "parameters": - { - "tests": ["0xbench"] - } - }, - { - "command": "lava_android_test_run", - "parameters": - { - "test_name": "0xbench" - } - }, - { - "command": "submit_results_on_host", - "parameters": - { - "server": "http://validation.linaro.org/lava-server/RPC2/", - "stream": "/anonymous/android-panda01-basic/" - } - } - ] - } - Linaro Android Images ===================== === modified file 'lava_dispatcher/actions/android_deploy.py' --- lava_dispatcher/actions/android_deploy.py 2012-07-06 01:43:50 +0000 +++ lava_dispatcher/actions/android_deploy.py 2012-09-06 20:41:35 +0000 @@ -32,14 +32,13 @@ 'boot': {'type': 'string'}, 'system': {'type': 'string'}, 'data': {'type': 'string'}, - 'pkg': {'type': 'string', 'optional': True}, 'rootfstype': {'type': 'string', 'optional': True, 'default': 'ext4'}, }, 'additionalProperties': False, } - def run(self, boot, system, data, pkg=None, rootfstype='ext4'): + def run(self, boot, system, data, rootfstype='ext4'): if not isinstance(self.client, LavaMasterImageClient) and \ not isinstance(self.client, LavaFastModelClient): raise RuntimeError("Invalid LavaClient for this action") - self.client.deploy_linaro_android(boot, system, data, pkg, rootfstype) + self.client.deploy_linaro_android(boot, system, data, rootfstype) === modified file 'lava_dispatcher/actions/deploy.py' --- lava_dispatcher/actions/deploy.py 2012-07-24 03:56:32 +0000 +++ lava_dispatcher/actions/deploy.py 2012-09-06 20:16:10 +0000 @@ -43,7 +43,6 @@ ## 'properties': { ## 'hwpack': {'type': 'string'}, ## 'rootfs': {'type': 'string'}, - ## 'kernel_matrix': {'type': 'string', 'optional': True}, ## 'rootfstype': {'type': 'string', 'optional': True, 'default': 'ext3'}, ## }, ## 'additionalProperties': False, @@ -57,7 +56,6 @@ 'hwpack': {'type': 'string', 'optional': True}, 'rootfs': {'type': 'string', 'optional': True}, 'image': {'type': 'string', 'optional': True}, - 'kernel_matrix': {'type': 'string', 'optional': True}, 'rootfstype': {'type': 'string', 'optional': True}, }, 'additionalProperties': False, @@ -73,16 +71,12 @@ raise ValueError('cannot specify image and hwpack') elif 'image' not in parameters: raise ValueError('must specify image if not specifying a hwpack') - elif 'kernel_matrix' in parameters: - raise ValueError('cannot specify kernel_matrix with an image') - def run(self, hwpack=None, rootfs=None, image=None, kernel_matrix=None, - rootfstype='ext3'): + def run(self, hwpack=None, rootfs=None, image=None, rootfstype='ext3'): if not isinstance(self.client, LavaMasterImageClient) and \ not isinstance(self.client, LavaQEMUClient) and \ not isinstance(self.client, LavaFastModelClient): raise RuntimeError("Invalid LavaClient for this action") self.client.deploy_linaro( - hwpack=hwpack, rootfs=rootfs, image=image, - kernel_matrix=kernel_matrix, rootfstype=rootfstype) + hwpack=hwpack, rootfs=rootfs, image=image, rootfstype=rootfstype) === modified file 'lava_dispatcher/client/fastmodel.py' --- lava_dispatcher/client/fastmodel.py 2012-08-23 15:00:18 +0000 +++ lava_dispatcher/client/fastmodel.py 2012-09-06 20:41:35 +0000 @@ -129,8 +129,7 @@ self._axf = '%s/%s' % (odir, os.path.split(src)[1]) shutil.copyfile(src, self._axf) - def deploy_linaro_android(self, boot, system, data, pkg=None, - rootfstype='ext4'): + def deploy_linaro_android(self, boot, system, data, rootfstype='ext4'): logging.info("Deploying Android on %s" % self.hostname) self._boot = download_image(boot, self.context, decompress=False) @@ -147,7 +146,7 @@ self._customize_android() def deploy_linaro(self, hwpack=None, rootfs=None, image=None, - kernel_matrix=None, rootfstype='ext3'): + rootfstype='ext3'): if image is None: if hwpack is None or rootfs is None: raise CriticalError( === modified file 'lava_dispatcher/client/lmc_utils.py' --- lava_dispatcher/client/lmc_utils.py 2012-08-10 10:18:06 +0000 +++ lava_dispatcher/client/lmc_utils.py 2012-09-06 20:16:10 +0000 @@ -4,11 +4,9 @@ import pexpect import re import os -import shutil from tempfile import mkdtemp import sys -from lava_dispatcher.client.base import CriticalError from lava_dispatcher.downloader import ( download_image, ) @@ -16,58 +14,16 @@ logging_system, ) -def _refresh_hwpack(client, tarball_dir, kernel_matrix, hwpack): - logging.info("Deploying new kernel") - new_kernel = kernel_matrix[0] - deb_prefix = kernel_matrix[1] - filesuffix = new_kernel.split(".")[-1] - - if filesuffix != "deb": - raise CriticalError("New kernel only support deb kernel package!") - - # download package to local - kernel_path = download_image(new_kernel, client.context, outdir, decompress=False) - hwpack_path = download_image(hwpack, client.context, outdir, decompress=False) - - cmd = ("sudo linaro-hwpack-replace -t %s -p %s -r %s" - % (hwpack_path, kernel_path, deb_prefix)) - - rc, output = getstatusoutput(cmd) - if rc: - raise RuntimeError("linaro-hwpack-replace failed: %s" % output) - - #fix it:l-h-r doesn't make a output option to specify the output hwpack, - #so it needs to do manually here - - #remove old hwpack and leave only new hwpack in tarball_dir - os.remove(hwpack_path) - hwpack_list = os.listdir(tarball_dir) - for hp in hwpack_list: - if hp.split(".")[-1] == "gz": - new_hwpack_path = os.path.join(tarball_dir, hp) - return new_hwpack_path - - -def generate_image(client, hwpack_url, rootfs_url, kernel_matrix, - outdir, rootfstype=None): + +def generate_image(client, hwpack_url, rootfs_url, outdir, rootfstype=None): """Generate image from a hwpack and rootfs url :param hwpack_url: url of the Linaro hwpack to download :param rootfs_url: url of the Linaro image to download """ - LAVA_IMAGE_TMPDIR = client.context.lava_image_tmpdir - LAVA_IMAGE_URL = client.context.lava_image_url logging.info("preparing to deploy on %s" % client.hostname) logging.info(" hwpack: %s" % hwpack_url) logging.info(" rootfs: %s" % rootfs_url) - if kernel_matrix: - logging.info(" package: %s" % kernel_matrix[0]) - hwpack_url = _refresh_hwpack(client, outdir, kernel_matrix, hwpack_url) - #make new hwpack downloadable - hwpack_url = hwpack_url.replace(LAVA_IMAGE_TMPDIR, '') - hwpack_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, hwpack_url]) - logging.info(" hwpack with new kernel: %s" % hwpack_url) logging.info("Downloading the %s file" % hwpack_url) hwpack_path = download_image(hwpack_url, client.context, outdir, decompress=False) === modified file 'lava_dispatcher/client/master.py' --- lava_dispatcher/client/master.py 2012-09-12 12:15:15 +0000 +++ lava_dispatcher/client/master.py 2012-09-20 21:34:24 +0000 @@ -134,7 +134,7 @@ _deploy_tarball_to_board(session, bootfs, '/mnt/boot') session.run('umount /mnt/boot') -def _deploy_linaro_android_testboot(session, boottbz2, pkgbz2=None): +def _deploy_linaro_android_testboot(session, boottbz2): logging.info("Deploying test boot filesystem") session.run('umount /dev/disk/by-label/testboot', failok=True) session.run('mkfs.vfat /dev/disk/by-label/testboot ' @@ -144,8 +144,6 @@ session.run('mount /dev/disk/by-label/testboot ' '/mnt/lava/boot') _deploy_tarball_to_board(session, boottbz2, '/mnt/lava') - if pkgbz2: - _deploy_tarball_to_board(session, pkgbz2, '/mnt/lava') _recreate_uInitrd(session) @@ -476,8 +474,7 @@ shutil.copy(boot_tgz, c_boot_tgz) shutil.copy(root_tgz, c_root_tgz) - def deploy_linaro(self, hwpack=None, rootfs=None, image=None, - kernel_matrix=None, rootfstype='ext3'): + def deploy_linaro(self, hwpack=None, rootfs=None, image=None, rootfstype='ext3'): LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir LAVA_IMAGE_URL = self.context.lava_image_url # validate in parameters @@ -486,7 +483,7 @@ raise CriticalError( "must specify both hwpack and rootfs when not specifying image") else: - if hwpack is not None or rootfs is not None or kernel_matrix is not None: + if hwpack is not None or rootfs is not None: raise CriticalError( "cannot specify hwpack or rootfs when specifying image") @@ -494,8 +491,7 @@ try: tarball_dir = self.get_www_scratch_dir() if image is None: - image_file = generate_image(self, hwpack, rootfs, - kernel_matrix, tarball_dir) + image_file = generate_image(self, hwpack, rootfs, tarball_dir) boot_tgz, root_tgz = self._generate_tarballs(image_file) else: os.chmod(tarball_dir, 0755) @@ -560,7 +556,7 @@ self.sio.write(tb) raise CriticalError("Deployment failed") - def deploy_linaro_android(self, boot, system, data, pkg=None, rootfstype='ext4'): + def deploy_linaro_android(self, boot, system, data, rootfstype='ext4'): LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir LAVA_IMAGE_URL = self.context.lava_image_url logging.info("Deploying Android on %s" % self.hostname) @@ -581,8 +577,8 @@ raise CriticalError("Unable to reach LAVA server, check network") try: - boot_tbz2, system_tbz2, data_tbz2, pkg_tbz2 = \ - self._download_tarballs(boot, system, data, pkg) + boot_tbz2, system_tbz2, data_tbz2 = \ + self._download_tarballs(boot, system, data) except: logging.error("Unable to download artifacts for deployment") tb = traceback.format_exc() @@ -600,15 +596,8 @@ data_url = '/'.join(u.strip('/') for u in [ LAVA_IMAGE_URL, data_tarball]) - if pkg_tbz2: - pkg_tarball = pkg_tbz2.replace(LAVA_IMAGE_TMPDIR, '') - pkg_url = '/'.join(u.strip('/') for u in [ - LAVA_IMAGE_URL, pkg_tarball]) - else: - pkg_url = None - try: - _deploy_linaro_android_testboot(session, boot_url, pkg_url) + _deploy_linaro_android_testboot(session, boot_url) _deploy_linaro_android_testrootfs(session, system_url, rootfstype) _deploy_linaro_android_data(session, data_url) if session.has_partition_with_label('userdata') and \ @@ -622,7 +611,7 @@ finally: logging.info("Android image deployment exiting") - def _download_tarballs(self, boot_url, system_url, data_url, pkg_url=None): + def _download_tarballs(self, boot_url, system_url, data_url): """Download tarballs from a boot, system and data tarball url :param boot_url: url of the Linaro Android boot tarball to download @@ -636,12 +625,8 @@ boot_path = download_image(boot_url, self.context, tarball_dir, decompress=False) system_path = download_image(system_url, self.context, tarball_dir, decompress=False) data_path = download_image(data_url, self.context, tarball_dir, decompress=False) - if pkg_url: - pkg_path = download_image(pkg_url, self.context, tarball_dir, decompress=False) - else: - pkg_path = None logging.info("Downloaded the image files") - return boot_path, system_path, data_path, pkg_path + return boot_path, system_path, data_path def boot_master_image(self): """ === modified file 'lava_dispatcher/client/qemu.py' --- lava_dispatcher/client/qemu.py 2012-08-01 17:52:55 +0000 +++ lava_dispatcher/client/qemu.py 2012-09-06 20:16:10 +0000 @@ -47,11 +47,10 @@ super(LavaQEMUClient, self).__init__(context, config) self._lava_image = None - def deploy_linaro(self, hwpack=None, rootfs=None, image=None, kernel_matrix=None, rootfstype='ext3'): + def deploy_linaro(self, hwpack=None, rootfs=None, image=None, rootfstype='ext3'): if image is None: odir = self.get_www_scratch_dir() - image_file = generate_image(self, hwpack, rootfs, kernel_matrix, - odir, rootfstype) + image_file = generate_image(self, hwpack, rootfs, odir, rootfstype) else: image_file = download_image(image, self.context) self._lava_image = image_file