From patchwork Thu Jul 18 07:40:25 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neil Williams X-Patchwork-Id: 18408 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-gg0-f197.google.com (mail-gg0-f197.google.com [209.85.161.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 650B520F44 for ; Thu, 18 Jul 2013 07:40:28 +0000 (UTC) Received: by mail-gg0-f197.google.com with SMTP id x14sf3271922ggx.8 for ; Thu, 18 Jul 2013 00:40:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-beenthere:x-forwarded-to:x-forwarded-for:delivered-to :mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state:x-original-sender :x-original-authentication-results:mailing-list:list-id :x-google-group-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=xjrs5FT00kDRdiMJX50r7p3J3fo6hSNF8iYGSAw5uhE=; b=Wm68VJN8mrPh16NMsE55/ITQdh8QWCUQInkcgkO6IrHixlArsYhWoFxu+1GpwXH4yl QuLBS7z2OAFJh4OVryj9F99zTQdohg6jw2/rrQ4GMC920q5fRxlXJf1TbXC7Jo1rtdKI 7QaUvD7WygkvZNPrDaiVhYwSFxylEvpkB3a4zgeRDgTurNIjRmEmHnvkFD4ccDLXbWnJ C/biBLABOhfR6Rx7LQWKRJ2CMtGhgXRp1cMk0Frn3NBQBJiwG1Tm6VNaBcyki/ueyalC FPgFdcyL2ja1NVPDga51pPbzLi1fm+AUXFaL7CPZ9aNe9BTDgF+CPINxrE3F6acmkAOK TjGg== X-Received: by 10.236.199.102 with SMTP id w66mr5961187yhn.41.1374133227729; Thu, 18 Jul 2013 00:40:27 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.107.129 with SMTP id hc1ls1281689qeb.78.gmail; Thu, 18 Jul 2013 00:40:27 -0700 (PDT) X-Received: by 10.52.228.33 with SMTP id sf1mr2983755vdc.86.1374133227444; Thu, 18 Jul 2013 00:40:27 -0700 (PDT) Received: from mail-ve0-f171.google.com (mail-ve0-f171.google.com [209.85.128.171]) by mx.google.com with ESMTPS id f8si2494934vec.47.2013.07.18.00.40.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 18 Jul 2013 00:40:27 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.171 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.171; Received: by mail-ve0-f171.google.com with SMTP id b10so2224681vea.30 for ; Thu, 18 Jul 2013 00:40:27 -0700 (PDT) X-Received: by 10.58.211.227 with SMTP id nf3mr3579369vec.20.1374133227154; Thu, 18 Jul 2013 00:40:27 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.149.77 with SMTP id s13csp211468vcv; Thu, 18 Jul 2013 00:40:26 -0700 (PDT) X-Received: by 10.180.108.129 with SMTP id hk1mr18227403wib.56.1374133225814; Thu, 18 Jul 2013 00:40:25 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id eq6si10346974wic.64.2013.07.18.00.40.25 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 18 Jul 2013 00:40:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Uzip7-00051D-6H for ; Thu, 18 Jul 2013 07:40:25 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 1F86BE1571 for ; Thu, 18 Jul 2013 07:40:25 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 638 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 638: Neil Williams 2013-07-17 Fix issues highlighted during review of merge Message-Id: <20130718074025.27995.72094.launchpad@ackee.canonical.com> Date: Thu, 18 Jul 2013 07:40:25 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: list X-Generated-By: Launchpad (canonical.com); Revision="16696"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 650e20dc5e453a04c96fa398721937593c9b1f99 X-Gm-Message-State: ALoCoQmyX6SkVsCnlNYQGATT9a8KlnBrvctAOyw8rRxLQ1wAoFKmYCSkWBEoWwK/6gDbh1f/STl9 X-Original-Sender: noreply@launchpad.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.171 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Merge authors: Neil Williams (codehelp) Related merge proposals: https://code.launchpad.net/~codehelp/lava-dispatcher/pep8/+merge/175231 proposed by: Neil Williams (codehelp) review: Approve - Antonio Terceiro (terceiro) ------------------------------------------------------------ revno: 638 [merge] committer: Neil Williams branch nick: lava-dispatcher timestamp: Thu 2013-07-18 08:35:51 +0100 message: Neil Williams 2013-07-17 Fix issues highlighted during review of merge proposal. Neil Williams 2013-07-17 Add PEP8 fixes for doc/ and setup.py Neil Williams 2013-07-16 PEP8 compliance and prevent local variables shadowing keywords. Neil Williams 2013-07-16 PEP8 compliance. Neil Williams 2013-07-16 PEP8 compliance and fix mutable default arguments Neil Williams 2013-07-16 PEP8 compliance. Neil Williams 2013-07-16 PEP8 Add missing default argument and allow proc to resolve locally. Neil Williams 2013-07-16 PEP8 compliance. Neil Williams 2013-07-16 Add default argument added in base class. Neil Williams 2013-07-16 PEP8 compliance. Neil Williams 2013-07-16 PEP8 compliance and add base class definitions Neil Williams 2013-07-16 PEP8 compliance. Neil Williams 2013-07-16 Typo in docs modified: doc/conf.py doc/examples/plugins/demo-action-plugin/demo_action_plugin/foo.py doc/examples/plugins/demo-action-plugin/setup.py doc/external_measurement.rst lava/dispatcher/commands.py lava_dispatcher/actions/__init__.py lava_dispatcher/actions/android_install_cts_medias.py lava_dispatcher/actions/boot_control.py lava_dispatcher/actions/deploy.py lava_dispatcher/actions/launch_control.py lava_dispatcher/actions/lava_android_test.py lava_dispatcher/actions/lava_test_shell.py lava_dispatcher/client/base.py lava_dispatcher/client/lmc_utils.py lava_dispatcher/client/targetdevice.py lava_dispatcher/config.py lava_dispatcher/context.py lava_dispatcher/device/capri.py lava_dispatcher/device/fastboot.py lava_dispatcher/device/fastmodel.py lava_dispatcher/device/ipmi_pxe.py lava_dispatcher/device/k3v2.py lava_dispatcher/device/master.py lava_dispatcher/device/nexus10.py lava_dispatcher/device/qemu.py lava_dispatcher/device/sdmux.py lava_dispatcher/device/target.py lava_dispatcher/device/uefi.py lava_dispatcher/device/vexpress.py lava_dispatcher/downloader.py lava_dispatcher/job.py lava_dispatcher/lava_test_shell.py lava_dispatcher/signals/__init__.py lava_dispatcher/signals/duration.py lava_dispatcher/signals/shellhooks.py lava_dispatcher/tarballcache.py lava_dispatcher/test_data.py lava_dispatcher/tests/__init__.py lava_dispatcher/tests/helper.py lava_dispatcher/tests/test_config.py lava_dispatcher/tests/test_device_version.py lava_dispatcher/utils.py setup.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'doc/conf.py' --- doc/conf.py 2012-11-28 21:52:36 +0000 +++ doc/conf.py 2013-07-17 09:16:24 +0000 @@ -11,7 +11,8 @@ # All configuration values have a default; values that are commented out # serve to show the default. -import sys, os +import sys +import os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -27,12 +28,12 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ - 'sphinx.ext.autodoc', - 'sphinx.ext.doctest', - 'sphinx.ext.intersphinx', - 'sphinx.ext.todo', - 'sphinx.ext.coverage', - 'sphinx.ext.viewcode'] + 'sphinx.ext.autodoc', + 'sphinx.ext.doctest', + 'sphinx.ext.intersphinx', + 'sphinx.ext.todo', + 'sphinx.ext.coverage', + 'sphinx.ext.viewcode'] # Configuration for sphinx.ext.todo @@ -189,8 +190,8 @@ # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ - ('index', 'LAVADispatcher.tex', u'LAVA Dispatcher Documentation', - u'Linaro Validation Team', 'manual'), + ('index', 'LAVADispatcher.tex', u'LAVA Dispatcher Documentation', + u'Linaro Validation Team', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of === modified file 'doc/examples/plugins/demo-action-plugin/demo_action_plugin/foo.py' --- doc/examples/plugins/demo-action-plugin/demo_action_plugin/foo.py 2011-11-14 23:09:29 +0000 +++ doc/examples/plugins/demo-action-plugin/demo_action_plugin/foo.py 2013-07-17 09:16:24 +0000 @@ -22,8 +22,9 @@ from lava_dispatcher.actions import BaseAction + class cmd_foo(BaseAction): + def run(self): """ do something """ print("Hello from demo-action-plugin") - === modified file 'doc/examples/plugins/demo-action-plugin/setup.py' --- doc/examples/plugins/demo-action-plugin/setup.py 2011-11-17 17:32:00 +0000 +++ doc/examples/plugins/demo-action-plugin/setup.py 2013-07-17 09:16:24 +0000 @@ -2,18 +2,18 @@ from setuptools import setup -setup ( +setup( name='demo-action-plugin', version='0.0.1', author='Paul Larson', author_email='paul.larson@linaro.org', url='', description='LAVA Dispatcher plugin test', - packages = ['demo_action_plugin'], + packages=['demo_action_plugin'], entry_points=""" [lava_dispatcher.actions] foo = demo_action_plugin.foo:cmd_foo """, zip_safe=False, include_package_data=True - ) +) === modified file 'doc/external_measurement.rst' --- doc/external_measurement.rst 2012-12-05 00:24:13 +0000 +++ doc/external_measurement.rst 2013-07-16 15:57:55 +0000 @@ -209,7 +209,7 @@ "parameters": { "testdef_repos": [{"bzr-repo": "lp:~linaro-validation/+junk/shell-hook-example"}], "timeout": 1800 - } + } }, .. _`lp:~linaro-validation/+junk/shell-hook-example`: http://bazaar.launchpad.net/~linaro-validation/+junk/shell-hook-example/files === modified file 'lava/dispatcher/commands.py' --- lava/dispatcher/commands.py 2013-06-11 20:27:05 +0000 +++ lava/dispatcher/commands.py 2013-07-16 15:58:16 +0000 @@ -68,7 +68,7 @@ help="Test scenario file") parser.add_argument( "--target", - default = None, + default=None, help="Run the job on a specific target device" ) @@ -109,7 +109,8 @@ if self.args.target is None: if 'target' not in json_jobdata: - logging.error("The job file does not specify a target device. You must inform one using the --target option.") + logging.error("The job file does not specify a target device. " + "You must specify one using the --target option.") exit(1) else: json_jobdata['target'] = self.args.target === modified file 'lava_dispatcher/actions/__init__.py' --- lava_dispatcher/actions/__init__.py 2012-11-22 01:00:07 +0000 +++ lava_dispatcher/actions/__init__.py 2013-07-16 15:59:00 +0000 @@ -31,7 +31,7 @@ null_or_empty_schema = { 'type': ['object', 'null'], 'additionalProperties': False, - } +} class classproperty(object): @@ -71,7 +71,8 @@ @classmethod def validate_parameters(cls, params): if cls.parameters_schema: - if params is None: params = {} + if params is None: + params = {} schema = Schema(cls.parameters_schema) Validator.validate(schema, params) === modified file 'lava_dispatcher/actions/android_install_cts_medias.py' --- lava_dispatcher/actions/android_install_cts_medias.py 2012-11-09 20:26:45 +0000 +++ lava_dispatcher/actions/android_install_cts_medias.py 2013-07-16 15:59:41 +0000 @@ -28,9 +28,9 @@ 'properties': { 'media_url': {'type': 'string', 'optional': True}, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def run(self, media_url=None, timeout=2400): if not media_url: === modified file 'lava_dispatcher/actions/boot_control.py' --- lava_dispatcher/actions/boot_control.py 2013-06-19 20:37:26 +0000 +++ lava_dispatcher/actions/boot_control.py 2013-07-16 15:59:32 +0000 @@ -33,9 +33,9 @@ 'properties': { 'options': {'type': 'array', 'items': {'type': 'string'}, 'optional': True}, - }, + }, 'additionalProperties': False, - } +} class cmd_boot_linaro_android_image(BaseAction): @@ -50,7 +50,9 @@ 'default': False, 'optional': True } - def run(self, options=[], adb_check=False, wait_for_home_screen=True): + def run(self, options=None, adb_check=False, wait_for_home_screen=True): + if not options: + options = [] client = self.client client.target_device.boot_options = options client.config.android_wait_for_home_screen = wait_for_home_screen @@ -75,7 +77,9 @@ 'default': False, 'optional': True } - def run(self, options=[], interactive_boot_cmds=False): + def run(self, options=None, interactive_boot_cmds=False): + if not options: + options = [] client = self.client if interactive_boot_cmds: client.config.boot_cmds = options === modified file 'lava_dispatcher/actions/deploy.py' --- lava_dispatcher/actions/deploy.py 2013-04-17 08:23:50 +0000 +++ lava_dispatcher/actions/deploy.py 2013-07-16 16:01:18 +0000 @@ -55,9 +55,9 @@ 'image': {'type': 'string', 'optional': True}, 'rootfstype': {'type': 'string', 'optional': True}, 'bootloader': {'type': 'string', 'optional': True, 'default': 'u_boot'}, - }, + }, 'additionalProperties': False, - } + } @classmethod def validate_parameters(cls, parameters): @@ -84,9 +84,9 @@ 'system': {'type': 'string'}, 'data': {'type': 'string'}, 'rootfstype': {'type': 'string', 'optional': True, 'default': 'ext4'}, - }, + }, 'additionalProperties': False, - } + } def run(self, boot, system, data, rootfstype='ext4'): self.client.deploy_linaro_android(boot, system, data, rootfstype) @@ -97,12 +97,12 @@ parameters_schema = { 'type': 'object', 'properties': { - 'type': {'type': 'string', 'enum':['ubuntu', 'oe', 'android', 'fedora']}, - }, + 'type': {'type': 'string', 'enum': ['ubuntu', 'oe', 'android', 'fedora']}, + }, 'additionalProperties': False, - } + } - def run(self, type): + def run(self, target_type): device = self.client.target_device device.boot_master_image() - device.deployment_data = device.target_map[type] + device.deployment_data = device.target_map[target_type] === modified file 'lava_dispatcher/actions/launch_control.py' --- lava_dispatcher/actions/launch_control.py 2013-01-30 10:30:37 +0000 +++ lava_dispatcher/actions/launch_control.py 2013-07-16 15:59:20 +0000 @@ -37,7 +37,9 @@ class GatherResultsError(Exception): - def __init__(self, msg, bundles=[]): + def __init__(self, msg, bundles=None): + if not bundles: + bundles = [] super(GatherResultsError, self).__init__(msg) self.bundles = bundles @@ -50,7 +52,8 @@ #Fix it: it's going to be deleted after transition if server.endswith("dashboard/"): server = ''.join([server, "xml-rpc/"]) - logging.warn("Please use whole endpoint URL not just end with 'dashboard/', 'xml-rpc/' is added automatically now!!!") + logging.warn("Please use whole endpoint URL not just end with 'dashboard/', " + "'xml-rpc/' is added automatically now!!!") parsed_server = urlparse.urlparse(server) auth_backend = MemoryAuthBackend([]) @@ -96,9 +99,9 @@ 'stream': {'type': 'string'}, 'result_disk': {'type': 'string', 'optional': True}, 'token': {'type': 'string', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def _get_bundles(self, files): bundles = [] @@ -208,9 +211,9 @@ def combine_bundles(self, all_bundles): if not all_bundles: main_bundle = { - "test_runs": [], - "format": "Dashboard Bundle Format 1.6" - } + "test_runs": [], + "format": "Dashboard Bundle Format 1.6" + } else: main_bundle = all_bundles.pop(0) test_runs = main_bundle['test_runs'] === modified file 'lava_dispatcher/actions/lava_android_test.py' --- lava_dispatcher/actions/lava_android_test.py 2013-04-23 10:33:49 +0000 +++ lava_dispatcher/actions/lava_android_test.py 2013-07-17 09:16:46 +0000 @@ -42,13 +42,12 @@ 'test_name': {'type': 'string'}, 'option': {'type': 'string', 'optional': True}, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def test_name(self, test_name, option=None, timeout=-1): - return super(cmd_lava_android_test_run, self).test_name() + \ - ' (%s)' % test_name + return super(cmd_lava_android_test_run, self).test_name() + ' (%s)' % test_name def run(self, test_name, option=None, timeout=-1): #Make sure in test image now @@ -56,9 +55,9 @@ with self.client.android_tester_session() as session: bundle_name = generate_bundle_file_name(test_name) cmds = ["lava-android-test", 'run', test_name, - '-s', session.dev_name, - '-o', '%s/%s.bundle' % (self.context.host_result_dir, - bundle_name)] + '-s', session.dev_name, + '-o', '%s/%s.bundle' % (self.context.host_result_dir, + bundle_name)] if option is not None: cmds.extend(['-O', option]) if timeout != -1: @@ -72,8 +71,8 @@ t.join() if rc == 124: raise TimeoutError( - "The test case(%s) on device(%s) times out" % ( - test_name, session.dev_name)) + "The test case(%s) on device(%s) timed out" % ( + test_name, session.dev_name)) elif rc != 0: raise OperationFailed( "Failed to run test case(%s) on device(%s) with return " @@ -86,15 +85,15 @@ 'type': 'object', 'properties': { 'commands': {'type': 'array', 'items': {'type': 'string'}, - 'optional': True}, + 'optional': True}, 'command_file': {'type': 'string', 'optional': True}, 'parser': {'type': 'string', 'optional': True}, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } - def test_name(self, commands=[], command_file=None, parser=None, + def test_name(self, commands=None, command_file=None, parser=None, timeout=-1): if commands: return '%s (commands=[%s])' % ( @@ -102,10 +101,15 @@ ','.join(commands)) elif command_file: return '%s (command-file=%s)' % ( - super(cmd_lava_android_test_run_custom, self).test_name(), - command_file) + super(cmd_lava_android_test_run_custom, self).test_name(), command_file) - def run(self, commands=[], command_file=None, parser=None, timeout=-1): + def run(self, commands=None, command_file=None, parser=None, timeout=-1): + """ + :param commands: a list of commands + :param command_file: a file containing commands + :param parser: The parser to use for the test + :param timeout: The timeout to apply. + """ #Make sure in test image now self.check_lava_android_test_installed() if commands or command_file: @@ -134,8 +138,8 @@ rc = self.context.run_command(cmds) if rc == 124: raise TimeoutError( - "The test (%s) on device(%s) times out." % ( - ' '.join(cmds), session.dev_name)) + "The test (%s) on device(%s) timed out." % ( + ' '.join(cmds), session.dev_name)) elif rc != 0: raise OperationFailed( "Failed to run test custom case[%s] on device(%s)" @@ -144,26 +148,24 @@ class cmd_lava_android_test_run_monkeyrunner(AndroidTestAction): - ''' + """ This action is added to make doing the monkeyrunner script test more easily from android build page. With this action, we only need to specify the url of the repository where the monkeyrunner script are stored. Then lava-android-test will run all the monkeyrunner scripts in that repository, and help to gather all the png files genereated when run - ''' + """ parameters_schema = { 'type': 'object', 'properties': { 'url': {'type': 'string'}, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def test_name(self, url=None, timeout=-1): - return '%s (url=[%s])' % ( - super(cmd_lava_android_test_run_monkeyrunner, self).test_name(), - url) + return '%s (url=[%s])' % (super(cmd_lava_android_test_run_monkeyrunner, self).test_name(), url) def run(self, url=None, timeout=-1): #Make sure in test image now @@ -181,9 +183,7 @@ logging.info("Execute command on host: %s" % (' '.join(cmds))) rc = self.context.run_command(cmds) if rc == 124: - raise TimeoutError( - "Failed to run monkeyrunner test url[%s] " - "on device(%s)" % (url, session.dev_name)) + raise TimeoutError("Failed to run monkeyrunner test url[%s] on device(%s)" % (url, session.dev_name)) elif rc != 0: raise OperationFailed( "Failed to run monkeyrunner test url[%s] on device(%s)" @@ -201,9 +201,9 @@ 'tests': {'type': 'array', 'items': {'type': 'string'}}, 'option': {'type': 'string', 'optional': True}, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def run(self, tests, option=None, timeout=2400): self.check_lava_android_test_installed() @@ -222,8 +222,7 @@ if rc == 124: raise OperationFailed( "The installation of test case(%s)" - " on device(%s) times out" % (test, - session.dev_name)) + " on device(%s) timed out" % (test, session.dev_name)) elif rc != 0: raise OperationFailed( "Failed to install test case(%s) on device(%s) with " === modified file 'lava_dispatcher/actions/lava_test_shell.py' --- lava_dispatcher/actions/lava_test_shell.py 2013-06-21 15:48:44 +0000 +++ lava_dispatcher/actions/lava_test_shell.py 2013-07-17 09:16:46 +0000 @@ -127,7 +127,6 @@ from lava_dispatcher import utils from lava_dispatcher.actions import BaseAction -from lava_dispatcher.client.base import wait_for_prompt from lava_dispatcher.device.target import Target from lava_dispatcher.downloader import download_image @@ -156,6 +155,7 @@ # 755 file permissions XMOD = stat.S_IRWXU | stat.S_IXGRP | stat.S_IRGRP | stat.S_IXOTH | stat.S_IROTH + def _get_testdef_git_repo(testdef_repo, tmpdir, revision): cwd = os.getcwd() gitdir = os.path.join(tmpdir, 'gittestrepo') @@ -241,6 +241,8 @@ def load_from_repo(self, testdef_repo): tmpdir = utils.mkdtemp(self.tmpbase) + repo = None + info = None if 'git-repo' in testdef_repo: repo = _get_testdef_git_repo( testdef_repo['git-repo'], tmpdir, testdef_repo.get('revision')) @@ -253,6 +255,8 @@ name = testdef_repo['bzr-repo'].replace('lp:', '').split('/')[-1] info = _bzr_info(testdef_repo['bzr-repo'], repo, name) + if not repo or not info: + logging.debug("Unable to identify specified repository. %s" % testdef_repo) test = testdef_repo.get('testdef', 'lavatest.yaml') with open(os.path.join(repo, test), 'r') as f: logging.info('loading test definition ...') @@ -273,7 +277,7 @@ 'branch_vcs': 'bzr', 'branch_revision': revno, 'branch_url': url, - } + } finally: os.chdir(cwd) @@ -289,7 +293,7 @@ 'branch_vcs': 'git', 'branch_revision': commit_id, 'branch_url': url, - } + } finally: os.chdir(cwd) @@ -344,7 +348,8 @@ # have non-reproducible behavior because it may rely on # bzr whoami value, presence of ssh keys, etc. subprocess.check_call(['bzr', 'branch', repo], - env={'BZR_HOME': '/dev/null', 'BZR_LOG': '/dev/null'}) + env={'BZR_HOME': '/dev/null', + 'BZR_LOG': '/dev/null'}) name = repo.replace('lp:', '').split('/')[-1] self._sw_sources.append(_bzr_info(repo, name, name)) @@ -412,8 +417,8 @@ f.write('read\n') steps = self.testdef['run'].get('steps', []) if steps: - for cmd in steps: - f.write('%s\n' % cmd) + for cmd in steps: + f.write('%s\n' % cmd) f.write('echo ""\n') f.write('#wait for an ack from the dispatcher\n') f.write('read\n') @@ -456,22 +461,22 @@ 'testdef_repos': {'type': 'array', 'items': {'type': 'object', 'properties': - {'git-repo': {'type': 'string', - 'optional': True}, - 'bzr-repo': {'type': 'string', - 'optional': True}, - 'revision': {'type': 'string', - 'optional': True}, - 'testdef': {'type': 'string', - 'optional': True} - }, + {'git-repo': {'type': 'string', + 'optional': True}, + 'bzr-repo': {'type': 'string', + 'optional': True}, + 'revision': {'type': 'string', + 'optional': True}, + 'testdef': {'type': 'string', + 'optional': True} + }, 'additionalProperties': False}, 'optional': True }, 'timeout': {'type': 'integer', 'optional': True}, - }, + }, 'additionalProperties': False, - } + } def run(self, testdef_urls=None, testdef_repos=None, timeout=-1): target = self.client.target_device @@ -499,11 +504,11 @@ def _keep_running(self, runner, timeout, signal_director): patterns = [ - ': exiting', - pexpect.EOF, - pexpect.TIMEOUT, - ']+)>', - ] + ': exiting', + pexpect.EOF, + pexpect.TIMEOUT, + ']+)>', + ] idx = runner._connection.expect(patterns, timeout=timeout) if idx == 0: @@ -609,4 +614,3 @@ prefix='lava-test-shell', suffix='.bundle', dir=rdir) with os.fdopen(fd, 'w') as f: DocumentIO.dump(f, bundle) - === modified file 'lava_dispatcher/client/base.py' --- lava_dispatcher/client/base.py 2013-05-22 16:15:22 +0000 +++ lava_dispatcher/client/base.py 2013-07-16 16:02:15 +0000 @@ -86,7 +86,7 @@ self.match_id = None self.match = None - def wait_for_prompt(self, timeout = -1): + def wait_for_prompt(self, timeout=-1): wait_for_prompt(self._connection, self._prompt_str, timeout) def run(self, cmd, response=None, timeout=-1, @@ -227,7 +227,7 @@ self.wait_until_attached() def _setup_adb_over_usb(self): - self.run('getprop ro.serialno', response = ['[0-9A-Fa-f]{16}']) + self.run('getprop ro.serialno', response=['[0-9A-Fa-f]{16}']) self.dev_name = self.match.group(0) def disconnect(self): @@ -350,6 +350,7 @@ self.proc = None # used for apt-get in lava-test.py self.aptget_cmd = "apt-get" + self.target_device = None @contextlib.contextmanager def tester_session(self): @@ -412,11 +413,17 @@ self.proc.sendline("export http_proxy=%s" % lava_proxy) self.proc.expect(prompt_str, timeout=30) self.aptget_cmd = ' '.join([self.aptget_cmd, - "-o Acquire::http::proxy=%s" % lava_proxy]) + "-o Acquire::http::proxy=%s" % lava_proxy]) def boot_master_image(self): raise NotImplementedError(self.boot_master_image) + def _boot_linaro_image(self): + pass + + def _boot_linaro_android_image(self): + pass + def boot_linaro_image(self): """ Reboot the system to the test image @@ -441,7 +448,7 @@ try: wait_for_prompt(self.proc, TESTER_PS1_PATTERN, timeout=timeout) - except (pexpect.TIMEOUT) as e: + except pexpect.TIMEOUT as e: msg = "Timeout waiting for boot prompt: %s" % e logging.info(msg) attempts += 1 @@ -462,7 +469,7 @@ return utils.mkdtemp(self.context.config.lava_image_tmpdir) def get_test_data_attachments(self): - '''returns attachments to go in the "lava_results" test run''' + """returns attachments to go in the "lava_results" test run""" return [] def retrieve_results(self, result_disk): @@ -546,7 +553,6 @@ if not in_linaro_android_image: raise OperationFailed("booting into android test image failed") - #check if the adb connection can be created. #by adb connect dev_ip command if adb_check: @@ -565,7 +571,7 @@ session.wait_home_screen() except: # ignore home screen exception if it is a health check job. - if not (self.context.job_data.has_key("health_check") and self.context.job_data["health_check"] == True): + if not ('health_check' in self.context.job_data and self.context.job_data["health_check"] is True): raise else: logging.info("Skip raising exception on the home screen has not displayed for health check jobs") @@ -582,7 +588,6 @@ session.run("netcfg %s dhcp" % self.config.default_network_interface, timeout=300) session.run("ifconfig " + self.config.default_network_interface, timeout=20) - def _enable_adb_over_tcp(self): logging.info("Enabling ADB over TCP") session = AndroidTesterCommandRunner(self) @@ -595,5 +600,3 @@ logging.info("Disabling adb over USB") session = AndroidTesterCommandRunner(self) session.run('echo 0>/sys/class/android_usb/android0/enable') - - === modified file 'lava_dispatcher/client/lmc_utils.py' --- lava_dispatcher/client/lmc_utils.py 2013-06-18 23:22:13 +0000 +++ lava_dispatcher/client/lmc_utils.py 2013-07-16 16:03:11 +0000 @@ -5,14 +5,13 @@ import re import os from tempfile import mkdtemp -import sys from lava_dispatcher.downloader import ( download_image, - ) +) from lava_dispatcher.utils import ( logging_system, - ) +) def generate_image(client, hwpack_url, rootfs_url, outdir, bootloader='u_boot', rootfstype=None, @@ -41,7 +40,7 @@ image_file = os.path.join(outdir, "lava.img") - logging.info("client.device_type = %s" %client.config.device_type) + logging.info("client.device_type = %s" % client.config.device_type) cmd = ("sudo flock /var/lock/lava-lmc.lck linaro-media-create --hwpack-force-yes --dev %s " "--image-file %s --binary %s --hwpack %s --image-size 3G --bootloader %s" % @@ -58,22 +57,23 @@ _run_linaro_media_create(client.context, cmd) return image_file + def generate_fastmodel_image(context, hwpack, rootfs, odir, bootloader='u_boot', size="2000M"): cmd = ("flock /var/lock/lava-lmc.lck sudo linaro-media-create " "--dev vexpress --output-directory %s --image-size %s " - "--hwpack %s --binary %s --hwpack-force-yes --bootloader %s" % - (odir, size, hwpack, rootfs, bootloader) ) + "--hwpack %s --binary %s --hwpack-force-yes --bootloader %s" % (odir, size, hwpack, rootfs, bootloader)) logging.info("Generating fastmodel image with: %s" % cmd) _run_linaro_media_create(context, cmd) + def generate_android_image(context, device, boot, data, system, ofile, size="2000M"): cmd = ("flock /var/lock/lava-lmc.lck linaro-android-media-create " "--dev %s --image_file %s --image_size %s " - "--boot %s --userdata %s --system %s" % - (device, ofile, size, boot, data, system) ) + "--boot %s --userdata %s --system %s" % (device, ofile, size, boot, data, system)) logging.info("Generating android image with: %s" % cmd) _run_linaro_media_create(context, cmd) + def get_partition_offset(image, partno): cmd = 'parted %s -m -s unit b print' % image part_data = getoutput(cmd) @@ -102,6 +102,7 @@ logging_system('sudo umount ' + mntdir) logging_system('rm -rf ' + mntdir) + def _run_linaro_media_create(context, cmd): """Run linaro-media-create and accept licenses thrown up in the process. """ @@ -130,62 +131,61 @@ 'waiting': { 'expectations': { "linaro-hwpack-install": 'default', - }, + }, 'timeout': 86400, - }, + }, 'default': { 'expectations': { "TI TSPA Software License Agreement": 'accept-tspa', "SNOWBALL CLICK-WRAP": 'accept-snowball', "LIMITED LICENSE AGREEMENT FOR APPLICATION DEVELOPERS": 'accept-snowball', - }, + }, 'timeout': 3600, - }, + }, 'accept-tspa': { 'expectations': {"": 'accept-tspa-1'}, 'timeout': 1, - }, + }, 'accept-tspa-1': { 'input': "\t ", 'expectations': { "Accept TI TSPA Software License Agreement": 'say-yes', - }, + }, 'timeout': 1, - }, + }, 'say-yes': { 'expectations': { " <(Yes|Ok)>": 'say-yes-tab', "\\033\[41m<(Yes|Ok)>": 'say-yes-space', - }, + }, 'timeout': 1, - }, + }, 'say-yes-tab': { 'input': "\t", 'expectations': { ".": 'say-yes', - }, + }, 'timeout': 1, - }, + }, 'say-yes-space': { 'input': " ", 'expectations': { ".": 'default', - }, + }, 'timeout': 1, - }, + }, 'accept-snowball': { 'expectations': {"": 'accept-snowball-1'}, 'timeout': 1, - }, + }, 'accept-snowball-1': { 'input': "\t ", 'expectations': { "Do you accept": 'say-yes', - }, + }, 'timeout': 1, - }, - } - + }, + } state = 'waiting' @@ -205,4 +205,3 @@ state = next_state_names[match_id] if state is None: return - === modified file 'lava_dispatcher/client/targetdevice.py' --- lava_dispatcher/client/targetdevice.py 2013-04-23 09:06:25 +0000 +++ lava_dispatcher/client/targetdevice.py 2013-07-16 16:01:28 +0000 @@ -57,7 +57,7 @@ "must specify both hwpack and rootfs when not specifying image") elif hwpack is not None or rootfs is not None: raise CriticalError( - "cannot specify hwpack or rootfs when specifying image") + "cannot specify hwpack or rootfs when specifying image") if image is None: self.target_device.deploy_linaro(hwpack, rootfs, bootloader) === modified file 'lava_dispatcher/config.py' --- lava_dispatcher/config.py 2013-07-01 01:46:25 +0000 +++ lava_dispatcher/config.py 2013-07-16 16:07:45 +0000 @@ -96,7 +96,8 @@ android_adb_over_tcp = schema.BoolOption(default=True) android_adb_port = schema.StringOption(default="5555") android_wait_for_home_screen = schema.BoolOption(default=True) - android_wait_for_home_screen_activity = schema.StringOption(default="Displayed com.android.launcher/com.android.launcher2.Launcher:") + android_wait_for_home_screen_activity = schema.StringOption( + default="Displayed com.android.launcher/com.android.launcher2.Launcher:") android_home_screen_timeout = schema.IntOption(default=1800) android_boot_prompt_timeout = schema.IntOption(default=1200) android_orig_block_device = schema.StringOption(default="mmcblk0") @@ -121,6 +122,7 @@ ecmeip = schema.StringOption() + class OptionDescriptor(object): def __init__(self, name): self.name = name @@ -175,6 +177,7 @@ custom_config_path = None + def search_path(): if custom_config_path: return [ @@ -188,6 +191,7 @@ default_config_path, ] + def write_path(): """ Returns the configuration directories where configuration files should be @@ -206,6 +210,7 @@ # dispatcher looking for them at ~root. return [system_config_path, user_config_path] + def _read_into(path, cp): s = StringIO.StringIO() s.write('[__main__]\n') @@ -307,6 +312,7 @@ devices.append(get_device_config(d)) return devices + def get_config_file(config_file): for config_dir in search_path(): config_file_path = os.path.join(config_dir, config_file) === modified file 'lava_dispatcher/context.py' --- lava_dispatcher/context.py 2013-06-28 12:34:59 +0000 +++ lava_dispatcher/context.py 2013-07-16 16:07:31 +0000 @@ -31,7 +31,7 @@ from lava_dispatcher.utils import ( logging_spawn, rmtree, - ) +) class Flusher(object): @@ -48,6 +48,7 @@ def __getattr__(self, name): return getattr(self.stream, name) + class Outputter(object): """ Handles the problem of where to send the output. Always sends to stdout, === modified file 'lava_dispatcher/device/capri.py' --- lava_dispatcher/device/capri.py 2013-06-18 19:34:56 +0000 +++ lava_dispatcher/device/capri.py 2013-07-16 16:04:57 +0000 @@ -22,13 +22,17 @@ from lava_dispatcher.device.target import ( Target ) +from lava_dispatcher.errors import ( + CriticalError, +) from lava_dispatcher.device.fastboot import ( FastbootTarget ) from lava_dispatcher.device.master import ( - MasterImageTarget + MasterImageTarget ) + class CapriTarget(FastbootTarget, MasterImageTarget): def __init__(self, context, config): @@ -47,7 +51,6 @@ self._enter_bootloader() self.proc.sendline("fastboot") - def deploy_android(self, boot, system, userdata): boot = self._get_image(boot) @@ -69,13 +72,13 @@ self._enter_fastboot() self.fastboot('reboot') self.proc.expect(self.context.device_config.master_str, - timeout=300) + timeout=300) # The capri does not yet have adb support, so we do not wait for adb. #self._adb('wait-for-device') self._booted = True - self.proc.sendline("") # required to put the adb shell in a reasonable state + self.proc.sendline("") # required to put the adb shell in a reasonable state self.proc.sendline("export PS1='%s'" % self.deployment_data['TESTER_PS1']) self._runner = self._get_runner(self.proc) === modified file 'lava_dispatcher/device/fastboot.py' --- lava_dispatcher/device/fastboot.py 2013-06-20 00:53:44 +0000 +++ lava_dispatcher/device/fastboot.py 2013-07-16 16:04:17 +0000 @@ -133,7 +133,7 @@ self._booted = True proc = self._adb('shell', spawn=True) - proc.sendline("") # required to put the adb shell in a reasonable state + proc.sendline("") # required to put the adb shell in a reasonable state proc.sendline("export PS1='%s'" % self.deployment_data['TESTER_PS1']) self._runner = self._get_runner(proc) @@ -192,8 +192,8 @@ @property def working_dir(self): - if (self.config.shared_working_directory is None or - self.config.shared_working_directory.strip() == ''): + if self.config.shared_working_directory is None or \ + self.config.shared_working_directory.strip() == '': return self.scratch_dir if self._working_dir is None: === modified file 'lava_dispatcher/device/fastmodel.py' --- lava_dispatcher/device/fastmodel.py 2013-07-01 10:45:36 +0000 +++ lava_dispatcher/device/fastmodel.py 2013-07-16 16:04:31 +0000 @@ -35,18 +35,18 @@ image_partition_mounted, generate_android_image, generate_fastmodel_image, - ) +) from lava_dispatcher.downloader import ( download_image, - ) +) from lava_dispatcher.test_data import ( create_attachment, - ) +) from lava_dispatcher.utils import ( ensure_directory, extract_targz, DrainConsoleOutput, - ) +) class FastModelTarget(Target): @@ -105,24 +105,24 @@ # Extract the bootwrapper from the image if self.config.simulator_axf_files and self._axf is None: self._axf = self._copy_first_find_from_list(subdir, odir, - self.config.simulator_axf_files) + self.config.simulator_axf_files) # Extract the kernel from the image if self.config.simulator_kernel_files and self._kernel is None: self._kernel = self._copy_first_find_from_list(subdir, odir, - self.config.simulator_kernel_files) + self.config.simulator_kernel_files) # Extract the initrd from the image if self.config.simulator_initrd_files and self._initrd is None: self._initrd = self._copy_first_find_from_list(subdir, odir, - self.config.simulator_initrd_files) + self.config.simulator_initrd_files) # Extract the dtb from the image if self.config.simulator_dtb and self._dtb is None: self._dtb = self._find_and_copy( - subdir, odir, self.config.simulator_dtb) + subdir, odir, self.config.simulator_dtb) elif self._bootloader == 'uefi': # Extract the uefi binary from the image if self.config.simulator_uefi and self._uefi is None: self._uefi = self._find_and_copy( - subdir, odir, self.config.simulator_uefi) + subdir, odir, self.config.simulator_uefi) def _check_needed_files(self): if self._bootloader == 'u_boot': @@ -159,7 +159,7 @@ generate_android_image( self.context, 'vexpress-a9', self._boot, self._data, self._system, self._sd_image - ) + ) self._copy_needed_files_from_partition(self.config.boot_part, '') @@ -204,11 +204,11 @@ extract_targz(tb, '%s/%s' % (mntdir, directory)) def _fix_perms(self): - ''' The directory created for the image download/creation gets created + """ The directory created for the image download/creation gets created with tempfile.mkdtemp which grants permission only to the creator of the directory. We need group access because the dispatcher may run the simulator as a different user - ''' + """ d = os.path.dirname(self._sd_image) os.chmod(d, stat.S_IRWXG | stat.S_IRWXU) os.chmod(self._sd_image, stat.S_IRWXG | stat.S_IRWXU) @@ -310,7 +310,7 @@ return self.proc def get_test_data_attachments(self): - '''returns attachments to go in the "lava_results" test run''' + """returns attachments to go in the "lava_results" test run""" # if the simulator never got started we won't even get to a logfile if getattr(self._sim_proc, 'logfile', None) is not None: if getattr(self._sim_proc.logfile, 'getvalue', None) is not None: === modified file 'lava_dispatcher/device/ipmi_pxe.py' --- lava_dispatcher/device/ipmi_pxe.py 2013-07-15 17:44:58 +0000 +++ lava_dispatcher/device/ipmi_pxe.py 2013-07-16 16:06:42 +0000 @@ -22,11 +22,8 @@ import contextlib import logging import os -import pexpect import time -from lava_dispatcher import tarballcache - from lava_dispatcher.device.master import ( MasterCommandRunner, ) @@ -34,14 +31,13 @@ Target ) from lava_dispatcher.errors import ( - NetworkError, CriticalError, OperationFailed, ) from lava_dispatcher.downloader import ( download_image, download_with_retry, - ) +) from lava_dispatcher.utils import ( mk_targz, rmtree, @@ -61,7 +57,7 @@ super(IpmiPxeTarget, self).__init__(context, config) self.proc = self.context.spawn(self.config.connection_command, timeout=1200) self.device_version = None - if self.config.ecmeip == None: + if self.config.ecmeip is None: msg = "The ecmeip address is not set for this target" logging.error(msg) raise CriticalError(msg) @@ -117,7 +113,7 @@ runner.run('mount -t tmpfs -o size=100%% tmpfs %s' % build_dir) runner.run('wget -O %s %s' % (image_file_base, image_url), timeout=1800) - if decompression_cmd != None: + if decompression_cmd is not None: cmd = '%s %s | dd bs=4M of=%s' % (decompression_cmd, image_file_base, device) else: cmd = 'dd bs=4M if=%s of=%s' % (image_file_base, device) @@ -141,7 +137,7 @@ partno = self.config.root_part start = None - runner.run('parted -s /dev/sda print', + runner.run('parted -s /dev/sda print', response='\s+%s\s+([0-9.]+.B)\s+\S+\s+\S+\s+primary\s+(\S+)' % partno, wait_prompt=False) if runner.match_id != 0: @@ -151,7 +147,7 @@ start = runner.match.group(1) parttype = runner.match.group(2) - if parttype == 'ext2' or parttype == 'ext3' or parttype == 'ext4': + if parttype == 'ext2' or parttype == 'ext3' or parttype == 'ext4': runner.run('parted -s /dev/sda rm %s' % partno) runner.run('parted -s /dev/sda mkpart primary %s 100%%' % start) runner.run('resize2fs -f /dev/sda%s' % partno) @@ -163,7 +159,7 @@ @contextlib.contextmanager def file_system(self, partition, directory): logging.info('attempting to access master filesystem %r:%s' % - (partition, directory)) + (partition, directory)) assert directory != '/', "cannot mount entire partition" @@ -181,7 +177,7 @@ runner.run('cd /tmp') # need to be in same dir as fs.tgz url_base = runner.start_http_server() - + url = url_base + '/fs.tgz' logging.info("Fetching url: %s" % url) tf = download_with_retry(self.context, self.scratch_dir, url, False) @@ -212,7 +208,7 @@ url = self.context.config.lava_image_url tar_file = tar_file.replace(tmpdir, '') tar_url = '/'.join(u.strip('/') for u in [url, tar_file]) - self._target_extract_url(runner,tar_url,dest,timeout=timeout) + self._target_extract_url(runner, tar_url, dest, timeout=timeout) def _target_extract_url(self, runner, tar_url, dest, timeout=-1): decompression_cmd = '' @@ -226,8 +222,8 @@ raise RuntimeError('bad file extension: %s' % tar_url) runner.run('wget -O - %s %s | /bin/tar -C %s -xmf -' - % (tar_url, decompression_cmd, dest), - timeout=timeout) + % (tar_url, decompression_cmd, dest), + timeout=timeout) @contextlib.contextmanager def _as_master(self): @@ -238,7 +234,7 @@ runner = BusyboxHttpdMasterCommandRunner(self) runner.run(". /scripts/functions") - runner.run("DEVICE=%s configure_networking" % + runner.run("DEVICE=%s configure_networking" % self.config.default_network_interface) # we call dhclient even though configure_networking above already @@ -257,7 +253,7 @@ try: yield runner finally: - logging.debug("deploy done") + logging.debug("deploy done") target_class = IpmiPxeTarget @@ -267,27 +263,26 @@ """A CommandRunner to use when the target is booted into the master image. """ http_pid = None - + def __init__(self, target): super(BusyboxHttpdMasterCommandRunner, self).__init__(target) def start_http_server(self): master_ip = self.get_master_ip() - if self.http_pid != None: + if self.http_pid is not None: raise OperationFailed("busybox httpd already running with pid %d" % self.http_pid) # busybox produces no output to parse for, so run it in the bg and get its pid self.run('busybox httpd -f &') - self.run('echo pid:$!:pid',response="pid:(\d+):pid",timeout=10) + self.run('echo pid:$!:pid', response="pid:(\d+):pid", timeout=10) if self.match_id != 0: raise OperationFailed("busybox httpd did not start") else: self.http_pid = self.match.group(1) - url_base = "http://%s" % (master_ip) + url_base = "http://%s" % master_ip return url_base def stop_http_server(self): - if self.http_pid == None: + if self.http_pid is None: raise OperationFailed("busybox httpd not running, but stop_http_server called.") self.run('kill %s' % self.http_pid) self.http_pid = None - === modified file 'lava_dispatcher/device/k3v2.py' --- lava_dispatcher/device/k3v2.py 2013-06-11 15:43:17 +0000 +++ lava_dispatcher/device/k3v2.py 2013-07-16 16:03:23 +0000 @@ -28,6 +28,10 @@ from lava_dispatcher.utils import ( connect_to_serial, ) +from lava_dispatcher.errors import ( + CriticalError, +) + class K3V2Target(FastbootTarget): @@ -41,7 +45,7 @@ userdata = self._get_image(userdata) self.fastboot.enter() - # Need to sleep and wait for the first stage bootloaders to initialize. + # Need to sleep and wait for the first stage bootloaders to initialize. sleep(10) self.fastboot.flash('boot', boot) self.fastboot.flash('system', system) @@ -59,14 +63,13 @@ self.fastboot.enter() self.fastboot('reboot') proc = connect_to_serial(self.context) - proc.expect(self.context.device_config.master_str, - timeout=300) + proc.expect(self.context.device_config.master_str, timeout=300) # The k3v2 does not yet have adb support, so we do not wait for adb. #self._adb('wait-for-device') self._booted = True - proc.sendline("") # required to put the adb shell in a reasonable state + proc.sendline("") # required to put the adb shell in a reasonable state proc.sendline("export PS1='%s'" % self.deployment_data['TESTER_PS1']) self._runner = self._get_runner(proc) === modified file 'lava_dispatcher/device/master.py' --- lava_dispatcher/device/master.py 2013-05-23 11:08:00 +0000 +++ lava_dispatcher/device/master.py 2013-07-17 09:16:46 +0000 @@ -85,7 +85,7 @@ 'oe': Target.oe_deployment_data, 'ubuntu': Target.ubuntu_deployment_data, 'fedora': Target.fedora_deployment_data, - } + } self.master_ip = None self.device_version = None @@ -128,7 +128,7 @@ self._deploy_android_tarballs(master, boot, system, data) if master.has_partition_with_label('userdata') and \ - master.has_partition_with_label('sdcard'): + master.has_partition_with_label('sdcard'): _purge_linaro_android_sdcard(master) self.deployment_data = Target.android_deployment_data @@ -192,7 +192,7 @@ def _rewrite_boot_cmds(self, boot_cmds): """ Returns boot_cmds list after rewriting things such as: - + * partition number from n to n + testboot_offset * root=LABEL=testrootfs instead of root=UUID=ab34-... """ @@ -202,7 +202,7 @@ pattern = "\s+\d+:(?P\d+)\s+" boot_cmds = re.sub( pattern, self._rewrite_partition_number, boot_cmds, re.MULTILINE) - + return boot_cmds.split('\n') def _read_boot_cmds(self, image=None, boot_tgz=None): @@ -246,7 +246,7 @@ logging.info("Format testboot and testrootfs partitions") runner.run('umount /dev/disk/by-label/testrootfs', failok=True) runner.run('mkfs -t %s -q /dev/disk/by-label/testrootfs -L testrootfs' - % fstype, timeout=1800) + % fstype, timeout=1800) runner.run('umount /dev/disk/by-label/testboot', failok=True) runner.run('mkfs.vfat /dev/disk/by-label/testboot -n testboot') @@ -288,7 +288,7 @@ return except (OperationFailed, pexpect.TIMEOUT): logging.warning(("transfering %s failed. %d retry left." - % (tar_url, num_retry - 1))) + % (tar_url, num_retry - 1))) if num_retry > 1: # send CTRL C in case wget still hasn't exited. @@ -299,7 +299,7 @@ sleep_time = 60 logging.info("Wait %d second before retry" % sleep_time) time.sleep(sleep_time) - num_retry = num_retry - 1 + num_retry -= 1 raise RuntimeError('extracting %s on target failed' % tar_url) @@ -321,7 +321,7 @@ @contextlib.contextmanager def file_system(self, partition, directory): logging.info('attempting to access master filesystem %r:%s' % - (partition, directory)) + (partition, directory)) assert directory != '/', "cannot mount entire partition" @@ -336,7 +336,7 @@ parent_dir, target_name = os.path.split(targetdir) runner.run('tar -czf /tmp/fs.tgz -C %s %s' % - (parent_dir, target_name)) + (parent_dir, target_name)) runner.run('cd /tmp') # need to be in same dir as fs.tgz self.proc.sendline('python -m SimpleHTTPServer 0 2>/dev/null') match_id = self.proc.expect([ @@ -545,8 +545,8 @@ pattern1 = "<(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)>" cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" - "awk '{print \"<\" $1 \">\"}'" % - self._client.config.default_network_interface) + "awk '{print \"<\" $1 \">\"}'" % + self._client.config.default_network_interface) self.run( cmd, [pattern1, pexpect.EOF, pexpect.TIMEOUT], timeout=5) if self.match_id != 0: @@ -568,7 +568,7 @@ "| sed 's/[^0-9-]//g; s/^-\+//')" "\"", [pattern, pexpect.EOF, pexpect.TIMEOUT], - timeout = 5) + timeout=5) device_version = None if self.match_id == 0: @@ -662,11 +662,10 @@ # delete use of cache partition session.run('sed -i "/\/dev\/block\/%s%s%s/d" %s' % (blkorg, partition_padding_string_org, cache_part_org, rc_filename)) - session.run('sed -i "s/%s%s%s/%s%s%s/g" %s' - % (blkorg, partition_padding_string_org, data_part_org, blklava, partition_padding_string_lava, data_part_lava, rc_filename)) - session.run('sed -i "s/%s%s%s/%s%s%s/g" %s' - % (blkorg, partition_padding_string_org, sys_part_org, blklava, partition_padding_string_lava, sys_part_lava, rc_filename)) - + session.run('sed -i "s/%s%s%s/%s%s%s/g" %s' % (blkorg, partition_padding_string_org, data_part_org, blklava, + partition_padding_string_lava, data_part_lava, rc_filename)) + session.run('sed -i "s/%s%s%s/%s%s%s/g" %s' % (blkorg, partition_padding_string_org, sys_part_org, blklava, + partition_padding_string_lava, sys_part_lava, rc_filename)) def _recreate_uInitrd(session, target): @@ -740,9 +739,9 @@ script_path = '%s/%s' % ('/mnt/lava', '/system/bin/disablesuspend.sh') if not session.is_file_exist(script_path): session.run("sh -c 'export http_proxy=%s'" % - target.context.config.lava_proxy) + target.context.config.lava_proxy) session.run('wget --no-check-certificate %s -O %s' % - (target.config.git_url_disablesuspend_sh, script_path)) + (target.config.git_url_disablesuspend_sh, script_path)) session.run('chmod +x %s' % script_path) session.run('chown :2000 %s' % script_path) @@ -772,9 +771,9 @@ data_label = _android_data_label(session) session.run('umount /dev/disk/by-label/%s' % data_label, failok=True) session.run('mkfs.ext4 -q /dev/disk/by-label/%s -L %s' % - (data_label, data_label)) + (data_label, data_label)) session.run('udevadm trigger') session.run('mkdir -p /mnt/lava/data') - session.run('mount /dev/disk/by-label/%s /mnt/lava/data' % (data_label)) + session.run('mount /dev/disk/by-label/%s /mnt/lava/data' % data_label) session._client.target_extract(session, datatbz2, '/mnt/lava', timeout=600) session.run('umount /mnt/lava/data') === modified file 'lava_dispatcher/device/nexus10.py' --- lava_dispatcher/device/nexus10.py 2013-06-20 01:07:00 +0000 +++ lava_dispatcher/device/nexus10.py 2013-07-16 16:03:44 +0000 @@ -18,14 +18,17 @@ # along # with this program; if not, see . -from time import sleep from lava_dispatcher.device.target import ( Target ) +from lava_dispatcher.errors import ( + CriticalError, +) from lava_dispatcher.device.fastboot import ( FastbootTarget ) + class Nexus10Target(FastbootTarget): def __init__(self, context, config): @@ -56,7 +59,7 @@ self._booted = True proc = self._adb('shell', spawn=True) - proc.sendline("") # required to put the adb shell in a reasonable state + proc.sendline("") # required to put the adb shell in a reasonable state proc.sendline("export PS1='%s'" % self.deployment_data['TESTER_PS1']) self._runner = self._get_runner(proc) === modified file 'lava_dispatcher/device/qemu.py' --- lava_dispatcher/device/qemu.py 2013-06-06 15:10:37 +0000 +++ lava_dispatcher/device/qemu.py 2013-07-16 16:03:32 +0000 @@ -29,14 +29,14 @@ from lava_dispatcher.client.lmc_utils import ( generate_image, image_partition_mounted, - ) +) from lava_dispatcher.downloader import ( download_image, - ) +) from lava_dispatcher.utils import ( ensure_directory, extract_targz, - ) +) class QEMUTarget(Target): === modified file 'lava_dispatcher/device/sdmux.py' --- lava_dispatcher/device/sdmux.py 2013-04-02 16:16:18 +0000 +++ lava_dispatcher/device/sdmux.py 2013-07-16 16:04:07 +0000 @@ -88,7 +88,7 @@ if config.pre_connect_command: self.context.run_command(config.pre_connect_command) - def deploy_linaro(self, hwpack=None, rootfs=None): + def deploy_linaro(self, hwpack=None, rootfs=None, bootloader=None): img = generate_image(self, hwpack, rootfs, self.scratch_dir) self._customize_linux(img) self._write_image(img) === modified file 'lava_dispatcher/device/target.py' --- lava_dispatcher/device/target.py 2013-06-28 02:52:46 +0000 +++ lava_dispatcher/device/target.py 2013-07-17 09:16:46 +0000 @@ -24,8 +24,7 @@ import re from lava_dispatcher.client.lmc_utils import ( - image_partition_mounted, - ) + image_partition_mounted) import lava_dispatcher.utils as utils @@ -49,7 +48,7 @@ 'TESTER_PS1': ANDROID_TESTER_PS1, 'TESTER_PS1_PATTERN': ANDROID_TESTER_PS1, 'TESTER_PS1_INCLUDES_RC': False, - } + } ubuntu_deployment_data = { 'TESTER_PS1': "linaro-test [rc=$(echo \$?)]# ", 'TESTER_PS1_PATTERN': "linaro-test \[rc=(\d+)\]# ", @@ -86,7 +85,7 @@ """ raise NotImplementedError('power_on') - def deploy_linaro(self, hwpack, rfs): + def deploy_linaro(self, hwpack, rfs, bootloader): raise NotImplementedError('deploy_image') def deploy_android(self, boot, system, userdata): @@ -173,6 +172,7 @@ return dest def _customize_bootloader(self): + # FIXME: proc is unresolved - is this the same as the proc from runner? self.proc.expect(self.config.bootloader_prompt, timeout=300) if isinstance(self.config.boot_cmds, basestring): boot_cmds = utils.string_to_list(self.config.boot_cmds.encode('ascii')) === modified file 'lava_dispatcher/device/uefi.py' --- lava_dispatcher/device/uefi.py 2013-05-30 10:38:23 +0000 +++ lava_dispatcher/device/uefi.py 2013-07-16 16:06:08 +0000 @@ -24,6 +24,7 @@ MasterImageTarget ) + class UEFITarget(MasterImageTarget): def __init__(self, context, config): @@ -63,5 +64,4 @@ self.proc.send(self.config.interrupt_boot_command) - target_class = UEFITarget === modified file 'lava_dispatcher/device/vexpress.py' --- lava_dispatcher/device/vexpress.py 2013-04-05 21:58:28 +0000 +++ lava_dispatcher/device/vexpress.py 2013-07-17 09:16:46 +0000 @@ -27,6 +27,7 @@ from lava_dispatcher.device.master import MasterImageTarget from lava_dispatcher.errors import CriticalError + class VexpressTarget(MasterImageTarget): def __init__(self, context, config): @@ -35,9 +36,9 @@ self.test_uefi = None if (self.config.uefi_image_filename is None or - self.config.vexpress_uefi_path is None or - self.config.vexpress_uefi_backup_path is None or - self.config.vexpress_usb_mass_storage_device is None): + self.config.vexpress_uefi_path is None or + self.config.vexpress_uefi_backup_path is None or + self.config.vexpress_usb_mass_storage_device is None): raise CriticalError( "Versatile Express devices must specify all " @@ -160,7 +161,7 @@ # # --no-anchored matches the name inside any directory in the tarball. self.context.run_command('tar --no-anchored -xaf %s -C %s %s' % (tarball, tmpdir, - uefi_on_image)) + uefi_on_image)) uefi_on_image = os.path.join(tmpdir, uefi_on_image) test_uefi = os.path.join(tmpdir, 'uefi.bin') === modified file 'lava_dispatcher/downloader.py' --- lava_dispatcher/downloader.py 2013-06-11 20:27:05 +0000 +++ lava_dispatcher/downloader.py 2013-07-16 16:08:22 +0000 @@ -24,7 +24,6 @@ import logging import os import re -import shutil import subprocess import time import traceback @@ -116,13 +115,13 @@ parts = filename.split('.') suffix = parts[-1] filename = os.path.join(path, '.'.join(parts[:-1])) - return (filename, suffix) + return filename, suffix def _url_mapping(url, context): - '''allows the downloader to override a URL so that something like: + """allows the downloader to override a URL so that something like: http://blah/ becomes file://localhost/blah - ''' + """ mappings = get_config_file('urlmappings.txt') if mappings: newurl = url @@ -139,10 +138,10 @@ def download_image(url, context, imgdir=None, - delete_on_exit=True, decompress=True): - '''downloads a image that's been compressed as .bz2 or .gz and + delete_on_exit=True, decompress=True): + """downloads a image that's been compressed as .bz2 or .gz and optionally decompresses it on the file to the cache directory - ''' + """ logging.info("Downloading image: %s" % url) if not imgdir: imgdir = mkdtemp(dir=context.config.lava_image_tmpdir) @@ -173,9 +172,9 @@ def download_with_retry(context, imgdir, url, decompress=True, timeout=300): - ''' + """ download test result with a retry mechanism and 5 minute default timeout - ''' + """ logging.info("About to download %s to the host" % url) now = time.time() tries = 0 === modified file 'lava_dispatcher/job.py' --- lava_dispatcher/job.py 2013-06-08 02:59:56 +0000 +++ lava_dispatcher/job.py 2013-07-16 16:08:07 +0000 @@ -48,55 +48,55 @@ 'command': { 'optional': False, 'type': 'string', - }, + }, 'parameters': { 'optional': True, 'type': 'object', - }, + }, 'metadata': { 'optional': True, - }, }, + }, 'additionalProperties': False, - }, }, + }, 'device_type': { 'type': 'string', 'optional': True, - }, + }, 'job_name': { 'type': 'string', 'optional': True, - }, + }, 'health_check': { 'optional': True, 'default': False, - }, + }, 'target': { 'type': 'string', 'optional': True, - }, + }, 'timeout': { 'type': 'integer', 'optional': False, - }, + }, 'logging_level': { 'type': 'string', 'enum': ["CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG"], 'optional': True, - }, + }, 'tags': { 'type': 'array', 'uniqueItems': True, 'items': {'type': 'string'}, 'optional': True, - }, + }, 'priority': { 'type': 'string', 'optional': True, - }, }, - } + }, +} def validate_job_data(job_data): @@ -142,7 +142,7 @@ lava_commands = get_all_cmds() if self.job_data['actions'][-1]['command'].startswith( - "submit_results"): + "submit_results"): submit_results = self.job_data['actions'].pop(-1) else: submit_results = None @@ -161,14 +161,14 @@ job_length = len(self.job_data['actions']) job_num = 0 for cmd in self.job_data['actions']: - job_num = job_num + 1 + job_num += 1 params = cmd.get('parameters', {}) if cmd.get('command').startswith('lava_android_test'): if not params.get('timeout') and \ self.job_data.get('timeout'): params['timeout'] = self.job_data['timeout'] - logging.info("[ACTION-B] %s is started with %s" % ( - cmd['command'], params)) + logging.info("[ACTION-B] %s is started with %s" % + (cmd['command'], params)) metadata = cmd.get('metadata', {}) self.context.test_data.add_metadata(metadata) action = lava_commands[cmd['command']](self.context) @@ -196,8 +196,8 @@ status = 'pass' except TimeoutError as err: if cmd.get('command').startswith('lava_android_test'): - logging.warning("[ACTION-E] %s times out." % ( - cmd['command'])) + logging.warning("[ACTION-E] %s times out." % + (cmd['command'])) if job_num == job_length: ## not reboot the android image for ## the last test action @@ -228,8 +228,8 @@ # XXX mwhudson, 2013-01-17: I have no idea what this # code is doing. logging.warning( - "[ACTION-E] %s is finished with error (%s)." % ( - cmd['command'], err)) + "[ACTION-E] %s is finished with error (%s)." % + (cmd['command'], err)) err_msg = ("Lava failed at action %s with error:" "%s\n") % (cmd['command'], unicode(str(err), @@ -241,8 +241,8 @@ print err_msg else: logging.info( - "[ACTION-E] %s is finished successfully." % ( - cmd['command'])) + "[ACTION-E] %s is finished successfully." % + (cmd['command'])) err_msg = "" self.context.test_data.add_result( action.test_name(**params), status, err_msg) === modified file 'lava_dispatcher/lava_test_shell.py' --- lava_dispatcher/lava_test_shell.py 2013-04-08 04:57:50 +0000 +++ lava_dispatcher/lava_test_shell.py 2013-07-16 16:09:56 +0000 @@ -103,13 +103,12 @@ def _get_sw_context(build, pkgs, sw_sources): - ctx = {} - ctx['image'] = {'name': build} + ctx = {'image': {'name': build}} pkglist = [] pattern = re.compile( ("^\s*package:\s*(?P[^:]+?)\s*:" - "\s*(?P[^\s].+)\s*$"), re.M) + "\s*(?P[^\s].+)\s*$"), re.M) for line in pkgs.split('\n'): match = pattern.search(line) if match: @@ -121,9 +120,9 @@ return ctx -def _attachments_from_dir(dir): +def _attachments_from_dir(from_dir): attachments = [] - for filename, filepath in _directory_names_and_paths(dir, ignore_missing=True): + for filename, filepath in _directory_names_and_paths(from_dir, ignore_missing=True): if filename.endswith('.mimetype'): continue mime_type = _read_content(filepath + '.mimetype', ignore_missing=True).strip() @@ -136,48 +135,47 @@ return attachments -def _attributes_from_dir(dir): +def _attributes_from_dir(from_dir): attributes = {} - for filename, filepath in _directory_names_and_paths(dir, ignore_missing=True): + for filename, filepath in _directory_names_and_paths(from_dir, ignore_missing=True): if os.path.isfile(filepath): attributes[filename] = _read_content(filepath) return attributes -def _result_to_dir(test_result, dir): +def _result_to_dir(test_result, res_dir): def w(name, content): - with open(os.path.join(dir, name), 'w') as f: + with open(os.path.join(res_dir, name), 'w') as f: f.write(str(content) + '\n') for name in 'result', 'measurement', 'units', 'message', 'timestamp', 'duration': if name in test_result: w(name, test_result[name]) - - os.makedirs(os.path.join(dir, 'attachments')) + os.makedirs(os.path.join(res_dir, 'attachments')) for attachment in test_result.get('attachments', []): path = 'attachments/' + attachment['pathname'] w(path, base64.b64decode(attachment['content'])) w(path + '.mimetype', attachment['mime_type']) - os.makedirs(os.path.join(dir, 'attributes')) + os.makedirs(os.path.join(res_dir, 'attributes')) for attrname, attrvalue in test_result.get('attributes', []).items(): path = 'attributes/' + attrname w(path, attrvalue) -def _result_from_dir(dir, test_case_id=None): +def _result_from_dir(res_dir, test_case_id=None): if not test_case_id: - test_case_id = os.path.basename(dir) + test_case_id = os.path.basename(res_dir) result = { 'test_case_id': test_case_id - } + } for fname in 'result', 'measurement', 'units', 'message', 'timestamp', 'duration': - fpath = os.path.join(dir, fname) + fpath = os.path.join(res_dir, fname) if os.path.isfile(fpath): result[fname] = _read_content(fpath).strip() @@ -185,11 +183,11 @@ try: result['measurement'] = decimal.Decimal(result['measurement']) except decimal.InvalidOperation: - logging.warning("Invalid measurement for %s: %s" % (dir, result['measurement'])) + logging.warning("Invalid measurement for %s: %s" % (res_dir, result['measurement'])) del result['measurement'] - result['attachments'] = _attachments_from_dir(os.path.join(dir, 'attachments')) - result['attributes'] = _attributes_from_dir(os.path.join(dir, 'attributes')) + result['attachments'] = _attachments_from_dir(os.path.join(res_dir, 'attachments')) + result['attributes'] = _attributes_from_dir(os.path.join(res_dir, 'attributes')) return result @@ -214,6 +212,7 @@ def _get_test_results(test_run_dir, testdef, stdout): results_from_log_file = [] fixupdict = {} + pattern = None if 'parse' in testdef: if 'fixupdict' in testdef['parse']: @@ -227,6 +226,8 @@ 'UNKNOWN': 'unknown'} logging.warning("""Using a default pattern to parse the test result. This may lead to empty test result in certain cases.""") + if not pattern: + logging.debug("No pattern set") for lineno, line in enumerate(stdout.split('\n'), 1): match = pattern.match(line.strip()) if match: @@ -244,7 +245,7 @@ res['measurement'] = decimal.Decimal(res['measurement']) except decimal.InvalidOperation: logging.warning("Invalid measurement %s" % ( - res['measurement'])) + res['measurement'])) del res['measurement'] results_from_log_file.append(res) @@ -277,10 +278,8 @@ def _get_run_attachments(test_run_dir, testdef, stdout): - attachments = [] - - attachments.append(create_attachment('stdout.log', stdout)) - attachments.append(create_attachment('testdef.yaml', testdef)) + attachments = [create_attachment('stdout.log', stdout), + create_attachment('testdef.yaml', testdef)] return_code = _read_content(os.path.join(test_run_dir, 'return_code'), ignore_missing=True) if return_code: attachments.append(create_attachment('return_code', return_code)) @@ -301,12 +300,12 @@ 'os': None, 'devices': None, 'environment': None - } + } metadata = _read_content(os.path.join(test_run_dir, 'testdef_metadata')) if metadata is not '': testdef_metadata = yaml.safe_load(metadata) - + return testdef_metadata @@ -339,7 +338,7 @@ 'attachments': attachments, 'attributes': attributes, 'testdef_metadata': _get_run_testdef_metadata(test_run_dir) - } + } def _read_content(filepath, ignore_missing=False): === modified file 'lava_dispatcher/signals/__init__.py' --- lava_dispatcher/signals/__init__.py 2012-12-20 03:35:44 +0000 +++ lava_dispatcher/signals/__init__.py 2013-07-16 16:06:51 +0000 @@ -123,7 +123,6 @@ pass - class SignalDirector(object): def __init__(self, client, testdefs_by_uuid): @@ -173,4 +172,3 @@ except: logging.exception( "postprocessing test run with uuid %s failed", uuid) - === modified file 'lava_dispatcher/signals/duration.py' --- lava_dispatcher/signals/duration.py 2012-11-22 21:33:41 +0000 +++ lava_dispatcher/signals/duration.py 2013-07-16 16:07:13 +0000 @@ -2,15 +2,15 @@ import time from json_schema_validator.extensions import timedelta_extension - from lava_dispatcher.signals import SignalHandler + class AddDuration(SignalHandler): def start_testcase(self, test_case_id): return { 'starttime': time.time() - } + } def end_testcase(self, test_case_id, data): data['endtime'] = time.time() === modified file 'lava_dispatcher/signals/shellhooks.py' --- lava_dispatcher/signals/shellhooks.py 2012-12-19 04:27:40 +0000 +++ lava_dispatcher/signals/shellhooks.py 2013-07-16 16:07:05 +0000 @@ -13,7 +13,11 @@ class ShellHooks(SignalHandler): - def __init__(self, testdef_obj, handlers={}, device_config_vars={}): + def __init__(self, testdef_obj, handlers=None, device_config_vars=None): + if not device_config_vars: + device_config_vars = {} + if not handlers: + handlers = {} SignalHandler.__init__(self, testdef_obj) self.result_dir = mkdtemp() self.handlers = handlers @@ -32,7 +36,9 @@ else: self.our_env[env_var] = config_value - def _invoke_hook(self, name, working_dir, args=[]): + def _invoke_hook(self, name, working_dir, args=None): + if not args: + args = [] script_name = self.handlers.get(name) if not script_name: return === modified file 'lava_dispatcher/tarballcache.py' --- lava_dispatcher/tarballcache.py 2012-10-18 15:10:06 +0000 +++ lava_dispatcher/tarballcache.py 2013-07-16 16:08:54 +0000 @@ -28,11 +28,11 @@ from lava_dispatcher.downloader import ( download_image, - ) +) def get_tarballs(context, image_url, scratch_dir, generator): - ''' + """ Tries to return a cached copy array of (boot_tgz, root_tgz). If no cache exists for this image_url, then it: * places a global lock for the image_url to prevent other dispatchers @@ -42,7 +42,7 @@ generator - a callback to a function that can generate the tarballs given a local copy .img file - ''' + """ logging.info('try to find cached tarballs for %s' % image_url) with _cache_locked(image_url, context.config.lava_cachedir) as cachedir: boot_tgz = os.path.join(cachedir, 'boot.tgz') @@ -54,7 +54,7 @@ if data is not None: logging.info('returning cached copies') (boot_tgz, root_tgz) = _link(boot_tgz, root_tgz, scratch_dir) - return (boot_tgz, root_tgz, data) + return boot_tgz, root_tgz, data else: logging.info('no cache found for %s' % image_url) @@ -65,7 +65,7 @@ f.write(data) _link(boot_tgz, root_tgz, cachedir) os.unlink(image) - return (boot_tgz, root_tgz, data) + return boot_tgz, root_tgz, data def _link(boot_tgz, root_tgz, destdir): @@ -73,7 +73,7 @@ droot_tgz = os.path.join(destdir, 'root.tgz') os.link(boot_tgz, dboot_tgz) os.link(root_tgz, droot_tgz) - return (dboot_tgz, droot_tgz) + return dboot_tgz, droot_tgz def _clear_cache(boot_tgz, root_tgz, data): === modified file 'lava_dispatcher/test_data.py' --- lava_dispatcher/test_data.py 2013-03-25 19:41:18 +0000 +++ lava_dispatcher/test_data.py 2013-07-16 16:08:33 +0000 @@ -30,11 +30,12 @@ 'content': base64.b64encode(content), } + class LavaTestData(object): def __init__(self, test_id='lava'): self.job_status = 'pass' self.metadata = {} - self._test_run = { 'test_results':[], 'attachments':[], 'tags':[] } + self._test_run = {'test_results': [], 'attachments': [], 'tags': []} self._test_run['test_id'] = test_id self._assign_date() self._assign_uuid() @@ -53,7 +54,7 @@ 'test_case_id': test_case_id, 'result': result, 'message': message - } + } self._test_run['test_results'].append(result_data) def add_attachments(self, attachments): === modified file 'lava_dispatcher/tests/__init__.py' --- lava_dispatcher/tests/__init__.py 2012-10-08 22:19:31 +0000 +++ lava_dispatcher/tests/__init__.py 2013-07-16 16:09:31 +0000 @@ -1,5 +1,6 @@ import unittest + def test_suite(): module_names = [ 'lava_dispatcher.tests.test_config', === modified file 'lava_dispatcher/tests/helper.py' --- lava_dispatcher/tests/helper.py 2012-10-20 19:45:13 +0000 +++ lava_dispatcher/tests/helper.py 2013-07-16 16:09:42 +0000 @@ -18,11 +18,12 @@ # along with this program; if not, see . import os -from lava_dispatcher.config import get_config, get_device_config +from lava_dispatcher.config import get_device_config __tmp_dir = os.getenv("TMPDIR") or '/tmp' __tmp_config_dir = os.path.join(__tmp_dir, 'lava-dispatcher-config') + def create_config(name, data): filename = os.path.join(__tmp_config_dir, name) if not os.path.exists(os.path.dirname(filename)): @@ -31,18 +32,22 @@ for key in data.keys(): f.write("%s = %s\n" % (key, data[key])) + def create_device_config(name, data): create_config("devices/%s.conf" % name, data) return get_device_config(name, __tmp_config_dir) + def setup_config_dir(): os.mkdir(__tmp_config_dir) + def cleanup_config_dir(): os.system('rm -rf %s' % __tmp_config_dir) from unittest import TestCase + class LavaDispatcherTestCase(TestCase): def setUp(self): === modified file 'lava_dispatcher/tests/test_config.py' --- lava_dispatcher/tests/test_config.py 2012-10-18 17:37:08 +0000 +++ lava_dispatcher/tests/test_config.py 2013-07-16 16:09:24 +0000 @@ -21,12 +21,12 @@ from lava_dispatcher.config import get_config, get_device_config from lava_dispatcher.utils import string_to_list -from lava_dispatcher.client.base import LavaClient from lava_dispatcher.tests.helper import * test_config_dir = os.path.join(os.path.dirname(__file__), 'test-config') + class TestConfigData(TestCase): def setUp(self): @@ -41,11 +41,11 @@ "mmc init", "mmc part 0", "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage; fatload mmc " - "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'", + "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'", "setenv bootargs ' console=tty0 console=ttyO2,115200n8 " - "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc " - "nocompcache vram=12M omapfb.debug=y " - "omapfb.mode=dvi:1280x720MR-16@60'", + "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc " + "nocompcache vram=12M omapfb.debug=y " + "omapfb.mode=dvi:1280x720MR-16@60'", "boot"] uboot_cmds = beagle01_config.boot_cmds self.assertEquals(expected, string_to_list(uboot_cmds)) === modified file 'lava_dispatcher/tests/test_device_version.py' --- lava_dispatcher/tests/test_device_version.py 2012-10-20 19:48:31 +0000 +++ lava_dispatcher/tests/test_device_version.py 2013-07-16 16:09:15 +0000 @@ -17,7 +17,6 @@ # You should have received a copy of the GNU General Public License # along with this program; if not, see . -from unittest import TestCase import re from lava_dispatcher.tests.helper import LavaDispatcherTestCase, create_device_config, create_config, __tmp_config_dir @@ -28,18 +27,22 @@ from lava_dispatcher.config import get_config def _create_fastmodel_target(): - config = create_device_config('fastmodel01', { 'device_type': 'fastmodel', 'simulator_binary': '/path/to/fastmodel', 'license_server': 'foo.local' }) + config = create_device_config('fastmodel01', {'device_type': 'fastmodel', + 'simulator_binary': '/path/to/fastmodel', + 'license_server': 'foo.local'}) target = FastModelTarget(None, config) return target + def _create_qemu_target(): create_config('lava-dispatcher.conf', {'default_qemu_binary': 'qemu-system-arm'}) - device_config = create_device_config('qemu01', { 'device_type': 'qemu' }) + device_config = create_device_config('qemu01', {'device_type': 'qemu'}) dispatcher_config = get_config(__tmp_config_dir) - context = LavaContext('qemu01', dispatcher_config, None, None) + context = LavaContext('qemu01', dispatcher_config, None, None, None) return QEMUTarget(context, device_config) + class TestDeviceVersion(LavaDispatcherTestCase): def test_base(self): @@ -57,7 +60,7 @@ "Copyright 2000-2012 ARM Limited.", "All Rights Reserved.", "Top component name: RTSM_VE_Cortex_A15x1_A7x1" - ]) + ]) target = _create_fastmodel_target() version = target._parse_fastmodel_version(banner) self.assertEqual('7.1.36', version) @@ -66,4 +69,3 @@ client = _create_fastmodel_target() version = client._parse_fastmodel_version('random string') self.assertEqual('unknown', version) - === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2013-04-02 16:16:18 +0000 +++ lava_dispatcher/utils.py 2013-07-16 16:09:56 +0000 @@ -39,9 +39,9 @@ def link_or_copy_file(src, dest): try: - dir = os.path.dirname(dest) - if not os.path.exists(dir): - os.makedirs(dir) + dirname = os.path.dirname(dest) + if not os.path.exists(dirname): + os.makedirs(dirname) os.link(src, dest) except OSError, err: if err.errno == errno.EXDEV: @@ -53,9 +53,9 @@ def copy_file(src, dest): - dir = os.path.dirname(dest) + dirname = os.path.dirname(dest) if not os.path.exists(dir): - os.makedirs(dir) + os.makedirs(dirname) shutil.copy(src, dest) @@ -124,7 +124,7 @@ def url_to_cache(url, cachedir): url_parts = urlparse.urlsplit(url) path = os.path.join(cachedir, url_parts.netloc, - url_parts.path.lstrip(os.sep)) + url_parts.path.lstrip(os.sep)) return path === modified file 'setup.py' --- setup.py 2013-01-10 03:12:48 +0000 +++ setup.py 2013-07-17 09:16:24 +0000 @@ -24,7 +24,7 @@ shell-hooks = lava_dispatcher.signals.shellhooks:ShellHooks """, packages=find_packages(), - package_data= { + package_data={ 'lava_dispatcher': [ 'default-config/lava-dispatcher/lava-dispatcher.conf', 'default-config/lava-dispatcher/lava-dispatcher.conf', @@ -32,8 +32,8 @@ 'default-config/lava-dispatcher/device-types/*.conf', 'default-config/lava-dispatcher/devices/*.conf', 'device/sdmux.sh', - ], - }, + ], + }, data_files=[ ('lava_test_shell', [ 'lava_test_shell/lava-test-case', @@ -42,7 +42,7 @@ 'lava_test_shell/lava-test-runner-android', 'lava_test_shell/lava-test-runner-ubuntu', 'lava_test_shell/lava-test-shell', - ]) + ]) ], install_requires=[ "json-schema-validator >= 2.3", @@ -57,7 +57,7 @@ setup_requires=[ 'versiontools >= 1.8', ], - scripts = [ + scripts=[ 'lava-dispatch' ], )