From patchwork Thu Mar 22 23:39:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Larson X-Patchwork-Id: 7426 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id CBF1423DEA for ; Thu, 22 Mar 2012 23:39:13 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 6D0EAA18411 for ; Thu, 22 Mar 2012 23:39:13 +0000 (UTC) Received: by iage36 with SMTP id e36so5038064iag.11 for ; Thu, 22 Mar 2012 16:39:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=yO7qUbnVP1GEOIJeL37mVZfmNEhgUmmMBOwrq9YZrAE=; b=QfOAZYXBCqsti/48OMePb1YrY43XYWxfAFzpMwTdubpWPahph10o3cdke5WmnMDFWN mmW09NlEan8eiBcVl3Dj4NVvlAoKxcngHn74397iEPmFDZxquZGb52cu80+TZAPN2xiy JTyg8ANk7Bh4zoYhI5LKqmLQMMLX014q6MlcHsijdkVMRIDzCXYLsvyGlD6XEW0KRcKW RhSRnVEhUoZn6cl48XwWgBjn+kgip/0KyRz3xyXlEmc2E5dn857/8GrOODugLpfcW7+N KQxeTVFhT5dDyqOD8cISBGc4iwW6tbopUzNAtIwb5DGthwpw/+LTCKihnw7eONfp91Lz GiOw== Received: by 10.50.153.132 with SMTP id vg4mr234580igb.2.1332459552864; Thu, 22 Mar 2012 16:39:12 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.203.79 with SMTP id fh15csp9726ibb; Thu, 22 Mar 2012 16:39:12 -0700 (PDT) Received: by 10.216.132.226 with SMTP id o76mr5633376wei.93.1332459551692; Thu, 22 Mar 2012 16:39:11 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id l5si4038264wiz.2.2012.03.22.16.39.11 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 22 Mar 2012 16:39:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1SArb5-0005Xc-5j for ; Thu, 22 Mar 2012 23:39:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 1BB07E0118 for ; Thu, 22 Mar 2012 23:39:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 262 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 262: * Convert the dispatcher to LAVA commnand. It can now be called from the shell Message-Id: <20120322233911.6864.35054.launchpad@ackee.canonical.com> Date: Thu, 22 Mar 2012 23:39:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14981"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 32a9183ad60776d1ab5fe354daa35e1eca7cbe29 X-Gm-Message-State: ALoCoQmHW4/HDFNM9pHK8IFotx0yclbTeDkYcWdNzFJ/+y6EPjvJSjP0hfp2uSQDZ8SBUijhrtGr Merge authors: Zygmunt Krynicki (zkrynicki) Related merge proposals: https://code.launchpad.net/~zkrynicki/lava-dispatcher/convert-to-lava-tool/+merge/98909 proposed by: Zygmunt Krynicki (zkrynicki) review: Approve - Paul Larson (pwlars) ------------------------------------------------------------ revno: 262 [merge] committer: Paul Larson branch nick: lava-dispatcher timestamp: Thu 2012-03-22 18:37:01 -0500 message: * Convert the dispatcher to LAVA commnand. It can now be called from the shell by running ``lava dispatch``. The old command line interface ``lava-dispatch`` is now deprecated and will be removed in the 0.8 release in three months. added: lava/ lava/__init__.py lava/dispatcher/ lava/dispatcher/__init__.py lava/dispatcher/commands.py modified: doc/changes.rst lava_dispatcher/actions/__init__.py lava_dispatcher/utils.py setup.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'doc/changes.rst' --- doc/changes.rst 2012-03-19 22:21:41 +0000 +++ doc/changes.rst 2012-03-20 09:27:01 +0000 @@ -8,6 +8,11 @@ * Omit the commands we send to the board from the log (as this output is invariably echoed back and so was ending up in the output twice) +* Convert the dispatcher to LAVA commnand. It can now be called from the shell + by running ``lava dispatch``. The old command line interface + ``lava-dispatch`` is now deprecated and will be removed in the 0.8 release in + three months. + .. _version_0_5_9: Version 0.5.9 === added directory 'lava' === added file 'lava/__init__.py' --- lava/__init__.py 1970-01-01 00:00:00 +0000 +++ lava/__init__.py 2012-03-19 13:29:52 +0000 @@ -0,0 +1,3 @@ +__import__('pkg_resources').declare_namespace(__name__) +# DO NOT ADD ANYTHING TO THIS FILE! +# IT MUST STAY AS IS (empty apart from the two lines above) === added directory 'lava/dispatcher' === added file 'lava/dispatcher/__init__.py' === added file 'lava/dispatcher/commands.py' --- lava/dispatcher/commands.py 1970-01-01 00:00:00 +0000 +++ lava/dispatcher/commands.py 2012-03-19 13:29:52 +0000 @@ -0,0 +1,89 @@ +import logging +import os +import sys + +from json_schema_validator.errors import ValidationError +from lava.tool.command import Command + +from lava_dispatcher.config import get_config +from lava_dispatcher.job import LavaTestJob, validate_job_data + + +class dispatch(Command): + """ + Run test scenarios on virtual and physical hardware + """ + + @classmethod + def register_arguments(self, parser): + # When we're working inside a virtual environment use venv-relative + # configuration directory. This works well with lava-deployment-tool + # and the directory layout it currently provides but will need to be + # changed for codeline support. + if "VIRTUAL_ENV" in os.environ: + default_config_dir = os.path.join( + os.environ["VIRTUAL_ENV"], "etc", "lava-dispatcher") + else: + default_config_dir = None + + parser.add_argument( + "--oob-fd", + default=None, + type=int, + help="Used internally by LAVA scheduler.") + parser.add_argument( + "--config-dir", + default=default_config_dir, + help="Configuration directory override (currently %(default)s") + parser.add_argument( + "--validate", action='store_true', + help="Just validate the job file, do not execute any steps.") + parser.add_argument( + "--job-id", action='store', default=None, + help=("Set the scheduler job identifier. " + "This alters process name for easier debugging")) + parser.add_argument( + "job-file", + metavar="JOB", + help="Test scenario file") + + def invoke(self): + if self.args.oob_fd: + oob_file = os.fdopen(self.args.oob_fd, 'w') + else: + oob_file = sys.stderr + + # config the python logging + # FIXME: move to lava-tool + FORMAT = '%(asctime)s %(levelname)s: %(message)s' + DATEFMT= '%Y-%m-%d %I:%M:%S %p' + logging.basicConfig(format=FORMAT,datefmt=DATEFMT) + config = get_config("lava-dispatcher", self.args.config_dir) + logging_level = config.get("LOGGING_LEVEL") + logging.root.setLevel(int(logging_level)) + + # Set process id if job-id was passed to dispatcher + if self.args.job_id: + try: + from setproctitle import getproctitle, setproctitle + except ImportError: + logging.warning( + ("Unable to set import 'setproctitle', " + "process name cannot be changed")) + else: + setproctitle("%s [job: %s]" % ( + getproctitle(), self.args.job_id)) + + # Load the scenario file + with open(args[0]) as stream: + jobdata = stream.read() + job = LavaTestJob(jobdata, oob_file, config) + + #FIXME Return status + if self.args.validate: + try: + validate_job_data(job.job_data) + except ValidationError as e: + print e + else: + job.run() === modified file 'lava_dispatcher/actions/__init__.py' --- lava_dispatcher/actions/__init__.py 2012-03-11 22:12:52 +0000 +++ lava_dispatcher/actions/__init__.py 2012-03-19 18:25:52 +0000 @@ -45,6 +45,7 @@ class BaseAction(object): + def __init__(self, context): self.context = context @@ -81,12 +82,13 @@ cmds[cls.command_name] = cls return cmds + def get_all_cmds(): import pkg_resources cmds = {} cmd_path = os.path.dirname(os.path.realpath(__file__)) - for f in glob(os.path.join(cmd_path,"*.py")): - module = imp.load_source("module", os.path.join(cmd_path,f)) + for f in glob(os.path.join(cmd_path, "*.py")): + module = imp.load_source("module", os.path.join(cmd_path, f)) cmds.update(_find_commands(module)) for ep in pkg_resources.iter_entry_points(group="lava_dispatcher.actions"): plugin = ep.load() === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2012-03-19 06:52:06 +0000 +++ lava_dispatcher/utils.py 2012-03-19 18:39:24 +0000 @@ -48,7 +48,8 @@ raise RuntimeError("Could not retrieve %s" % url) return filename - +# XXX: duplication, we have similar code in lava-test, we need to move that to +# lava.utils -> namespace as standalone package def download_with_cache(url, path="", cachedir=""): cache_loc = url_to_cache(url, cachedir) if os.path.exists(cache_loc): @@ -132,6 +133,7 @@ return super(logging_spawn, self).expect(*args, **kw) +# XXX Duplication: we should reuse lava-test TestArtifacts def generate_bundle_file_name(test_name): return ("{test_id}.{time.tm_year:04}-{time.tm_mon:02}-{time.tm_mday:02}T" "{time.tm_hour:02}:{time.tm_min:02}:{time.tm_sec:02}Z").format( === modified file 'setup.py' --- setup.py 2012-01-09 03:08:37 +0000 +++ setup.py 2012-03-22 18:22:14 +0000 @@ -10,6 +10,11 @@ description="Part of the LAVA framework for dispatching test jobs", author='Linaro Validation Team', author_email='linaro-dev@lists.linaro.org', + namespace_packages=['lava'], + entry_points=""" + [lava.commands] + dispatch = lava.dispatcher.commands:dispatch + """, packages=find_packages(), package_data= { 'lava_dispatcher': [ @@ -21,9 +26,10 @@ ], }, install_requires=[ + "json-schema-validator", + "lava-tool >= 0.4", + "lava-utils-interface", "pexpect >= 2.3", - "lava-tool", - "json-schema-validator", ], setup_requires=[ 'versiontools >= 1.8',