From patchwork Wed Dec 5 10:01:10 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 13364 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 1B0B923E0B for ; Wed, 5 Dec 2012 10:01:15 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 73454A186B0 for ; Wed, 5 Dec 2012 10:01:14 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id c10so7524851ieb.11 for ; Wed, 05 Dec 2012 02:01:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=QzJuaTaG6IfFzOCWeCsCjxVylx/OKcG2LeGlPTmpHPU=; b=GNiicM0cmbMco2TClRjJjiLhSf1LImzsGQr7zJTvE2lZulzbwYAs03HEsbf00ncnGN T4W8sSCgTwnFITkRlgb67JUKfGCOLbYgZFnWtIFb6DbTIELlyFbY/oFbQ+Zh3/qpgMVo kDwnQj8FahEGgHqKOs+kahtLZFdaDD6rHnIpABtSR6YoJoZocCcivjLXTpCI/9dh+Wdy Hi5aPRAUiyKrLnUgUoDUP28SMLMbQON76C679KPUEHcK/e+5FttdF7uSdNvVV4K+xsk1 mWiW+cpuSYDwtXdBBOTBaTCSiVj7kdpEeDaNv8HIwx2pFTBDnfrMnnBC1gIS6LkCgcZA grag== Received: by 10.43.46.2 with SMTP id um2mr13761185icb.18.1354701673625; Wed, 05 Dec 2012 02:01:13 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp327961igt; Wed, 5 Dec 2012 02:01:12 -0800 (PST) Received: by 10.180.19.73 with SMTP id c9mr2112002wie.8.1354701672085; Wed, 05 Dec 2012 02:01:12 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id y45si2531791wey.157.2012.12.05.02.01.11 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 05 Dec 2012 02:01:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TgBmx-0000ij-0Z for ; Wed, 05 Dec 2012 10:01:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id D65D2E0288 for ; Wed, 5 Dec 2012 10:01:10 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 486 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 486: allow signal handlers to be written in shell and part of the testdef repo Message-Id: <20121205100110.7593.31154.launchpad@ackee.canonical.com> Date: Wed, 05 Dec 2012 10:01:10 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16335"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 1f01634ba3c46f5fd7b7e3e58e4587d7b223b0bb X-Gm-Message-State: ALoCoQnqlukLfkBdMgsI5+EZVR23d8N4tgG2JF0/BPQV3oa04lnNM6rnKRaSGKjWLhX5DxwPaUuB Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-dispatcher/shell-hooks/+merge/135810 proposed by: Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 486 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Wed 2012-12-05 22:59:53 +1300 message: allow signal handlers to be written in shell and part of the testdef repo added: lava_dispatcher/signals/shellhooks.py modified: doc/external_measurement.rst lava_dispatcher/lava_test_shell.py lava_dispatcher/signals/__init__.py setup.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'doc/external_measurement.rst' --- doc/external_measurement.rst 2012-11-22 21:58:51 +0000 +++ doc/external_measurement.rst 2012-12-05 00:24:13 +0000 @@ -68,6 +68,9 @@ should not be possible to crash the whole dispatcher with a typo in one of them. +There are other methods you might want to override in some situations +-- see the source for more. + Here is a very simple complete handler:: import datetime @@ -106,5 +109,107 @@ .. _`entry point`: http://packages.python.org/distribute/pkg_resources.html#entry-points -We will soon provide a way to bundle the signal handler along with the -test definition. +Providing handlers as shell scripts +----------------------------------- + +Using the 'shell-hooks' handler that is distributed with the +dispatcher it is possible to write handlers as scripts in the same VCS +repository as the test definition itself. + +The simplest usage looks like this:: + + handler: + handler-name: shell-hooks + params: + handlers: + start_testcase: start-testcase.sh + end_testcase: end-testcase.sh + postprocess_test_result: postprocess-test-result.sh + +The scripts named in ``handlers`` are invoked with a test-case +specific directory as the current working directory so they can store +and access data in local paths. The scripts named by +``start_testcase`` and ``end_testcase`` are invoked with no arguments +but ``postprocess_test_result`` is invoked with a single argument: a +directory which contains the on-disk representation of the test result +as produced on the device (this on-disk representation is not yet +fully documented). If a hook produces output, it will be attached to +the test result. + +As many interesting hooks need to have information about the device +being tested, there is a facility for putting values from the device +config into the environment of the hooks. For example, the following +test definition sets the environment variable ``$DEVICE_TYPE`` to the +value of the ``device_type`` key:: + + handler: + handler-name: shell-hooks + params: + device_config_vars: + DEVICE_TYPE: device_type + handlers: + ... + +For a slightly silly example of a shell hook, let's try to mark any +test that takes more than 10 seconds (as viewed from the host) as +failed, even if they report success on the device, and also attach +some meta data about the device to each test result. + +The start hook (``start-hook.sh`` in the repository) just records the +current unix timestamp in a file (we can just use the cwd as a scratch +storage area):: + + #!/bin/sh + date +%s > start-time + +The end hook (``end-hook.sh``) just records the end time:: + + #!/bin/sh + date +%s > end-time + +The postprocess hook (``post-process-result-hook.sh``) reads the times +recorded by the above hooks, overwrites the result if necessary and +creates an attachment containing the device type:: + + #!/bin/sh + start_time=`cat start-time` + end_time=`cat end-time` + if [ $((end_time - start_time)) -gt 10 ]; then + echo fail > $1/result + fi + echo $DEVICE_TYPE > $1/attachments/device-type.txt + +A test definition that glues this all together would be:: + + metadata: + format: Lava-Test Test Definition 1.0 + name: shell-hook-example + + run: + steps: + - lava-test-case pass-test --shell sleep 5 + - lava-test-case fail-test --shell sleep 15 + + handler: + handler-name: shell-hooks + params: + device_config_vars: + DEVICE_TYPE: device_type + handlers: + start_testcase: start-hook.sh + end_testcase: end-hook.sh + postprocess_test_result: post-process-result-hook.sh + +A repository with all the above piece is on Launchpad in the branch +`lp:~linaro-validation/+junk/shell-hook-example`_ so an action for +your job file might look like:: + + { + "command": "lava_test_shell", + "parameters": { + "testdef_repos": [{"bzr-repo": "lp:~linaro-validation/+junk/shell-hook-example"}], + "timeout": 1800 + } + }, + +.. _`lp:~linaro-validation/+junk/shell-hook-example`: http://bazaar.launchpad.net/~linaro-validation/+junk/shell-hook-example/files === modified file 'lava_dispatcher/lava_test_shell.py' --- lava_dispatcher/lava_test_shell.py 2012-12-04 21:47:49 +0000 +++ lava_dispatcher/lava_test_shell.py 2012-12-04 22:20:34 +0000 @@ -25,6 +25,7 @@ by a lava-test-shell run. """ +import base64 import datetime import decimal import mimetypes @@ -143,6 +144,31 @@ return attributes +def _result_to_dir(test_result, dir): + + def w(name, content): + with open(os.path.join(dir, name), 'w') as f: + f.write(str(content) + '\n') + + for name in 'result', 'measurement', 'units', 'message', 'timestamp', 'duration': + if name in test_result: + w(name, test_result[name]) + + + os.makedirs(os.path.join(dir, 'attachments')) + + for attachment in test_result.get('attachments', []): + path = 'attachments/' + attachment['pathname'] + w(path, base64.b64decode(attachment['content'])) + w(path + '.mimetype', attachment['mime_type']) + + os.makedirs(os.path.join(dir, 'attributes')) + + for attrname, attrvalue in test_result.get('attributes', []).items(): + path = 'attributes/' + attrname + w(path, attrvalue) + + def _result_from_dir(dir): result = { 'test_case_id': os.path.basename(dir), === modified file 'lava_dispatcher/signals/__init__.py' --- lava_dispatcher/signals/__init__.py 2012-11-22 21:33:41 +0000 +++ lava_dispatcher/signals/__init__.py 2012-11-23 00:39:23 +0000 @@ -1,5 +1,3 @@ -#!/usr/bin/python - # Copyright (C) 2012 Linaro Limited # # Author: Andy Doan === added file 'lava_dispatcher/signals/shellhooks.py' --- lava_dispatcher/signals/shellhooks.py 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/signals/shellhooks.py 2012-12-05 00:24:13 +0000 @@ -0,0 +1,87 @@ +from ConfigParser import NoOptionError +import logging +import shutil +import subprocess +import os +import tempfile + +from lava_dispatcher.lava_test_shell import ( + _read_content, + _result_to_dir, + _result_from_dir) +from lava_dispatcher.signals import SignalHandler +from lava_dispatcher.test_data import create_attachment +from lava_dispatcher.utils import mkdtemp + +class ShellHooks(SignalHandler): + + def __init__(self, testdef_obj, handlers={}, device_config_vars={}): + SignalHandler.__init__(self, testdef_obj) + self.result_dir = mkdtemp() + self.handlers = handlers + self.scratch_dir = mkdtemp() + self.code_dir = os.path.join(self.scratch_dir, 'code') + shutil.copytree(testdef_obj.repo, self.code_dir) + device_config = testdef_obj.context.client.target_device.config + self.our_env = os.environ.copy() + for env_var, config_var in device_config_vars.iteritems(): + try: + config_value = device_config.cp.get('__main__', config_var) + except NoOptionError: + logging.warning( + "No value found for device config %s; leaving %s unset " + "in environment", config_var, env_var) + else: + self.our_env[env_var] = config_value + + def _invoke_hook(self, name, working_dir, args=[]): + script_name = self.handlers.get(name) + if not script_name: + return + script = os.path.join(self.code_dir, script_name) + if not os.path.exists(script): + logging.warning("handler script %s not found", script_name) + return + (fd, path) = tempfile.mkstemp(dir=self.code_dir) + status = subprocess.call( + [script] + args, cwd=working_dir, env=self.our_env, + stdout=fd, stderr=subprocess.STDOUT) + if status != 0: + logging.warning( + "%s handler script exited with code %s", name, status) + return path + + def start_testcase(self, test_case_id): + case_dir = os.path.join(self.result_dir, test_case_id) + os.mkdir(case_dir) + case_data = {'case_dir': case_dir} + case_data['start_testcase_output'] = self._invoke_hook( + 'start_testcase', case_dir) + return case_data + + def end_testcase(self, test_case_id, case_data): + case_data['end_testcase_output'] = self._invoke_hook( + 'end_testcase', case_data['case_dir']) + + def postprocess_test_result(self, test_result, case_data): + test_case_id = test_result['test_case_id'] + scratch_dir = tempfile.mkdtemp() + try: + result_dir = os.path.join(scratch_dir, test_case_id) + os.mkdir(result_dir) + _result_to_dir(test_result, result_dir) + case_data['postprocess_test_result_output'] = self._invoke_hook( + 'postprocess_test_result', case_data['case_dir'], [result_dir]) + test_result.clear() + test_result.update(_result_from_dir(result_dir)) + finally: + shutil.rmtree(scratch_dir) + for key in 'start_testcase_output', 'end_testcase_output', \ + 'postprocess_test_result_output': + path = case_data.get(key) + if path is None: + continue + content = _read_content(path, ignore_missing=True) + if content: + test_result['attachments'].append( + create_attachment(key + '.txt', _read_content(path))) === modified file 'setup.py' --- setup.py 2012-11-30 19:14:23 +0000 +++ setup.py 2012-12-03 20:54:59 +0000 @@ -17,8 +17,10 @@ connect = lava.dispatcher.commands:connect devices = lava.dispatcher.commands:devices power-cycle = lava.dispatcher.commands:power_cycle + [lava.signal_handlers] add-duration = lava_dispatcher.signals.duration:AddDuration + shell-hooks = lava_dispatcher.signals.shellhooks:ShellHooks """, packages=find_packages(), package_data= {