From patchwork Mon Nov 19 20:03:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 12952 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 12CAC23FC6 for ; Mon, 19 Nov 2012 20:03:15 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 81264A1875D for ; Mon, 19 Nov 2012 20:03:14 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id e10so7089199iej.11 for ; Mon, 19 Nov 2012 12:03:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=kbZzISZwsyS88muiIfABC9yiid/JJs8H4/5kJGVRwcg=; b=iBdizdanSrqx6WkxOSdWcBK2sEPTWhKwbBgnPVil7bwx7guzkY/xkl/Q1fnbJGAWk7 /IwnIoU4c/VojvM3VsItV5+U65woJHDJJAG42Nlj3Cls9Jr+44V+/pewhj2SWGgKlBCw GSwXh84COVzQWTOP1H0d0C8r4yZgV1+SX8vD6jwRV7hBro19Jlm59cCmQcebFIEoCKKU SxGBDA/axVse0BApJrm/uHBNVbyhV1Sjvs8V9hlBEqfn4xjCTRBs+MBAmQ472n9TYlFo fPoeEgMBkZAl7MsZTka3XULc8bIYWNYwYzXdw9EjfC+/k/jLMH4dvXvq8l4KhsvsVmT/ Q98w== Received: by 10.50.161.169 with SMTP id xt9mr7564631igb.62.1353355393935; Mon, 19 Nov 2012 12:03:13 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp216249igt; Mon, 19 Nov 2012 12:03:13 -0800 (PST) Received: by 10.180.101.231 with SMTP id fj7mr10799755wib.4.1353355392412; Mon, 19 Nov 2012 12:03:12 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id n17si5738347wen.119.2012.11.19.12.03.11 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 19 Nov 2012 12:03:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TaXYl-0007lN-GX for ; Mon, 19 Nov 2012 20:03:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 6A059E027B for ; Mon, 19 Nov 2012 20:03:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 456 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 456: add support for attaching files to test results from lava-test-shell tests Message-Id: <20121119200311.12834.2026.launchpad@ackee.canonical.com> Date: Mon, 19 Nov 2012 20:03:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16286"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 2c7b0d4f121f2eb2bd71ce33edf2cb5bb590d791 X-Gm-Message-State: ALoCoQmzm3djwIZH9Tpgqe/RHoR2XkYOPrHuQDmfAOSJnuMApg9sIJCQL4PNPQ/WTiFQIjR6MWsP Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-dispatcher/lava-test-shell-attach/+merge/134597 proposed by: Michael Hudson-Doyle (mwhudson) https://code.launchpad.net/~mwhudson/lava-dispatcher/make-1.5-format-bundles/+merge/133598 proposed by: Michael Hudson-Doyle (mwhudson) review: Approve - Zygmunt Krynicki (zkrynicki) ------------------------------------------------------------ revno: 456 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Tue 2012-11-20 09:02:08 +1300 message: add support for attaching files to test results from lava-test-shell tests added: lava_test_shell/lava-test-case-attach modified: lava_dispatcher/actions/launch_control.py lava_dispatcher/actions/lava_test_shell.py lava_dispatcher/lava_test_shell.py lava_test_shell/lava-test-shell --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/launch_control.py' --- lava_dispatcher/actions/launch_control.py 2012-11-07 22:13:24 +0000 +++ lava_dispatcher/actions/launch_control.py 2012-11-08 23:35:16 +0000 @@ -207,7 +207,7 @@ if not all_bundles: main_bundle = { "test_runs": [], - "format": "Dashboard Bundle Format 1.3" + "format": "Dashboard Bundle Format 1.5" } else: main_bundle = all_bundles.pop(0) === modified file 'lava_dispatcher/actions/lava_test_shell.py' --- lava_dispatcher/actions/lava_test_shell.py 2012-11-08 14:08:30 +0000 +++ lava_dispatcher/actions/lava_test_shell.py 2012-11-19 19:55:35 +0000 @@ -20,6 +20,68 @@ # along # with this program; if not, see . +# LAVA Test Shell implementation details +# ====================================== +# +# The idea of lava-test-shell is a YAML test definition is "compiled" into a +# job that is run when the device under test boots and then the output of this +# job is retrieved and analyzed and turned into a bundle of results. +# +# In practice, this means a hierarchy of directories and files is created +# during test installation, a sub-hierarchy is created during execution to +# hold the results and these latter sub-hierarchy whole lot is poked at on the +# host during analysis. +# +# On Ubuntu and OpenEmbedded, the hierarchy is rooted at /lava. / is mounted +# read-only on Android, so there we root the hierarchy at /data/lava. I'll +# assume Ubuntu paths from here for simplicity. +# +# The directory tree that is created during installation looks like this: +# +# /lava/ +# bin/ This directory is put on the path when the +# test code is running -- these binaries can +# be viewed as a sort of device-side "API" +# for test authors. +# lava-test-runner The job that runs the tests on boot. +# lava-test-shell A helper to run a test suite. +# lava-test-case-attach A helper to attach a file to a test result. +# tests/ +# ${IDX}_${TEST_ID}/ One directory per test to be executed. +# testdef.yml The test definition. +# install.sh The install steps. +# run.sh The run steps. +# [repos] The test definition can specify bzr or git +# repositories to clone into this directory. +# +# In addition, a file /etc/lava-test-runner.conf is created containing the +# names of the directories in /lava/tests/ to execute. +# +# During execution, the following files are created: +# +# /lava/ +# results/ +# cpuinfo.txt Hardware info. +# meminfo.txt Ditto. +# build.txt Software info. +# pkgs.txt Ditto +# ${IDX}_${TEST_ID}-${TIMESTAMP}/ +# testdef.yml Attached to the test run in the bundle for +# archival purposes. +# install.sh Ditto. +# run.sh Ditto. +# stdout.log The standard output of run.sh. +# stderr.log The standard error of run.sh (actually not +# created currently) +# return_code The exit code of run.sh. +# attachments/ Contains attachments for test results. +# ${TEST_CASE_ID}/ Names the test result. +# ${FILENAME} The attached data. +# ${FILENAME}.mimetype The mime type of the attachment. +# +# After the test run has completed, the /lava/results directory is pulled over +# to the host and turned into a bundle for submission to the dashboard. + import json import yaml import logging @@ -43,21 +105,25 @@ LAVA_TEST_UPSTART = '%s/lava-test-runner.conf' % LAVA_TEST_DIR LAVA_TEST_INITD = '%s/lava-test-runner.init.d' % LAVA_TEST_DIR LAVA_TEST_SHELL = '%s/lava-test-shell' % LAVA_TEST_DIR +LAVA_TEST_CASE_ATTACH = '%s/lava-test-case-attach' % LAVA_TEST_DIR Target.android_deployment_data['lava_test_runner'] = LAVA_TEST_ANDROID Target.android_deployment_data['lava_test_shell'] = LAVA_TEST_SHELL +Target.android_deployment_data['lava_test_case_attach'] = LAVA_TEST_CASE_ATTACH Target.android_deployment_data['lava_test_sh_cmd'] = '/system/bin/mksh' Target.android_deployment_data['lava_test_dir'] = '/data/lava' Target.android_deployment_data['lava_test_results_part_attr'] = 'data_part_android_org' Target.ubuntu_deployment_data['lava_test_runner'] = LAVA_TEST_UBUNTU Target.ubuntu_deployment_data['lava_test_shell'] = LAVA_TEST_SHELL +Target.ubuntu_deployment_data['lava_test_case_attach'] = LAVA_TEST_CASE_ATTACH Target.ubuntu_deployment_data['lava_test_sh_cmd'] = '/bin/sh' Target.ubuntu_deployment_data['lava_test_dir'] = '/lava' Target.ubuntu_deployment_data['lava_test_results_part_attr'] = 'root_part' Target.oe_deployment_data['lava_test_runner'] = LAVA_TEST_UBUNTU Target.oe_deployment_data['lava_test_shell'] = LAVA_TEST_SHELL +Target.oe_deployment_data['lava_test_case_attach'] = LAVA_TEST_CASE_ATTACH Target.oe_deployment_data['lava_test_sh_cmd'] = '/bin/sh' Target.oe_deployment_data['lava_test_dir'] = '/lava' Target.oe_deployment_data['lava_test_results_part_attr'] = 'root_part' @@ -146,6 +212,13 @@ fout.write(fin.read()) os.fchmod(fout.fileno(), XMOD) + tc = target.deployment_data['lava_test_case_attach'] + with open(tc, 'r') as fin: + with open('%s/bin/lava-test-case-attach' % mntdir, 'w') as fout: + fout.write('#!%s\n\n' % shcmd) + fout.write(fin.read()) + os.fchmod(fout.fileno(), XMOD) + def _bzr_info(self, url, bzrdir): cwd = os.getcwd() try: === modified file 'lava_dispatcher/lava_test_shell.py' --- lava_dispatcher/lava_test_shell.py 2012-11-19 13:38:33 +0000 +++ lava_dispatcher/lava_test_shell.py 2012-11-19 20:02:08 +0000 @@ -20,7 +20,7 @@ import datetime import errno -import json +import mimetypes import yaml import logging import os @@ -115,7 +115,7 @@ return ctx -def _get_test_results(testdef, stdout): +def _get_test_results(testdef, stdout, attachments_dir): results = [] fixupdict = {} @@ -141,6 +141,24 @@ if res['result'] not in ('pass', 'fail', 'skip', 'unknown'): logging.error('bad test result line: %s' % line.strip()) continue + tc_id = res.get('test_case_id') + if tc_id is not None: + d = os.path.join(attachments_dir, tc_id) + if os.path.isdir(d): + attachments = os.listdir(d) + for filename in attachments: + if filename.endswith('.mimetype'): + continue + filepath = os.path.join(d, filename) + if os.path.exists(filepath + '.mimetype'): + mime_type = open(filepath + '.mimetype').read().strip() + else: + mime_type = mimetypes.guess_type(filepath)[0] + if mime_type is None: + mime_type = 'application/octet-stream' + attachment = create_attachment(filename, open(filepath).read(), mime_type) + res.setdefault('attachments', []).append(attachment) + results.append(res) return results @@ -169,6 +187,8 @@ stdout = _get_content(results_dir, '%s/stdout.log' % dirname) attachments = _get_attachments(results_dir, dirname, testdef, stdout) + attachments_dir = os.path.join(results_dir, dirname, 'attachments') + testdef = yaml.load(testdef) return { @@ -176,7 +196,7 @@ 'analyzer_assigned_date': now, 'analyzer_assigned_uuid': str(uuid4()), 'time_check_performed': False, - 'test_results': _get_test_results(testdef, stdout), + 'test_results': _get_test_results(testdef, stdout, attachments_dir), 'software_context': swcontext, 'hardware_context': hwcontext, 'attachments': attachments, @@ -215,4 +235,4 @@ except: logging.exception('error processing results for: %s' % d) - return {'test_runs': testruns, 'format': 'Dashboard Bundle Format 1.3'} + return {'test_runs': testruns, 'format': 'Dashboard Bundle Format 1.5'} === added file 'lava_test_shell/lava-test-case-attach' --- lava_test_shell/lava-test-case-attach 1970-01-01 00:00:00 +0000 +++ lava_test_shell/lava-test-case-attach 2012-11-19 19:53:13 +0000 @@ -0,0 +1,38 @@ +#NOTE the lava_test_shell_action fills in the proper interpreter path +# above during target deployment + +set -x + +# basename is not present on AOSP builds, but the /*\// thing does not +# work with dash (Ubuntu builds) or busybox (OpenEmbedded). Both of +# those have basename though. +which basename || basename () { echo ${1/*\//}; } + +usage () { + echo "USAGE" +} + +TEST_CASE_ID="$1" +shift +FILE="$1" +shift +MIMETYPE="$1" +shift +if [ $# -gt 0 ]; then + usage + exit 1 +fi +if [ ! -f "$FILE" ]; then + usage + exit 1 +fi +if [ -z "$TEST_CASE_ID" ]; then + usage + exit 1 +fi +# $LAVA_ATTACHMENT_DIR is set by lava-test-shell +mkdir -p "$LAVA_ATTACHMENT_DIR/$TEST_CASE_ID" +cp "$FILE" "$LAVA_ATTACHMENT_DIR/$TEST_CASE_ID" +if [ ! -z "$MIMETYPE" ]; then + echo "$MIMETYPE" > "$LAVA_ATTACHMENT_DIR/$TEST_CASE_ID/$(basename FILE).mimetype" +fi === modified file 'lava_test_shell/lava-test-shell' --- lava_test_shell/lava-test-shell 2012-10-03 16:37:39 +0000 +++ lava_test_shell/lava-test-shell 2012-11-16 00:51:25 +0000 @@ -6,6 +6,7 @@ shift TEST=$* RC=0 +export LAVA_ATTACHMENT_DIR=${ODIR}/attachments { $TEST echo $? > ${ODIR}/return_code