From patchwork Mon Nov 5 11:36:13 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Senthil Kumaran X-Patchwork-Id: 12663 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 67C7523F56 for ; Mon, 5 Nov 2012 11:36:17 +0000 (UTC) Received: from mail-ia0-f180.google.com (mail-ia0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id DD688A191D2 for ; Mon, 5 Nov 2012 11:36:16 +0000 (UTC) Received: by mail-ia0-f180.google.com with SMTP id f6so4062776iag.11 for ; Mon, 05 Nov 2012 03:36:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=CLTm3A4sMSNmd6ArjrNZkuMnaM1mUIePXvlVWuNfGkg=; b=itHxOMmTx/S/AqXoyDFSDVrSITCVq5d7I9QK7yIfH/pD1o3pyRAcacc4j3LDhbD9KZ YA4NMuVmoZRytk6+3u2ENilhgolb+sShFwHy48j7mvtzfvw+4DGszlRD3GhgKIl4HzB1 96bETBN0JEGqThMdLV1ZYgGNED6zupFOfGlXLpxwv6biy87JI2B/jLWWBSELEovgVNny 6a8UAEMzU8XyBWvxALNFfUCM42pSDUjABC8jHDayJWvFYWm+liU2e5LE8N1m8b5rpv+T Iphzda2Ub2R97344Dz/wDaX2MkP6I7Y4hwpkceplncbmXu79u4NoNYg/RS26eOmAiJr0 W43w== Received: by 10.50.237.69 with SMTP id va5mr4528620igc.62.1352115376365; Mon, 05 Nov 2012 03:36:16 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp196710igt; Mon, 5 Nov 2012 03:36:15 -0800 (PST) Received: by 10.180.97.35 with SMTP id dx3mr12717346wib.14.1352115375013; Mon, 05 Nov 2012 03:36:15 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id x77si9217084wei.152.2012.11.05.03.36.14 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 05 Nov 2012 03:36:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TVKyT-0006Rz-TT for ; Mon, 05 Nov 2012 11:36:13 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id CD770E031E for ; Mon, 5 Nov 2012 11:36:13 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 435 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 435: Support to create test definitions in yaml instead of json. Message-Id: <20121105113613.32033.21803.launchpad@ackee.canonical.com> Date: Mon, 05 Nov 2012 11:36:13 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16232"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 7192335606f42dca630f5ee9a4f6256a1105cd5b X-Gm-Message-State: ALoCoQk9hoaDgVZ4wZDWMMjMNbdXMRd5+aovND137R9MuChqNawQlZx2TRuLNqRAbTykfsWYoPEz Merge authors: Senthil Kumaran S (stylesen) Related merge proposals: https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-yaml-support/+merge/132663 proposed by: Senthil Kumaran S (stylesen) review: Approve - Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 435 [merge] committer: Senthil Kumaran branch nick: trunk timestamp: Mon 2012-11-05 17:05:11 +0530 message: Support to create test definitions in yaml instead of json. modified: lava_dispatcher/actions/lava_test_shell.py lava_dispatcher/lava_test_shell.py lava_test_shell/lava-test-runner-android lava_test_shell/lava-test-runner-ubuntu --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/lava_test_shell.py' --- lava_dispatcher/actions/lava_test_shell.py 2012-10-23 04:46:20 +0000 +++ lava_dispatcher/actions/lava_test_shell.py 2012-11-02 08:33:10 +0000 @@ -21,6 +21,7 @@ # with this program; if not, see . import json +import yaml import logging import os import pexpect @@ -131,7 +132,7 @@ testdef_file = download_image(testdef_url, self.context, tmpdir) with open(testdef_file, 'r') as f: logging.info('loading test definition') - return json.load(f) + return yaml.load(f) def _copy_runner(self, mntdir, target): runner = target.deployment_data['lava_test_runner'] @@ -197,22 +198,24 @@ f.write('cd %s\n' % targetdir) # TODO how should we handle this for Android? - if 'deps' in testdef['install']: + if 'deps' in testdef['install'] and \ + testdef['install']['deps'] is not None: f.write('sudo apt-get update\n') f.write('sudo apt-get install -y ') for dep in testdef['install']['deps']: f.write('%s ' % dep) f.write('\n') - if 'steps' in testdef['install']: + if 'steps' in testdef['install'] and \ + testdef['install']['steps'] is not None: for cmd in testdef['install']['steps']: f.write('%s\n' % cmd) def _copy_test(self, hostdir, targetdir, testdef): self._sw_sources = [] utils.ensure_directory(hostdir) - with open('%s/testdef.json' % hostdir, 'w') as f: - f.write(json.dumps(testdef)) + with open('%s/testdef.yaml' % hostdir, 'w') as f: + f.write(yaml.dump(testdef)) if 'install' in testdef: self._create_repos(testdef, hostdir) @@ -221,8 +224,10 @@ with open('%s/run.sh' % hostdir, 'w') as f: f.write('set -e\n') f.write('cd %s\n' % targetdir) - for cmd in testdef['run']['steps']: - f.write('%s\n' % cmd) + if 'steps' in testdef['run'] \ + and testdef['run']['steps'] is not None: + for cmd in testdef['run']['steps']: + f.write('%s\n' % cmd) def _mk_runner_dirs(self, mntdir): utils.ensure_directory('%s/bin' % mntdir) @@ -243,8 +248,8 @@ # android mount the partition under /system, while ubuntu # mounts under /, so we have hdir for where it is on the host # and tdir for how the target will see the path - hdir = '%s/tests/%d_%s' % (d, i, testdef['test_id']) - tdir = '%s/tests/%d_%s' % (ldir, i, testdef['test_id']) + hdir = '%s/tests/%d_%s' % (d, i, testdef.get('metadata').get('name')) + tdir = '%s/tests/%d_%s' % (ldir, i, testdef.get('metadata').get('name')) self._copy_test(hdir, tdir, testdef) testdirs.append(tdir) === modified file 'lava_dispatcher/lava_test_shell.py' --- lava_dispatcher/lava_test_shell.py 2012-10-14 21:44:55 +0000 +++ lava_dispatcher/lava_test_shell.py 2012-11-02 08:33:10 +0000 @@ -21,6 +21,7 @@ import datetime import errno import json +import yaml import logging import os import re @@ -119,7 +120,6 @@ pattern = re.compile(testdef['parse']['pattern']) - fixupdict = {} if 'fixupdict' in testdef['parse']: fixupdict = testdef['parse']['fixupdict'] @@ -143,7 +143,7 @@ attachments = [] attachments.append(create_attachment('stdout.txt', stdout)) - attachments.append(create_attachment('testdef.json', testdef)) + attachments.append(create_attachment('testdef.yaml', testdef)) for f in files: fname = '%s/%s' % (dirname, f) @@ -157,14 +157,14 @@ def _get_test_run(results_dir, dirname, hwcontext, swcontext): now = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ') - testdef = _get_content(results_dir, '%s/testdef.json' % dirname) + testdef = _get_content(results_dir, '%s/testdef.yaml' % dirname) stdout = _get_content(results_dir, '%s/stdout.log' % dirname) attachments = _get_attachments(results_dir, dirname, testdef, stdout) - testdef = json.loads(testdef) + testdef = yaml.load(testdef) return { - 'test_id': testdef['test_id'], + 'test_id': testdef.get('metadata').get('name'), 'analyzer_assigned_date': now, 'analyzer_assigned_uuid': str(uuid4()), 'time_check_performed': False, === modified file 'lava_test_shell/lava-test-runner-android' --- lava_test_shell/lava-test-runner-android 2012-10-22 20:41:43 +0000 +++ lava_test_shell/lava-test-runner-android 2012-11-02 08:33:10 +0000 @@ -79,7 +79,7 @@ echo "${PREFIX} running ${test} under lava-test-shell..." odir=${RESULTSDIR}/${test}-`date +%s` mkdir ${odir} - cp ${line}/testdef.json ${odir}/ + cp ${line}/testdef.yaml ${odir}/ cp ${line}/run.sh ${odir}/ [ -f ${line}/install.sh ] && cp ${line}/install.sh ${odir}/ lava-test-shell --output_dir ${odir} /system/bin/sh -e "${line}/run.sh" === modified file 'lava_test_shell/lava-test-runner-ubuntu' --- lava_test_shell/lava-test-runner-ubuntu 2012-10-22 20:41:43 +0000 +++ lava_test_shell/lava-test-runner-ubuntu 2012-11-02 08:33:10 +0000 @@ -63,7 +63,7 @@ echo "${PREFIX} running ${test} under lava-test-shell..." odir=${RESULTSDIR}/${test}-`date +%s` mkdir ${odir} - cp ${line}/testdef.json ${odir}/ + cp ${line}/testdef.yaml ${odir}/ cp ${line}/run.sh ${odir}/ [ -f ${line}/install.sh ] && cp ${line}/install.sh ${odir}/ lava-test-shell --output_dir ${odir} /bin/sh -e "${line}/run.sh"