From patchwork Thu Aug 2 23:09:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Doan X-Patchwork-Id: 10474 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id D461224050 for ; Thu, 2 Aug 2012 23:09:15 +0000 (UTC) Received: from mail-yw0-f52.google.com (mail-yw0-f52.google.com [209.85.213.52]) by fiordland.canonical.com (Postfix) with ESMTP id 8DCE0A19296 for ; Thu, 2 Aug 2012 23:09:15 +0000 (UTC) Received: by yhpp61 with SMTP id p61so97632yhp.11 for ; Thu, 02 Aug 2012 16:09:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=gOfzwzisNSLve59eTy4IEkshr86Tum+JjI0nuslKCPg=; b=CGg2RrtJATHSh7yTtTRUZGdbty9/MfO0QMnofs2eyEtFo3y1yTno0Gsf7OaxrgAhQ1 GU2vcy3nQilYsmboR/FdoKwUU55r0opbDNvm2W66PdWPp84yrxJkLSj8uXIWMycuGE2A keoiYQDidIDaeyCKtIMwemS20zvM2pcit5TNJLXe4CRyuIQt27pw+sqnUxiwjrPNNs/p tHaXJnIAuLWA3DLyOV2Aen+2Z8DejTyCf6ZnFVpGvj1TTwajI7S2SLq/DDeAhtSufqQR nO3xRwWcXaoLZpikbxbnqsiDrOdn8X/hFm4Xwp1EXNB8HYyTh4uhbXnenh+X8UTkRUub vS1Q== Received: by 10.43.69.12 with SMTP id ya12mr6402886icb.50.1343948954848; Thu, 02 Aug 2012 16:09:14 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.87.40 with SMTP id u8csp283704igz; Thu, 2 Aug 2012 16:09:13 -0700 (PDT) Received: by 10.180.84.1 with SMTP id u1mr8323447wiy.15.1343948952349; Thu, 02 Aug 2012 16:09:12 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id o3si406022wic.0.2012.08.02.16.09.11 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 02 Aug 2012 16:09:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Sx4Vz-0008Vk-Lq for ; Thu, 02 Aug 2012 23:09:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 9B5F1E017C for ; Thu, 2 Aug 2012 23:09:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 363 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 363: import attachment support for client implementations Message-Id: <20120802230911.21293.93263.launchpad@ackee.canonical.com> Date: Thu, 02 Aug 2012 23:09:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15719"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 80840023fb0f066ad8001b678c9218359f4cdb58 X-Gm-Message-State: ALoCoQnnj6uSncC6z7GTzGCnsI69/R0Ucra1bOVK1GyOAoa2JKd0AqWIYEYiEathlHBFArS6sayz Merge authors: Andy Doan (doanac) Related merge proposals: https://code.launchpad.net/~doanac/lava-dispatcher/attchments/+merge/117522 proposed by: Andy Doan (doanac) review: Approve - Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 363 [merge] committer: Andy Doan branch nick: lava-dispatcher timestamp: Thu 2012-08-02 18:05:53 -0500 message: import attachment support for client implementations modified: lava_dispatcher/actions/launch_control.py lava_dispatcher/client/base.py lava_dispatcher/client/fastmodel.py lava_dispatcher/test_data.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/launch_control.py' --- lava_dispatcher/actions/launch_control.py 2012-07-24 04:01:01 +0000 +++ lava_dispatcher/actions/launch_control.py 2012-07-31 20:20:02 +0000 @@ -168,8 +168,8 @@ for bundle in all_bundles: test_runs += bundle['test_runs'] - self.context.test_data.add_seriallog( - self.context.client.get_seriallog()) + attachments = self.client.get_test_data_attachments() + self.context.test_data.add_attachments(attachments) main_bundle['test_runs'].append(self.context.test_data.get_test_run()) === modified file 'lava_dispatcher/client/base.py' --- lava_dispatcher/client/base.py 2012-08-01 17:52:55 +0000 +++ lava_dispatcher/client/base.py 2012-08-02 23:05:53 +0000 @@ -33,6 +33,8 @@ from cStringIO import StringIO from tempfile import mkdtemp +from lava_dispatcher.test_data import create_attachment + class CommandRunner(object): """A convenient way to run a shell command and wait for a shell prompt. @@ -424,9 +426,6 @@ self.setup_proxy(self.tester_str) logging.info("System is in test image now") - def get_seriallog(self): - return self.sio.getvalue() - def get_www_scratch_dir(self): ''' returns a temporary directory available for downloads that's gets deleted when the process exits ''' @@ -436,6 +435,10 @@ os.chmod(d, 0755) return d + def get_test_data_attachments(self): + '''returns attachments to go in the "lava_results" test run''' + return [ create_attachment('serial.log', self.sio.getvalue()) ] + # Android stuff def get_android_adb_interface(self): === modified file 'lava_dispatcher/client/fastmodel.py' --- lava_dispatcher/client/fastmodel.py 2012-08-01 17:52:55 +0000 +++ lava_dispatcher/client/fastmodel.py 2012-08-02 23:05:53 +0000 @@ -19,7 +19,9 @@ # with this program; if not, see . import atexit +import codecs import contextlib +import cStringIO import logging import os import pexpect @@ -40,6 +42,9 @@ from lava_dispatcher.downloader import ( download_image, ) +from lava_dispatcher.test_data import ( + create_attachment, + ) from lava_dispatcher.utils import ( logging_spawn, logging_system, @@ -192,6 +197,17 @@ if self._sim_proc is not None: self._sim_proc.close() + def _drain_sim_proc(self): + '''pexpect will continue to get data for the simproc process. We need + to keep this pipe drained so that it won't get full and then stop block + the process from continuing to execute''' + + # NOTE: the RTSM binary uses the windows code page(cp1252), but the + # dashboard needs this with a utf-8 encoding + f = cStringIO.StringIO() + self._sim_proc.logfile = codecs.EncodedFile(f, 'cp1252', 'utf-8') + _pexpect_drain(self._sim_proc).start() + def _boot_linaro_image(self): self._stop() @@ -215,7 +231,7 @@ if match == 0: raise RuntimeError("fast model license check failed") - _pexpect_drain(self._sim_proc).start() + self._drain_sim_proc() logging.info('simulator is started connecting to serial port') self.proc = logging_spawn( @@ -242,6 +258,16 @@ tarfile, mnt, self.context.lava_result_dir)) return 'pass', '', tarfile + def get_test_data_attachments(self): + '''returns attachments to go in the "lava_results" test run''' + a = super(LavaFastModelClient, self).get_test_data_attachments() + + # if the simulator never got started we won't even get to a logfile + if getattr(self._sim_proc, 'logfile', None) is not None: + content = self._sim_proc.logfile.getvalue() + a.append( create_attachment('rtsm.log', content) ) + return a + class _pexpect_drain(threading.Thread): ''' The simulator process can dump a lot of information to its console. If don't actively read from it, the pipe will get full and the process will @@ -254,5 +280,4 @@ def run(self): # change simproc's stdout so it doesn't overlap the stdout from our # serial console logging - self.proc.logfile = open('/dev/null', 'w') self.proc.drain() === modified file 'lava_dispatcher/test_data.py' --- lava_dispatcher/test_data.py 2012-05-08 18:05:52 +0000 +++ lava_dispatcher/test_data.py 2012-07-31 20:20:02 +0000 @@ -22,6 +22,14 @@ from uuid import uuid1 import base64 + +def create_attachment(pathname, content, mime_type='text/plain'): + return { + 'pathname': pathname, + 'mime_type': mime_type, + 'content': base64.b64encode(content), + } + class LavaTestData(object): def __init__(self, test_id='lava'): self.job_status = 'pass' @@ -48,8 +56,8 @@ } self._test_run['test_results'].append(result_data) - def add_attachment(self, attachment): - self._test_run['attachments'].append(attachment) + def add_attachments(self, attachments): + self._test_run['attachments'].extend(attachments) def add_tag(self, tag): self._test_run['tags'].append(tag) @@ -68,13 +76,3 @@ self.add_result('job_complete', self.job_status) return self._test_run - def add_seriallog(self, serial_log): - """ - Add serial log to the "attachments" field, it aligns bundle 1.2 format - """ - serial_log_base64 = base64.b64encode(serial_log) - attachment = { - "pathname": "serial.log", - "mime_type": "text/plain", - "content": serial_log_base64 } - self.add_attachment(attachment)