From patchwork Thu Mar 8 14:41:09 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zygmunt Krynicki X-Patchwork-Id: 7172 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id D369C23E80 for ; Thu, 8 Mar 2012 14:46:47 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 9B221A185F4 for ; Thu, 8 Mar 2012 14:46:47 +0000 (UTC) Received: by mail-iy0-f180.google.com with SMTP id e36so1044221iag.11 for ; Thu, 08 Mar 2012 06:46:47 -0800 (PST) Received: by 10.50.197.135 with SMTP id iu7mr7170935igc.50.1331218007410; Thu, 08 Mar 2012 06:46:47 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.53.18 with SMTP id k18csp11229ibg; Thu, 8 Mar 2012 06:46:46 -0800 (PST) Received: by 10.180.104.137 with SMTP id ge9mr13307810wib.20.1331218005826; Thu, 08 Mar 2012 06:46:45 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id a1si2685135wed.31.2012.03.08.06.46.45 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 08 Mar 2012 06:46:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1S5ec9-00015V-6u for ; Thu, 08 Mar 2012 14:46:45 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id BC098E0464 for ; Thu, 8 Mar 2012 14:41:09 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-test X-Launchpad-Branch: ~linaro-validation/lava-test/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 126 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-test/trunk] Rev 126: Fix and re-enable Cache class Message-Id: <20120308144109.14129.7260.launchpad@ackee.canonical.com> Date: Thu, 08 Mar 2012 14:41:09 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14914"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 1d725b78b596097f644f0f7aaf769563b8773b8d X-Gm-Message-State: ALoCoQnezGXW4GWIq/tSlRfygLmHYV0kfsVt66Yn7TdEUzl21xTsR02wq2EmhsjWFgra21Tdfmgk Merge authors: Le Chi Thu le.chi.thu@linaro.org Zygmunt Krynicki (zkrynicki) Related merge proposals: https://code.launchpad.net/~le-chi-thu/lava-test/enabled-file-cache/+merge/96015 proposed by: Le Chi Thu (le-chi-thu) review: Approve - Zygmunt Krynicki (zkrynicki) review: Resubmit - Le Chi Thu (le-chi-thu) ------------------------------------------------------------ revno: 126 [merge] committer: Zygmunt Krynicki branch nick: trunk timestamp: Thu 2012-03-08 15:39:04 +0100 message: Fix and re-enable Cache class modified: doc/changes.rst lava_test/utils.py --- lp:lava-test https://code.launchpad.net/~linaro-validation/lava-test/trunk You are subscribed to branch lp:lava-test. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-test/trunk/+edit-subscription === modified file 'doc/changes.rst' --- doc/changes.rst 2012-03-06 17:54:26 +0000 +++ doc/changes.rst 2012-03-08 10:21:25 +0000 @@ -11,6 +11,7 @@ https://bugs.launchpad.net/lava-dispatcher/+bug/934164 * Improved error output when running inside virtualenv where the apt and lsb_release modules are not visible to python. + * Enabled URL cache used by the ``lava-test register`` command. .. _version_0_4: === modified file 'lava_test/utils.py' --- lava_test/utils.py 2011-09-12 09:19:10 +0000 +++ lava_test/utils.py 2012-03-08 09:21:36 +0000 @@ -20,6 +20,7 @@ import shutil import urllib2 import urlparse +import sys _fake_files = None _fake_paths = None @@ -181,28 +182,27 @@ cls._instance = cls() return cls._instance - def open_cached(self, key, mode="r"): + def _open_cached(self, key, mode="r"): """ Acts like open() but the pathname is relative to the lava_test-specific cache directory. """ + if "w" in mode and not os.path.exists(self.cache_dir): os.makedirs(self.cache_dir) if os.path.isabs(key): raise ValueError("key cannot be an absolute path") - try: - stream = open(os.path.join(self.cache_dir, key), mode) - yield stream - finally: - stream.close() + stream = open(os.path.join(self.cache_dir, key), mode) + return stream + def _key_for_url(self, url): return hashlib.sha1(url).hexdigest() def _refresh_url_cache(self, key, url): with contextlib.nested( contextlib.closing(urllib2.urlopen(url)), - self.open_cached(key, "wb")) as (in_stream, out_stream): + self._open_cached(key, "wb")) as (in_stream, out_stream): out_stream.write(in_stream.read()) @contextlib.contextmanager @@ -212,18 +212,15 @@ """ # Do not cache local files, this is not what users would expect - # workaround - not using cache at all. - # TODO: fix this and use the cache - # if url.startswith("file://"): - if True: + if url.startswith("file://"): stream = urllib2.urlopen(url) else: key = self._key_for_url(url) try: - stream = self.open_cached(key, "rb") + stream = self._open_cached(key, "rb") except IOError: self._refresh_url_cache(key, url) - stream = self.open_cached(key, "rb") + stream = self._open_cached(key, "rb") try: yield stream finally: