From patchwork Tue Aug 6 08:40:38 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milo Casagrande X-Patchwork-Id: 18790 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ye0-f200.google.com (mail-ye0-f200.google.com [209.85.213.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id F113623A4E for ; Tue, 6 Aug 2013 08:40:40 +0000 (UTC) Received: by mail-ye0-f200.google.com with SMTP id r13sf95576yen.3 for ; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:delivered-to:mime-version:to:from:subject :message-id:date:reply-to:sender:errors-to:precedence :x-original-sender:x-original-authentication-results:mailing-list :list-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=reQkzy2GroAwnBa9+PrNDbUiGefHgxa+yTtI/CBX46U=; b=cCTs9VmqnvNKkCUL9GcgmWkFnbXHu+Y2eVr1JzvNhqicAtywyEKrpCm9D3f4O0Cpp9 lEr1BaaZe/hFHugDCWH4kXGsx1JA9exMnQbSMd/OFU44AL8GnF6IaxnoquFj02auHZMI /kGvY2NcKfPlvIm34dCYW/35lyhKdhDFqU4MeGfBBeD3IW7ksP9JLTy0SYXu3guBI7Zl omvVqfq6auFV462fJM8uKe5tB9JE+P5GUUw9ktbVFRVnkJUG6t8jlsEVwFl6ZyE6cq9c GDlRrE43zDjDIz16Yq7s6FF+WRb1xYsb5W+8/IAm21ymTHTwAo3P/6oZfPBTmEF/cCN4 /ZYg== X-Received: by 10.224.103.68 with SMTP id j4mr1992090qao.8.1375778440790; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.34.230 with SMTP id c6ls112591qej.79.gmail; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) X-Received: by 10.58.128.71 with SMTP id nm7mr59892veb.51.1375778440674; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) Received: from mail-ve0-f178.google.com (mail-ve0-f178.google.com [209.85.128.178]) by mx.google.com with ESMTPS id o8si67759vcy.94.2013.08.06.01.40.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 06 Aug 2013 01:40:40 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.178; Received: by mail-ve0-f178.google.com with SMTP id ox1so97743veb.23 for ; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) X-Gm-Message-State: ALoCoQnYhpE5LK/wZVGiqjfJQtGZjO4HQCWZMlqmcpYB6OEKRTetpEF0y19USYNN+qO1u55gVbJ2 X-Received: by 10.220.198.133 with SMTP id eo5mr75424vcb.24.1375778440544; Tue, 06 Aug 2013 01:40:40 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.221.11.8 with SMTP id pc8csp133948vcb; Tue, 6 Aug 2013 01:40:39 -0700 (PDT) X-Received: by 10.194.240.101 with SMTP id vz5mr146479wjc.69.1375778438771; Tue, 06 Aug 2013 01:40:38 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id da6si192776wjc.57.2013.08.06.01.40.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 06 Aug 2013 01:40:38 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1V6coo-0000iX-Ao for ; Tue, 06 Aug 2013 08:40:38 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 45423E04F3 for ; Tue, 6 Aug 2013 08:40:38 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-tool X-Launchpad-Branch: ~linaro-validation/lava-tool/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 190 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-tool/trunk] Rev 190: Added support for XDG directories. Message-Id: <20130806084038.22025.35033.launchpad@ackee.canonical.com> Date: Tue, 06 Aug 2013 08:40:38 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: list X-Generated-By: Launchpad (canonical.com); Revision="16718"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 856f6e69e2da75333cedda62ba91ab1386a569b2 X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: noreply@launchpad.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.178 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Merge authors: Milo Casagrande (milo) Related merge proposals: https://code.launchpad.net/~milo/lava-tool/bug1206579/+merge/177829 proposed by: Milo Casagrande (milo) review: Approve - Antonio Terceiro (terceiro) ------------------------------------------------------------ revno: 190 [merge] committer: Milo Casagrande branch nick: trunk timestamp: Tue 2013-08-06 10:10:58 +0200 message: Added support for XDG directories. modified: integration-tests.d/lava-job-new-with-config.sh integration-tests.d/lava-job-submit.sh lava/config.py lava/helper/command.py lava/testdef/tests/test_commands.py lava/tests/test_config.py setup.py --- lp:lava-tool https://code.launchpad.net/~linaro-validation/lava-tool/trunk You are subscribed to branch lp:lava-tool. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-tool/trunk/+edit-subscription === modified file 'integration-tests.d/lava-job-new-with-config.sh' --- integration-tests.d/lava-job-new-with-config.sh 2013-07-30 15:45:44 +0000 +++ integration-tests.d/lava-job-new-with-config.sh 2013-08-06 08:10:58 +0000 @@ -5,6 +5,6 @@ [device_type=panda] image = file:///path/to/panda.img EOF -LAVACONFIG="${tmpdir}/config" lava job new "${tmpdir}/job.json" -n +LAVACACHE="${tmpdir}/config" lava job new "${tmpdir}/job.json" -n cat "${tmpdir}/job.json" grep "device_type.*panda" "${tmpdir}/job.json" && grep "image.*path.to.panda.img" "${tmpdir}/job.json" === modified file 'integration-tests.d/lava-job-submit.sh' --- integration-tests.d/lava-job-submit.sh 2013-07-30 17:42:36 +0000 +++ integration-tests.d/lava-job-submit.sh 2013-08-06 08:10:58 +0000 @@ -1,12 +1,12 @@ fixed_response 999 -lava_config < $tmpdir/output +LAVACACHE="${tmpdir}/config" lava job submit integration-tests.d/sample/nexus.json -n > $tmpdir/output grep "Job submitted with job ID 999" $tmpdir/output === modified file 'lava/config.py' --- lava/config.py 2013-07-26 13:48:06 +0000 +++ lava/config.py 2013-07-31 12:27:59 +0000 @@ -23,6 +23,7 @@ import atexit import os import readline +import xdg.BaseDirectory as xdgBaseDir from ConfigParser import ( ConfigParser, @@ -33,12 +34,16 @@ from lava.parameter import Parameter from lava.tool.errors import CommandError -__all__ = ['Config', 'InteractiveConfig'] +__all__ = ['Config', 'InteractiveCache', 'InteractiveConfig'] # Store for function calls to be made at exit time. AT_EXIT_CALLS = set() # Config default section. DEFAULT_SECTION = "DEFAULT" +# This is the default base name used to create XDG resources. +DEFAULT_XDG_RESOURCE = "linaro" +# This is the default name for lava-tool resources. +DEFAULT_LAVA_TOOL_RESOURCE = "lava-tool" HISTORY = os.path.join(os.path.expanduser("~"), ".lava_history") try: @@ -69,8 +74,8 @@ def config_file(self): if self._config_file is None: self._config_file = (os.environ.get('LAVACONFIG') or - os.path.join(os.path.expanduser('~'), - '.lavaconfig')) + os.path.join(self._ensure_xdg_dirs(), + 'lava-tool.ini')) return self._config_file @config_file.setter @@ -84,6 +89,14 @@ self._config_backend.read([self.config_file]) return self._config_backend + def _ensure_xdg_dirs(self): + """Make sure we have the default resource. + + :return The path to the XDG resource. + """ + return xdgBaseDir.save_config_path(DEFAULT_XDG_RESOURCE, + DEFAULT_LAVA_TOOL_RESOURCE) + def _calculate_config_section(self, parameter): """Calculates the config section of the specified parameter. @@ -248,3 +261,34 @@ if value is not None and parameter.store: self.put(parameter.id, value, section) return value + + +class InteractiveCache(InteractiveConfig): + + """An interactive cache where parameters that can change are stored. + + This class is basically the same as the Confing and InteractiveConfig ones, + only the base directory where the cache file is stored is different. + + In this case it will use the $XDG_CACHE_HOME value as defined in XDG. + """ + + @property + def config_file(self): + if self._config_file is None: + self._config_file = (os.environ.get('LAVACACHE') or + os.path.join(self._ensure_xdg_dirs(), + 'parameters.ini')) + return self._config_file + + @config_file.setter + def config_file(self, value): + self._config_file = value + + def _ensure_xdg_dirs(self): + """Make sure we have the default resource. + + :return The path to the XDG resource. + """ + return xdgBaseDir.save_cache_path(DEFAULT_XDG_RESOURCE, + DEFAULT_LAVA_TOOL_RESOURCE) === modified file 'lava/helper/command.py' --- lava/helper/command.py 2013-07-29 10:09:09 +0000 +++ lava/helper/command.py 2013-07-31 10:24:23 +0000 @@ -22,7 +22,9 @@ import sys import xmlrpclib -from lava.config import InteractiveConfig +from lava.config import ( + InteractiveCache, +) from lava.helper.dispatcher import get_devices from lava.job import Job from lava.job.templates import ( @@ -58,7 +60,7 @@ verify_and_create_url, ) -CONFIG = InteractiveConfig() +CONFIG = InteractiveCache() class BaseCommand(Command): === modified file 'lava/testdef/tests/test_commands.py' --- lava/testdef/tests/test_commands.py 2013-07-26 08:10:16 +0000 +++ lava/testdef/tests/test_commands.py 2013-07-31 12:36:18 +0000 @@ -25,10 +25,11 @@ import yaml from mock import ( + MagicMock, patch, ) -from lava.config import InteractiveConfig +from lava.config import InteractiveCache from lava.helper.tests.helper_test import HelperTest from lava.testdef.commands import ( new, @@ -39,8 +40,7 @@ class NewCommandTest(HelperTest): """Class for the lava.testdef new command tests.""" - @patch("lava.config.Config.save") - def setUp(self, mocked_save): + def setUp(self): super(NewCommandTest, self).setUp() self.file_name = "fake_testdef.yaml" self.file_path = os.path.join(tempfile.gettempdir(), self.file_name) @@ -50,7 +50,8 @@ delete=False) self.config_file = tempfile.NamedTemporaryFile(delete=False) - self.config = InteractiveConfig() + self.config = InteractiveCache() + self.config.save = MagicMock() self.config.config_file = self.config_file.name # Patch class raw_input, start it, and stop it on tearDown. self.patcher1 = patch("lava.parameter.raw_input", create=True) @@ -84,6 +85,7 @@ # file system. self.mocked_raw_input.return_value = "\n" new_command = new(self.parser, self.args) + new_command.config = self.config new_command.invoke() self.assertTrue(os.path.exists(self.file_path)) @@ -92,6 +94,7 @@ # thrown. self.args.FILE = self.temp_yaml.name new_command = new(self.parser, self.args) + new_command.config = self.config self.assertRaises(CommandError, new_command.invoke) def test_invoke_2(self): @@ -124,6 +127,7 @@ self.args.FILE = "/test_file.yaml" self.mocked_raw_input.return_value = "\n" new_command = new(self.parser, self.args) + new_command.config = self.config self.assertRaises(CommandError, new_command.invoke) self.assertFalse(os.path.exists(self.args.FILE)) @@ -133,6 +137,7 @@ self.mocked_raw_input.side_effect = ["foo", "\n", "\n", "\n", "\n", "\n"] new_command = new(self.parser, self.args) + new_command.config = self.config new_command.invoke() expected = {'run': {'steps': ["./mytest.sh"]}, 'metadata': { === modified file 'lava/tests/test_config.py' --- lava/tests/test_config.py 2013-07-26 08:10:16 +0000 +++ lava/tests/test_config.py 2013-07-31 12:35:26 +0000 @@ -20,6 +20,8 @@ lava.config unit tests. """ +import os +import shutil import sys from StringIO import StringIO @@ -31,6 +33,7 @@ from lava.config import ( Config, + InteractiveCache, InteractiveConfig, ) from lava.helper.tests.helper_test import HelperTest @@ -49,17 +52,75 @@ self.param2 = Parameter("bar", depends=self.param1) +class TestConfigSave(ConfigTestCase): + + """Used to test the save() method of config class. + + Done here since in the other tests we want to mock the atexit save call + in order not to write the file, or accidentaly overwrite the real + user file. + """ + + def setUp(self): + super(TestConfigSave, self).setUp() + self.config = Config() + self.config.config_file = self.temp_file.name + + def test_config_save(self): + self.config.put_parameter(self.param1, "foo") + self.config.save() + + expected = "[DEFAULT]\nfoo = foo\n\n" + obtained = "" + with open(self.temp_file.name) as tmp_file: + obtained = tmp_file.read() + self.assertEqual(expected, obtained) + + def test_save_list_param(self): + # Tests that when saved to file, the ListParameter parameter is stored + # correctly. + param_values = ["foo", "more than one words", "bar"] + list_param = ListParameter("list") + list_param.set(param_values) + + self.config.put_parameter(list_param, param_values) + self.config.save() + + expected = "[DEFAULT]\nlist = " + ",".join(param_values) + "\n\n" + obtained = "" + with open(self.temp_file.name, "r") as read_file: + obtained = read_file.read() + self.assertEqual(expected, obtained) + + class ConfigTest(ConfigTestCase): - @patch("lava.config.Config.save") - def setUp(self, mocked_save): + def setUp(self): super(ConfigTest, self).setUp() + self.patcher = patch("lava.config.DEFAULT_XDG_RESOURCE", "a_temp_dir") + self.patcher.start() + self.xdg_resource = os.path.join( + os.path.expanduser("~"), ".config/a_temp_dir") + self.lavatool_resource = os.path.join(self.xdg_resource, "lava-tool") self.config = Config() - self.config.config_file = self.temp_file.name - - def test_assert_temp_config_file(self): - # Dummy test to make sure we are overriding correctly the Config class. - self.assertEqual(self.config.config_file, self.temp_file.name) + self.config.save = MagicMock() + + def tearDown(self): + super(ConfigTest, self).tearDown() + self.patcher.stop() + if os.path.isdir(self.xdg_resource): + shutil.rmtree(self.xdg_resource) + + def test_ensure_xdg_dirs(self): + # Test that xdg can create the correct cache path, we remove it + # at the end since we patch the default value. + obtained = self.config._ensure_xdg_dirs() + self.assertEquals(self.lavatool_resource, obtained) + + def test_config_file(self): + expected = os.path.join(self.lavatool_resource, "lava-tool.ini") + obtained = self.config.config_file + self.assertEquals(expected, obtained) def test_config_put_in_cache_0(self): self.config._put_in_cache("key", "value", "section") @@ -138,16 +199,6 @@ obtained = self.config._calculate_config_section(self.param2) self.assertEqual(expected, obtained) - def test_config_save(self): - self.config.put_parameter(self.param1, "foo") - self.config.save() - - expected = "[DEFAULT]\nfoo = foo\n\n" - obtained = "" - with open(self.temp_file.name) as tmp_file: - obtained = tmp_file.read() - self.assertEqual(expected, obtained) - def test_config_get_from_backend_public(self): # Need to to this, since we want a clean Config instance, with # a config_file with some content. @@ -160,10 +211,10 @@ class InteractiveConfigTest(ConfigTestCase): - @patch("lava.config.Config.save") - def setUp(self, mocked_save): + def setUp(self): super(InteractiveConfigTest, self).setUp() self.config = InteractiveConfig() + self.config.save = MagicMock() self.config.config_file = self.temp_file.name @patch("lava.config.Config.get", new=MagicMock(return_value=None)) @@ -264,18 +315,38 @@ self.assertIsInstance(obtained, list) self.assertEqual(expected, obtained) - def test_interactive_save_list_param(self): - # Tests that when saved to file, the ListParameter parameter is stored - # correctly. - param_values = ["foo", "more than one words", "bar"] - list_param = ListParameter("list") - list_param.set(param_values) - - self.config.put_parameter(list_param, param_values) - self.config.save() - - expected = "[DEFAULT]\nlist = " + ",".join(param_values) + "\n\n" - obtained = "" - with open(self.temp_file.name, "r") as read_file: - obtained = read_file.read() - self.assertEqual(expected, obtained) + +class TestInteractiveCache(HelperTest): + + def setUp(self): + super(TestInteractiveCache, self).setUp() + self.patcher = patch("lava.config.DEFAULT_XDG_RESOURCE", "a_temp_dir") + self.patcher.start() + self.cache = InteractiveCache() + self.cache.save = MagicMock() + self.xdg_resource = os.path.join( + os.path.expanduser("~"), ".cache/a_temp_dir") + self.lavatool_resource = os.path.join(self.xdg_resource, "lava-tool") + + def tearDown(self): + super(TestInteractiveCache, self).tearDown() + self.patcher.stop() + if os.path.isdir(self.xdg_resource): + shutil.rmtree(self.xdg_resource) + + def test_default_xdg(self): + # Dummy test, only to make sure patching the module attribute works. + from lava.config import DEFAULT_XDG_RESOURCE + self.assertEquals(DEFAULT_XDG_RESOURCE, "a_temp_dir") + + def test_ensure_xdg_dirs(self): + # Test that xdg can create the correct cache path, we remove it + # at the end since we patch the default value. + obtained = self.cache._ensure_xdg_dirs() + self.assertEquals(self.lavatool_resource, obtained) + + def test_config_file(self): + expected = os.path.join(self.lavatool_resource, "parameters.ini") + obtained = self.cache.config_file + self.assertEquals(expected, obtained) + === modified file 'setup.py' --- setup.py 2013-07-17 14:30:42 +0000 +++ setup.py 2013-07-31 10:24:54 +0000 @@ -51,7 +51,8 @@ 'argcomplete >= 0.3', 'keyring', 'json-schema-validator >= 2.0', - 'versiontools >= 1.3.1' + 'versiontools >= 1.3.1', + 'pyxdg == 0.25', ], setup_requires=['versiontools >= 1.3.1'], tests_require=[