[Branch,~linaro-validation/lava-dispatcher/trunk] Rev 105: merge the config branch of doom.

Message ID 20110909051113.7902.39243.launchpad@ackee.canonical.com
State Accepted
Headers show

Commit Message

Michael-Doyle Hudson Sept. 9, 2011, 5:11 a.m.
Merge authors:
  David Schwarz (david-schwarz)
  Michael Hudson-Doyle (mwhudson)
  Paul Larson (pwlars)
  Paul Larson <paul.larson@linaro.org>
  Spring Zhang (qzhang)
Related merge proposals:
  https://code.launchpad.net/~mwhudson/lava-dispatcher/config-clients-merge/+merge/74712
  proposed by: Michael Hudson-Doyle (mwhudson)
  review: Approve - Paul Larson (pwlars)
  review: Needs Fixing - Spring Zhang (qzhang)
  https://code.launchpad.net/~pwlars/lava-dispatcher/config-clients-merge/+merge/71414
  proposed by: Paul Larson (pwlars)
  review: Needs Fixing - Spring Zhang (qzhang)
  https://code.launchpad.net/~david-schwarz/lava-dispatcher/ssh_qemu_clients_v3/+merge/70892
  proposed by: David Schwarz (david-schwarz)
------------------------------------------------------------
revno: 105 [merge]
committer: Michael-Doyle Hudson <michael.hudson@linaro.org>
branch nick: trunk
timestamp: Fri 2011-09-09 17:08:38 +1200
message:
  merge the config branch of doom.
  all configuration is now done through config files which can be overridden
  in site wide and user local locations.
removed:
  lava_dispatcher/android_config.py
added:
  MANIFEST.in
  lava_dispatcher/default-config/
  lava_dispatcher/default-config/lava-dispatcher/
  lava_dispatcher/default-config/lava-dispatcher/README
  lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf
  lava_dispatcher/default-config/lava-dispatcher/device-types/
  lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf
  lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf
  lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf
  lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf
  lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf
  lava_dispatcher/default-config/lava-dispatcher/devices/
  lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf
modified:
  .bzrignore
  README
  doc/QUICKSTART
  lava_dispatcher/__init__.py
  lava_dispatcher/actions/android_basic.py
  lava_dispatcher/actions/android_deploy.py
  lava_dispatcher/actions/deploy.py
  lava_dispatcher/actions/launch_control.py*
  lava_dispatcher/actions/lava-test.py*
  lava_dispatcher/android_client.py
  lava_dispatcher/android_util.py
  lava_dispatcher/client.py
  lava_dispatcher/config.py
  lava_dispatcher/tests/test_config.py
  lava_dispatcher/utils.py
  setup.py


--
lp:lava-dispatcher
https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk

You are subscribed to branch lp:lava-dispatcher.
To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription

Patch

=== modified file '.bzrignore'
--- .bzrignore	2011-04-07 01:21:51 +0000
+++ .bzrignore	2011-09-09 00:57:07 +0000
@@ -3,3 +3,6 @@ 
 ./.virtualenv
 ./bin
 ./lava.egg-info
+./dist
+./lava_dispatcher.egg-info
+./build

=== added file 'MANIFEST.in'
--- MANIFEST.in	1970-01-01 00:00:00 +0000
+++ MANIFEST.in	2011-09-09 00:57:07 +0000
@@ -0,0 +1,2 @@ 
+include README
+recursive-include lava_dispatcher/default-config *.conf
\ No newline at end of file

=== modified file 'README'
--- README	2011-06-27 04:55:08 +0000
+++ README	2011-09-09 04:53:56 +0000
@@ -1,33 +1,12 @@ 
-LAVA - Automated Validation for Linaro
-======================================
-
-LAVA is a bundle of software that allows automated testing of Linaro images.
-
-It contains code both for running the tests (the "dispatcher" in local jargon)
-and requesting/organizing the running of the tests (the "scheduler").
-
-Code-wise, it consists of three major parts:
-
- - the dispatcher
- - the scheduler interface
- - the scheduler daemon
+LAVA dispatcher - automated testing of Linaro images
+====================================================
+
+The LAVA dispatcher allows automated testing of Linaro images.
 
 Quickstart
 ----------
 
-You need to have the python-virtualenv, python-pip, xxx, yyy and zzz
-packages installed.
-
-To set up the virtual environment with all the Python dependencies,
-run "make".  You can load some sample data to bootstrap interactive playing
-around with "make loadsampledata".
-
-Then, to run the interface, run "./bin/manage runserver" (after
-running "make").
-
-Running the dispatcher requires a bit of setting up.  See below.
-
-To run the daemon ...
+See doc/QUICKSTART.
 
 
 Android Debug Bridge (adb) installation
@@ -44,36 +23,3 @@ 
 
 You might want to add the location of the SDK's tools/ and platform-tools 
 to your PATH environment variable, to provide easy access to the tools.
-
-
-The Dispatcher
---------------
-
-The code for the dispatcher lives in lava_dispatcher.
-
-XXX write this -- reference ./doc/QUICKSTART or copy it here.
-
-The Scheduler Interface
------------------------
-
-The code for the scheduler interface lives in
-lava.scheduler.interface.
-
-It is a Django application that provides both a human (i.e. HTML) and
-machine (i.e. XML-RPC) interface to the scheduling of jobs.  It is a
-fairly simple interface to a database in the traditional Django-Python
-webapp style.
-
-It will delegate to Launchpad for auth and the API side will probably
-use OAuth for authentication.
-
-To be able to use the admin interface, log in via Launchpad OpenID and
-then run "bin/manage makesuperuser $your_lp_id".
-
-
-The Scheduler Daemon
---------------------
-
-The scheduler daemon usage:
-
- * ./lava-schedulerd start|stop|restart

=== modified file 'doc/QUICKSTART'
--- doc/QUICKSTART	2011-06-10 04:46:04 +0000
+++ doc/QUICKSTART	2011-09-09 04:53:56 +0000
@@ -1,14 +1,19 @@ 
+Dispatcher Quick Start
+======================
+
 Dependencies
 ------------
 
-LAVA needs python >= 2.6.
-
-For the LAVA client, you need the pexpect module.
+LAVA needs python >= 2.6.  For the dispatcher, you need the pexpect
+module.
 
 
 Configuring for local development
 ---------------------------------
 
+Configuring conmux
+++++++++++++++++++
+
 You will need to have a card containing a 'master image' for your
 board.  The process of creating a master image is outlined on
 https://wiki.linaro.org/Platform/Validation/Specs/MasterBootImage.
@@ -17,7 +22,8 @@ 
 you should be able to make use of most features, even without the use of
 special equipment such as a console server.
 
-First install conmux and cu:
+First install conmux and cu::
+
     sudo add-apt-repository ppa:linaro-maintainers/tools
     sudo apt-get update
     sudo apt-get install conmux cu
@@ -26,92 +32,110 @@ 
 You may have permission problem with cu running as root under conmux.
 
 Create a configuration file for your board under /etc/conmux which
-should look something like this:
+should look something like this::
+
     listener panda01
     application console 'panda01 console' 'cu -l /dev/ttyUSB0 -s 115200'
+
 Make sure to give the file a '.cf' extension (e.g. panda01.cf).
 
 If you see this permission problem when running cu, you can try
 adjusting your .cf file to call cu using sg, and the group name owning
-the device.  For example:
+the device.  For example::
+
     sg dialout "cu -l ttyUSB0 -s 115200"
 
-Finally restart conmux:
+Finally restart conmux::
+
     sudo stop conmux
     sudo start conmux
 
-You can test the connection using:
+You can test the connection using::
+
     conmux-console panda01
     (use ~$quit to exit)
 
 You should be able to type commands and interact with the shell inside
 conmux-console.  If you cannot, run "sudo stop conmux" and try running
 'sg dialout "cu -l ttyUSB0 -s 115200"'.  If that doesn't work, you
-probably need to add some files to /etc/uucp.  Add
-
-port ttyUSB0
-type direct
-device /dev/ttyUSB0
-hardflow false
-speed 115200
-
-to /etc/uucp/port and append
-
-system  panda01
-port    ttyUSB0
-time    any
+probably need to add some files to /etc/uucp.  Add ::
+
+    port ttyUSB0
+    type direct
+    device /dev/ttyUSB0
+    hardflow false
+    speed 115200
+
+to /etc/uucp/port and append ::
+
+    system  panda01
+    port    ttyUSB0
+    time    any
 
 to /etc/uucp/sys.  If this doesn't let you interact with the shell in
 conmux-console, complain in #linaro on freenode and hopefully someone
 will help you out :)
 
-Make a directory which is defined as LAVA_IMAGE_TMPDIR in 
-lava/dispatcher/config.py:
-mkdir -p /linaro/images/tmp
-
-Define LAVA_SERVER_IP to your local LAVA server if you want to use local one in
-config.py.
-
-Setup a web server(like Apache), and make root directory as /linaro. 
-
-Setup a dashboard server launch-control:
-add-apt-repository ppa:zkrynicki/lava
-apt-get update
-apt-get purge launch-control (if there is an old one)
-apt-get install launch-control-pqsql
-apt-get install launch-control
-sudo -u www-data python /usr/lib/launch-control/manage.py createsuperuser
+
+Configuring the dispatcher
+++++++++++++++++++++++++++
+
+The dispatcher looks for config values in ~/.config/lava-dispatcher/,
+then /etc/xdg/lava-dispatcher, then its own source tree.
+
+To get started, you ned to make a directory for storing artefacts
+before they are downloaded to the device being tested
+(LAVA_IMAGE_TMPDIR in the config and /linaro/images/tmp by default)::
+
+    # mkdir -p /linaro/images/tmp
+
+You will need to set LAVA_SERVER_IP to the address of the machine
+running the dispatcher in ~/.config/lava-dispatcher/lava-dispatcher.conf::
+
+    $ cat ~/.config/lava-dispatcher/lava-dispatcher.conf
+    LAVA_SERVER_IP = 192.168.88.77
+
+You will need to add a configuration file for your device. It can be
+extremely simple, just identifying the type of the device::
+
+    $ cat ~/.config/lava-dispatcher/devices/panda01.conf
+    device_type = panda
+
+You also need a webserver such as Apache set up, and serve
+LAVA_IMAGE_TMPDIR/images as /images.  This snippet does this for
+Apache::
+
+    Alias /images/ "/linaro/images/"
+    <Directory "/linaro/images/">
+        Options Indexes MultiViews FollowSymLinks
+    </Directory>
+
+If you want to upload the results to a local dashboard instance, you
+need to set one of those up.  XXX link to doc on this!
 
 Modify the server in job file doc/lava-ltp-job.json to local launch-control
 url, like:
-http://127.0.0.1/launch-control
+
     {
       "command": "submit_results",
       "parameters":
         {
-          "server": "http://dashboard.linaro.org",
+          "server": "http://staging.linaro.dev/lava-server",
           "stream": "panda01-ltp"
         }
     }
 
 
-Logging:
-Conmux keeps logs in /var/log/conmux.  If it cannot attach to the
-device, it may continually send output to the log every few seconds.
-For non-production purposes, you may simply want to run 'sudo stop
-conmux' when you are not using the console, to avoid needlessly growing
-large logfiles.
+A note on logging: Conmux keeps logs in /var/log/conmux.  If it cannot
+attach to the device, it may continually send output to the log every
+few seconds.  For non-production purposes, you may simply want to run
+'sudo stop conmux' when you are not using the console, to avoid
+needlessly growing large logfiles.
 
 
 Dispatching a job
 -----------------
 
-You will want to edit lava/dispatcher/config.py to change the value of
-LAVA_SERVER_IP to, at least, a machine that exists on your network.
-
-There are other things that you need to edit in here XXX explain some
-of this stuff.
-
 From the toplevel, run (as root):
 
     ./lava-dispatch doc/lava-ltp-job.json

=== modified file 'lava_dispatcher/__init__.py'
--- lava_dispatcher/__init__.py	2011-09-08 22:12:27 +0000
+++ lava_dispatcher/__init__.py	2011-09-09 04:34:34 +0000
@@ -27,16 +27,19 @@ 
 import pexpect
 
 from lava_dispatcher.actions import get_all_cmds
+from lava_dispatcher.config import get_config, get_device_config
 from lava_dispatcher.client import LavaClient, CriticalError, GeneralError
 from lava_dispatcher.android_client import LavaAndroidClient
 
-__version__ = "0.2.2"
+__version__ = "0.3a1"
 
 class LavaTestJob(object):
     def __init__(self, job_json, oob_file):
         self.job_status = 'pass'
         self.load_job_data(job_json)
-        self.context = LavaContext(self.target, self.image_type, oob_file)
+        dispatcher_config = get_config("lava-dispatcher")
+        self.context = LavaContext(
+            self.target, self.image_type, dispatcher_config, oob_file)
 
     def load_job_data(self, job_json):
         self.job_data = json.loads(job_json)
@@ -76,6 +79,7 @@ 
                 else:
                     status = 'pass'
                 finally:
+                    err_msg = ""
                     if status == 'fail':
                         err_msg = "Lava failed at action %s with error: %s\n" %\
                                   (cmd['command'], err)
@@ -102,11 +106,17 @@ 
 
 
 class LavaContext(object):
-    def __init__(self, target, image_type, oob_file):
-        if image_type != "android":
-            self._client = LavaClient(target)
+    def __init__(self, target, image_type, dispatcher_config, oob_file):
+        self.config = dispatcher_config
+        device_config = get_device_config(target)
+        if device_config.get('client_type') != 'conmux':
+            raise RuntimeError(
+                "this version of lava-dispatcher only supports conmux "
+                "clients, not %r" % device_config.get('client_type'))
+        if image_type == "android":
+            self._client = LavaAndroidClient(self, device_config)
         else:
-            self._client = LavaAndroidClient(target)
+            self._client = LavaClient(self, device_config)
         self.test_data = LavaTestData()
         self.oob_file = oob_file
 
@@ -114,6 +124,26 @@ 
     def client(self):
         return self._client
 
+    @property
+    def lava_server_ip(self):
+        return self.config.get("LAVA_SERVER_IP")
+
+    @property
+    def lava_image_tmpdir(self):
+        return self.config.get("LAVA_IMAGE_TMPDIR")
+
+    @property
+    def lava_image_url(self):
+        return self.config.get("LAVA_IMAGE_URL")
+
+    @property
+    def lava_result_dir(self):
+        return self.config.get("LAVA_RESULT_DIR")
+
+    @property
+    def lava_cachedir(self):
+        return self.config.get("LAVA_CACHEDIR")
+
 
 class LavaTestData(object):
     def __init__(self, test_id='lava'):

=== modified file 'lava_dispatcher/actions/android_basic.py'
--- lava_dispatcher/actions/android_basic.py	2011-09-02 03:00:59 +0000
+++ lava_dispatcher/actions/android_basic.py	2011-09-05 22:39:47 +0000
@@ -60,7 +60,7 @@ 
             test_case_result['result'] = "fail"
 
         results['test_results'].append(test_case_result)
-        savebundlefile("monkey", results, timestring)
+        savebundlefile("monkey", results, timestring, self.context.lava_result_dir)
         self.client.proc.sendline("")
 
 
@@ -146,5 +146,5 @@ 
             test_case_result['result'] = "fail"
 
         results['test_results'].append(test_case_result)
-        savebundlefile("basic", results, timestring)
+        savebundlefile("basic", results, timestring, self.context.lava_result_dir)
         self.client.proc.sendline("")

=== modified file 'lava_dispatcher/actions/android_deploy.py'
--- lava_dispatcher/actions/android_deploy.py	2011-09-05 21:52:35 +0000
+++ lava_dispatcher/actions/android_deploy.py	2011-09-05 22:39:47 +0000
@@ -20,7 +20,6 @@ 
 # along with this program; if not, see <http://www.gnu.org/licenses>.
 
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL
 import os
 import sys
 import shutil
@@ -31,6 +30,8 @@ 
 
 class cmd_deploy_linaro_android_image(BaseAction):
     def run(self, boot, system, data, use_cache=True):
+        LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
+        LAVA_IMAGE_URL = self.context.lava_image_url
         client = self.client
         print "deploying Android on %s" % client.hostname
         print "  boot: %s" % boot
@@ -85,14 +86,16 @@ 
         :param data_url: url of the Linaro Android data tarball to download
         :param use_cache: whether or not to use the cached copy (if it exists)
         """
+        lava_cachedir = self.context.lava_cachedir
+        LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
         self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR)
         tarball_dir = self.tarball_dir
         os.chmod(tarball_dir, 0755)
 
         if use_cache:
-            boot_path = download_with_cache(boot_url, tarball_dir)
-            system_path = download_with_cache(system_url, tarball_dir)
-            data_path = download_with_cache(data_url, tarball_dir)
+            boot_path = download_with_cache(boot_url, tarball_dir, lava_cachedir)
+            system_path = download_with_cache(system_url, tarball_dir, lava_cachedir)
+            data_path = download_with_cache(data_url, tarball_dir, lava_cachedir)
         else:
             boot_path = download(boot_url, tarball_dir)
             system_path = download(system_url, tarball_dir)

=== modified file 'lava_dispatcher/actions/deploy.py'
--- lava_dispatcher/actions/deploy.py	2011-09-08 04:23:59 +0000
+++ lava_dispatcher/actions/deploy.py	2011-09-08 23:35:05 +0000
@@ -26,13 +26,14 @@ 
 from tempfile import mkdtemp
 
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL
 from lava_dispatcher.utils import download, download_with_cache
 from lava_dispatcher.client import CriticalError
 
 
 class cmd_deploy_linaro_image(BaseAction):
     def run(self, hwpack, rootfs, use_cache=True):
+        LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
+        LAVA_IMAGE_URL = self.context.lava_image_url
         client = self.client
         print "deploying on %s" % client.hostname
         print "  hwpack: %s" % hwpack
@@ -112,30 +113,31 @@ 
         :param hwpack_url: url of the Linaro hwpack to download
         :param rootfs_url: url of the Linaro image to download
         """
+        lava_cachedir = self.context.lava_cachedir
+        LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
         client = self.client
         self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR)
         tarball_dir = self.tarball_dir
         os.chmod(tarball_dir, 0755)
         if use_cache:
-            hwpack_path = download_with_cache(hwpack_url, tarball_dir)
-            rootfs_path = download_with_cache(rootfs_url, tarball_dir)
+            hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir)
+            rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir)
         else:
             hwpack_path = download(hwpack_url, tarball_dir)
             rootfs_path = download(rootfs_url, tarball_dir)
 
         image_file = os.path.join(tarball_dir, "lava.img")
-        board = client.board
         cmd = ("sudo linaro-media-create --hwpack-force-yes --dev %s "
                "--image_file %s --binary %s --hwpack %s --image_size 3G" %
-               (board.type, image_file, rootfs_path, hwpack_path))
+               (client.device_type, image_file, rootfs_path, hwpack_path))
         rc, output = getstatusoutput(cmd)
         if rc:
             shutil.rmtree(tarball_dir)
             tb = traceback.format_exc()
             client.sio.write(tb)
             raise RuntimeError("linaro-media-create failed: %s" % output)
-        boot_offset = self._get_partition_offset(image_file, board.boot_part)
-        root_offset = self._get_partition_offset(image_file, board.root_part)
+        boot_offset = self._get_partition_offset(image_file, client.boot_part)
+        root_offset = self._get_partition_offset(image_file, client.root_part)
         boot_tgz = os.path.join(tarball_dir, "boot.tgz")
         root_tgz = os.path.join(tarball_dir, "root.tgz")
         try:
@@ -159,7 +161,7 @@ 
         client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root')
         client.run_cmd_master(
             'wget -qO- %s |tar --numeric-owner -C /mnt/root -xzf -' % rootfs,
-            timeout = 3600)
+            timeout=3600)
         client.run_cmd_master('echo linaro > /mnt/root/etc/hostname')
         #DO NOT REMOVE - diverting flash-kernel and linking it to /bin/true
         #prevents a serious problem where packages getting installed that

=== modified file 'lava_dispatcher/actions/launch_control.py' (properties changed: +x to -x)
--- lava_dispatcher/actions/launch_control.py	2011-09-06 09:35:13 +0000
+++ lava_dispatcher/actions/launch_control.py	2011-09-08 04:28:39 +0000
@@ -25,8 +25,6 @@ 
 import shutil
 import tarfile
 from lava_dispatcher.actions import BaseAction
-from lava_dispatcher.config import LAVA_RESULT_DIR
-from lava_dispatcher.config import LAVA_IMAGE_TMPDIR
 from lava_dispatcher.client import NetworkError
 from lava_dispatcher.utils import download
 from tempfile import mkdtemp
@@ -40,9 +38,9 @@ 
                 allow_none=True, use_datetime=True)
 
         #Upload bundle files to dashboard
-        bundle_list = os.listdir("/tmp/%s" % LAVA_RESULT_DIR)
+        bundle_list = os.listdir("/tmp/%s" % self.context.lava_result_dir)
         for bundle_name in bundle_list:
-            bundle = "/tmp/%s/%s" % (LAVA_RESULT_DIR, bundle_name)
+            bundle = "/tmp/%s/%s" % (self.context.lava_result_dir, bundle_name)
             f = open(bundle)
             content = f.read()
             f.close()
@@ -53,6 +51,7 @@ 
                 print "xmlrpclib.Fault occurred"
                 print "Fault code: %d" % err.faultCode
                 print "Fault string: %s" % err.faultString
+
             # After uploading, remove the bundle file at the host side
             os.remove(bundle)
 
@@ -79,49 +78,50 @@ 
         client.run_cmd_master('mkdir -p /mnt/root')
         client.run_cmd_master(
             'mount /dev/disk/by-label/%s /mnt/root' % result_disk)
-        client.run_cmd_master('mkdir -p /tmp/%s' % LAVA_RESULT_DIR)
+        client.run_cmd_master('mkdir -p /tmp/%s' % self.context.lava_result_dir)
         client.run_cmd_master(
-            'cp /mnt/root/%s/*.bundle /tmp/%s' % (LAVA_RESULT_DIR,
-                LAVA_RESULT_DIR))
+            'cp /mnt/root/%s/*.bundle /tmp/%s' % (self.context.lava_result_dir,
+                self.context.lava_result_dir))
         client.run_cmd_master('umount /mnt/root')
 
         #Create tarball of all results
         client.run_cmd_master('cd /tmp')
         client.run_cmd_master(
-            'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % LAVA_RESULT_DIR)
+            'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir)
 
         master_ip = client.get_master_ip()
-        if master_ip != None:
-            # Set 80 as server port
-            client.run_cmd_master('python -m SimpleHTTPServer 80 &> /dev/null &')
-            time.sleep(3)
-
-            result_tarball = "http://%s/lava_results.tgz" % master_ip
-            tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR)
-            os.chmod(tarball_dir, 0755)
-
-            # download test result with a retry mechanism
-            # set retry timeout to 2mins
-            now = time.time()
-            timeout = 120
-            while time.time() < now+timeout:
-                try:
-                    result_path = download(result_tarball, tarball_dir)
-                except:
-                    if time.time() >= now+timeout:
-                        raise
-
-            client.run_cmd_master('kill %1')
-
-            tar = tarfile.open(result_path)
-            for tarinfo in tar:
-                if os.path.splitext(tarinfo.name)[1] == ".bundle":
-                    f = tar.extractfile(tarinfo)
-                    content = f.read()
-                    f.close()
-                    self.all_bundles.append(json.loads(content))
-            tar.close()
-            shutil.rmtree(tarball_dir)
+        if master_ip == None:
+            raise NetworkError("Getting master image IP address failed")
+        # Set 80 as server port
+        client.run_cmd_master('python -m SimpleHTTPServer 80 &> /dev/null &')
+        time.sleep(3)
+
+        result_tarball = "http://%s/lava_results.tgz" % master_ip
+        tarball_dir = mkdtemp(dir=self.context.lava_image_tmpdir)
+        os.chmod(tarball_dir, 0755)
+
+        # download test result with a retry mechanism
+        # set retry timeout to 2mins
+        now = time.time()
+        timeout = 120
+        while time.time() < now+timeout:
+            try:
+                result_path = download(result_tarball, tarball_dir)
+            except:
+                if time.time() >= now+timeout:
+                    raise
+
+        client.run_cmd_master('kill %1')
+
+        tar = tarfile.open(result_path)
+        for tarinfo in tar:
+            if os.path.splitext(tarinfo.name)[1] == ".bundle":
+                f = tar.extractfile(tarinfo)
+                content = f.read()
+                f.close()
+                self.all_bundles.append(json.loads(content))
+        tar.close()
+        shutil.rmtree(tarball_dir)
 
         #flush the serial log
         client.run_shell_command("")

=== modified file 'lava_dispatcher/actions/lava-test.py' (properties changed: +x to -x)
--- lava_dispatcher/actions/lava-test.py	2011-09-08 04:23:59 +0000
+++ lava_dispatcher/actions/lava-test.py	2011-09-08 04:28:39 +0000
@@ -24,7 +24,7 @@ 
 import traceback
 from lava_dispatcher.actions import BaseAction
 from lava_dispatcher.client import OperationFailed
-from lava_dispatcher.config import LAVA_RESULT_DIR, MASTER_STR
+
 
 
 def _setup_testrootfs(client):
@@ -50,10 +50,9 @@ 
         'cp -f /mnt/root/etc/resolv.conf.bak /mnt/root/etc/resolv.conf')
     cmd = ('cat /proc/mounts | awk \'{print $2}\' | grep "^/mnt/root/dev"'
         '| sort -r | xargs umount')
-    client.run_cmd_master(
-        cmd)
-    client.run_cmd_master(
-        'umount /mnt/root')
+    client.run_cmd_master(cmd)
+    client.run_cmd_master('umount /mnt/root')
+
 
 
 def _install_lava_test(client):
@@ -73,11 +72,10 @@ 
         client.run_shell_command(
             'chroot /mnt/root lava-test help',
             response="list-test", timeout=10)
-        client.proc.expect(MASTER_STR, timeout=10)
+        client.proc.expect(client.master_str, timeout=10)
     except:
         tb = traceback.format_exc()
         client.sio.write(tb)
-        _teardown_testrootfs(client)
         raise OperationFailed("lava-test deployment failed")
 
 
@@ -86,12 +84,12 @@ 
         #Make sure in test image now
         client = self.client
         client.in_test_shell()
-        client.run_cmd_tester('mkdir -p %s' % LAVA_RESULT_DIR)
+        client.run_cmd_tester('mkdir -p %s' % self.context.lava_result_dir)
         client.export_display()
         bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S")
         client.run_cmd_tester(
             'lava-test run %s -o %s/%s.bundle' % (
-                test_name, LAVA_RESULT_DIR, bundle_name),
+                test_name, self.context.lava_result_dir, bundle_name),
             timeout=timeout)
 
 

=== modified file 'lava_dispatcher/android_client.py'
--- lava_dispatcher/android_client.py	2011-09-08 04:23:59 +0000
+++ lava_dispatcher/android_client.py	2011-09-08 04:28:39 +0000
@@ -20,12 +20,9 @@ 
 import pexpect
 import sys
 from lava_dispatcher.client import LavaClient, OperationFailed
-from lava_dispatcher.android_config import BOARDS, TESTER_STR
+from utils import string_to_list
 
 class LavaAndroidClient(LavaClient):
-    def __init__(self, hostname):
-        super(LavaAndroidClient, self).__init__(hostname)
-        self.board = BOARDS[hostname]
 
     def run_adb_shell_command(self, dev_id, cmd, response, timeout=-1):
         adb_cmd = "adb -s %s shell %s" % (dev_id, cmd)
@@ -42,7 +39,7 @@ 
         """ Check that we are in a shell on the test image
         """
         self.proc.sendline("")
-        id = self.proc.expect([TESTER_STR , pexpect.TIMEOUT])
+        id = self.proc.expect([self.tester_str , pexpect.TIMEOUT])
         if id == 1:
             raise OperationFailed
 
@@ -55,11 +52,11 @@ 
         except:
             self.hard_reboot()
             self.enter_uboot()
-        uboot_cmds = self.board.uboot_cmds
-        self.proc.sendline(uboot_cmds[0])
-        for line in range(1, len(uboot_cmds)):
+        boot_cmds = string_to_list(self.config.get('boot_cmds_android'))
+        self.proc.sendline(boot_cmds[0])
+        for line in range(1, len(boot_cmds)):
             self.proc.expect("#")
-            self.proc.sendline(uboot_cmds[line])
+            self.proc.sendline(boot_cmds[line])
         self.in_test_shell()
         self.proc.sendline("export PS1=\"root@linaro: \"")
 
@@ -106,7 +103,7 @@ 
 
     def check_adb_status(self):
         # XXX: IP could be assigned in other way in the validation farm
-        network_interface = self.board.default_network_interface 
+        network_interface = self.default_network_interface
         try:
             self.run_cmd_tester(
                 'netcfg %s dhcp' % network_interface, timeout=60)

=== removed file 'lava_dispatcher/android_config.py'
--- lava_dispatcher/android_config.py	2011-08-16 00:15:08 +0000
+++ lava_dispatcher/android_config.py	1970-01-01 00:00:00 +0000
@@ -1,65 +0,0 @@ 
-# Copyright (C) 2011 Linaro Limited
-#
-# Author: Linaro Validation Team <linaro-dev@lists.linaro.org>
-#
-# This file is part of LAVA Dispatcher.
-#
-# LAVA Dispatcher is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# LAVA Dispatcher is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, see <http://www.gnu.org/licenses>.
-
-from lava_dispatcher.config import Board
-
-class BeagleBoard(Board):
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage;"
-        "fatload mmc 0:3 0x81600000 uInitrd;"
-        "bootm 0x80000000 0x81600000'",
-        "setenv bootargs 'console=tty0 console=ttyO2,115200n8 "
-        "rootwait rw earlyprintk fixrtc nocompcache "
-        "vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 "
-        "init=/init androidboot.console=ttyO2'",
-        "boot"]
-    type = "beagle"
-
-
-class PandaBoard(Board):
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:3 0x80200000 uImage;"
-        "fatload mmc 0:3 0x81600000 uInitrd;"
-        "bootm 0x80200000 0x81600000'",
-        "setenv bootargs 'console=tty0 console=ttyO2,115200n8 "
-        "rootwait rw earlyprintk fixrtc nocompcache vram=32M "
-        "omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000 "
-        "omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 "
-        "init=/init androidboot.console=ttyO2'",
-        "boot"]
-    type = "panda"
-
-
-#Here, it still needs to maintain a map from boardid to board, for there is only
-#boardid in jobfile.json
-BOARDS = {
-        "beagle01": BeagleBoard,
-        "beagle02": BeagleBoard,
-        "beagle03": BeagleBoard,
-        "beagle04": BeagleBoard,
-        "panda01": PandaBoard,
-        "panda02": PandaBoard,
-        "panda03": PandaBoard,
-        "panda04": PandaBoard,
-        }
-
-#Test image recognization string
-TESTER_STR = "root@linaro:"

=== modified file 'lava_dispatcher/android_util.py'
--- lava_dispatcher/android_util.py	2011-06-27 04:55:08 +0000
+++ lava_dispatcher/android_util.py	2011-08-10 18:01:25 +0000
@@ -22,11 +22,10 @@ 
 from datetime import datetime
 import json
 import subprocess
-from lava_dispatcher.config import LAVA_RESULT_DIR
 import time
 
 # TODO: Result saving could be replaced by linaro_dashboard_bundle probably.
-def savebundlefile(testname, results, starttime):
+def savebundlefile(testname, results, starttime, lava_result_dir):
     """
     Save results as .bundle file under /tmp/LAVA_RESULT_DIR/
     """
@@ -42,9 +41,9 @@ 
     testdata['test_runs'] = test_runs
     testdata['test_runs'][0].update(results)
     bundle = testdata
-    subprocess.call(["mkdir", "-p", "/tmp/%s" % LAVA_RESULT_DIR])
+    subprocess.call(["mkdir", "-p", "/tmp/%s" % lava_result_dir])
     # The file name should be unique to be distinguishable from others
-    filename = "/tmp/%s/" % LAVA_RESULT_DIR + testname + \
+    filename = "/tmp/%s/" % lava_result_dir + testname + \
         str(time.mktime(datetime.utcnow().timetuple())) + ".bundle"
     with open(filename, "wt") as stream:
         json.dump(bundle, stream)

=== modified file 'lava_dispatcher/client.py'
--- lava_dispatcher/client.py	2011-09-08 04:23:59 +0000
+++ lava_dispatcher/client.py	2011-09-08 23:35:05 +0000
@@ -22,35 +22,56 @@ 
 import sys
 import time
 from cStringIO import StringIO
-
-from lava_dispatcher.config import (
-    BOARDS,
-    LAVA_SERVER_IP,
-    MASTER_STR,
-    TESTER_STR,
-    )
-
+from utils import string_to_list
 
 class LavaClient(object):
-    def __init__(self, hostname):
-        self._master_str = MASTER_STR
-        self._tester_str = TESTER_STR
-        cmd = "conmux-console %s" % hostname
+    def __init__(self, context, config):
+        self.context = context
+        self.config = config
+        cmd = "conmux-console %s" % self.hostname
         self.sio = SerialIO(sys.stdout)
         self.proc = pexpect.spawn(cmd, timeout=3600, logfile=self.sio)
         #serial can be slow, races do funny things if you don't increase delay
         self.proc.delaybeforesend=1
-        self.hostname = hostname
-        # will eventually come from the database
-        self.board = BOARDS[hostname]
+
+    def device_option(self, option_name):
+        return self.config.get(option_name)
+
+    def device_option_int(self, option_name):
+        return self.config.getint(option_name)
+
+    @property
+    def hostname(self):
+        return self.device_option("hostname")
+
+    @property
+    def tester_str(self):
+        return self.device_option("TESTER_STR")
 
     @property
     def master_str(self):
-        return self._master_str
-
-    @property
-    def tester_str(self):
-        return self._tester_str
+        return self.device_option("MASTER_STR")
+
+    @property
+    def boot_cmds(self):
+        uboot_str = self.device_option("boot_cmds")
+        return string_to_list(uboot_str)
+
+    @property
+    def device_type(self):
+        return self.device_option("device_type")
+
+    @property
+    def boot_part(self):
+        return self.device_option_int("boot_part")
+
+    @property
+    def root_part(self):
+        return self.device_option_int("root_part")
+
+    @property
+    def default_network_interface(self):
+        return self.device_option("default_network_interface")
 
     def in_master_shell(self):
         """ Check that we are in a shell on the master image
@@ -91,16 +112,16 @@ 
         except:
             self.hard_reboot()
             self.enter_uboot()
-        uboot_cmds = self.board.uboot_cmds
-        self.proc.sendline(uboot_cmds[0])
-        for line in range(1, len(uboot_cmds)):
-            if self.board.type in ["mx51evk", "mx53loco"]:
+        boot_cmds = self.boot_cmds
+        self.proc.sendline(boot_cmds[0])
+        for line in range(1, len(boot_cmds)):
+            if self.device_type in ["mx51evk", "mx53loco"]:
                 self.proc.expect(">", timeout=300)
-            elif self.board.type == "snowball_sd":
+            elif self.device_type == "snowball_sd":
                 self.proc.expect("\$", timeout=300)
             else:
                 self.proc.expect("#", timeout=300)
-            self.proc.sendline(uboot_cmds[line])
+            self.proc.sendline(boot_cmds[line])
         self.in_test_shell()
 
     def enter_uboot(self):
@@ -131,7 +152,8 @@ 
         self.run_shell_command(cmd, self.tester_str, timeout)
 
     def check_network_up(self):
-        self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % LAVA_SERVER_IP)
+        lava_server_ip = self.context.lava_server_ip
+        self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % lava_server_ip)
         id = self.proc.expect(["1 received", "0 received",
             "Network is unreachable"], timeout=5)
         self.proc.expect(self.master_str)
@@ -154,7 +176,7 @@ 
         #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
         pattern1 = "(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)"
         cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |"
-                "awk '{print $1}'" % self.board.default_network_interface)
+                "awk '{print $1}'" % self.default_network_interface)
         self.proc.sendline(cmd)
         #if running from ipython, it needs another Enter, don't know why:
         #self.proc.sendline("")

=== modified file 'lava_dispatcher/config.py'
--- lava_dispatcher/config.py	2011-08-16 00:15:08 +0000
+++ lava_dispatcher/config.py	2011-09-09 00:52:17 +0000
@@ -18,113 +18,72 @@ 
 # along
 # with this program; if not, see <http://www.gnu.org/licenses>.
 
-"""
-This is an ugly hack, the uboot commands for a given board type and the board
-type of a test machine need to come from the device registry.  This is an
-easy way to look it up for now though, just to show the rest of the code
-around it
-"""
-
-class Board:
-    uboot_cmds = None
-    type = None
-    # boot partition number, counting from 1
-    boot_part = 1
-    # root partition number, counting from 1
-    root_part = 2
-    default_network_interface = "eth0"
-
-class BeagleBoard(Board):
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage; fatload mmc "
-        "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'",
-        "setenv bootargs ' console=tty0 console=ttyO2,115200n8 "
-        "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache "
-        "vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60'",
-        "boot"]
-    type = "beagle"
-
-class PandaBoard(Board):
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:3 0x80200000 uImage; fatload mmc "
-        "0:3 0x81600000 uInitrd; bootm 0x80200000 0x81600000'",
-        "setenv bootargs ' console=tty0 console=ttyO2,115200n8 "
-        "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache "
-        "vram=48M omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000'",
-        "boot"]
-    type = "panda"
-
-class Snowball(Board):
-    uboot_cmds = ["mmc init",
-        "mmc rescan 1",
-        "setenv bootcmd 'fat load mmc 1:3 0x00100000 /uImage;"
-        "bootm 0x00100000'",
-        "setenv bootargs 'console=tty0 console=ttyAMA2,115200n8 "
-        "root=LABEL=testrootfs rootwait ro earlyprintk rootdelay=1 "
-        "fixrtc nocompcache mem=96M@0 mem_modem=32M@96M mem=44M@128M "
-        "pmem=22M@172M mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M "
-        "hwmem=48M@302M mem=152M@360M'",
-        "boot"]
-    type = "snowball_sd"
-
-class Mx51evkBoard(Board):
-    boot_part = 2
-    root_part = 3
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:5 0x90000000 uImage; fatload mmc 0:5 "
-        "0x92000000 uInitrd; fatload mmc 0:5 0x91ff0000 board.dtb; bootm "
-        "0x90000000 0x92000000 0x91ff0000'",
-        "setenv bootargs ' console=tty0 console=ttymxc0,115200n8 "
-        "root=LABEL=testrootfs rootwait ro'",
-        "boot"]
-    type = "mx51evk"
-
-class Mx53locoBoard(Board):
-    boot_part = 2
-    root_part = 3
-    uboot_cmds = ["mmc init",
-        "mmc part 0",
-        "setenv bootcmd 'fatload mmc 0:5 0x70800000 uImage; fatload mmc "
-        "0:5 0x71800000 uInitrd; bootm 0x70800000 0x71800000'",
-        "setenv bootargs ' console=tty0 console=ttymxc0,115200n8 "
-        "root=LABEL=testrootfs rootwait ro'",
-        "boot"]
-    type = "mx53loco"
-
-#Here, it still needs to maintain a map from boardid to board, for there is
-#only boardid in jobfile.json
-BOARDS = {
-        "panda01": PandaBoard,
-        "panda02": PandaBoard,
-        "panda03": PandaBoard,
-        "panda04": PandaBoard,
-        "beaglexm01": BeagleBoard,
-        "beaglexm02": BeagleBoard,
-        "beaglexm03": BeagleBoard,
-        "beaglexm04": BeagleBoard,
-        "mx51evk01": Mx51evkBoard,
-        "mx53loco01": Mx53locoBoard,
-        "snowball01": Snowball,
-        "snowball02": Snowball,
-        "snowball03": Snowball,
-        "snowball04": Snowball,
-        }
-
-#Main LAVA server IP in the boards farm
-LAVA_SERVER_IP = "192.168.1.10"
-#Location for hosting rootfs/boot tarballs extracted from images
-LAVA_IMAGE_TMPDIR = "/linaro/images/tmp"
-#URL where LAVA_IMAGE_TMPDIR can be accessed remotely
-LAVA_IMAGE_URL = "http://%s/images/tmp" % LAVA_SERVER_IP
-#Default test result storage path
-LAVA_RESULT_DIR = "/lava/results"
-#Location for caching downloaded artifacts such as hwpacks and images
-LAVA_CACHEDIR = "/linaro/images/cache"
-
-#Master image recognization string
-MASTER_STR = "root@master:"
-#Test image recognization string
-TESTER_STR = "root@linaro:"
+from ConfigParser import ConfigParser
+import os
+import StringIO
+
+
+default_config_path = os.path.join(
+    os.path.dirname(__file__), 'default-config')
+
+
+def load_config_paths(name):
+    for directory in [os.path.expanduser("~/.config"),
+                      "/etc/xdg", default_config_path]:
+        path = os.path.join(directory, name)
+        if os.path.isdir(path):
+            yield path
+
+
+def _read_into(path, cp):
+    s = StringIO.StringIO()
+    s.write('[DEFAULT]\n')
+    s.write(open(path).read())
+    s.seek(0)
+    cp.readfp(s)
+
+
+def _get_config(name, cp=None):
+    """Read a config file named name + '.conf'.
+
+    This checks and loads files from the source tree, site wide location and
+    home directory -- in that order, so home dir settings override site
+    settings which override source settings.
+    """
+    config_files = []
+    for directory in load_config_paths('lava-dispatcher'):
+        path = os.path.join(directory, '%s.conf' % name)
+        if os.path.exists(path):
+            config_files.append(path)
+    if not config_files:
+        raise Exception("no config files named %r found" % (name + ".conf"))
+    config_files.reverse()
+    if cp is None:
+        cp = ConfigParser()
+    print "About to read %s" % str(config_files)
+    for path in config_files:
+        _read_into(path, cp)
+    return cp
+
+
+class ConfigWrapper(object):
+    def __init__(self, cp):
+        self.cp = cp
+    def get(self, key):
+        return self.cp.get("DEFAULT", key)
+    def getint(self, key):
+        return self.cp.getint("DEFAULT", key)
+
+
+def get_config(name):
+    return ConfigWrapper(_get_config(name))
+
+
+def get_device_config(name):
+    device_config = _get_config("devices/%s" % name)
+    cp = _get_config("device-defaults")
+    _get_config(
+        "device-types/%s" % device_config.get('DEFAULT', 'device_type'), cp)
+    _get_config("devices/%s" % name, cp)
+    cp.set("DEFAULT", "hostname", name)
+    return ConfigWrapper(cp)

=== added directory 'lava_dispatcher/default-config'
=== added directory 'lava_dispatcher/default-config/lava-dispatcher'
=== added file 'lava_dispatcher/default-config/lava-dispatcher/README'
--- lava_dispatcher/default-config/lava-dispatcher/README	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/README	2011-09-08 23:35:05 +0000
@@ -0,0 +1,41 @@ 
+Configuration files for lava-dispatcher
+=======================================
+
+lava-dispatcher looks for files in:
+
+ * Alongside the installation/source tree for the default values
+   (i.e. this directory).
+
+ * /etc/xdg/lava-dispatcher for system-wide settings.
+
+ * ~/.config/lava-dispatcher for user settings.
+
+Each config directory can contain two files and two directories:
+
+ * lava-dispatcher.conf
+
+   This file defines global settings of the dispatcher.  You will
+   almost certainly need to customize LAVA_SERVER_IP for your install.
+
+ * device-defaults.conf
+
+   This file defines default values for all devices.  You probably
+   won't need to customize it.
+
+ * device-types/
+
+   This directory contains a config file for each device type.  You
+   probably won't need to customize the settings for device types that
+   are already supported by lava-dispatcher, but if you are working on
+   supporting a new class of device, you will need to add a file here.
+
+   Note that the device-type name must match the --dev argument to
+   linaro-media-create.
+
+ * devices/
+
+   This directory contains a file per device that can be targeted by
+   lava-dispatcher.  For the most part this file just needs to contain
+   a line "device_type = <device type>", although other settings can
+   be included here.  You will definitely need to tell lava-dispatcher
+   about the devices you have!

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf	2011-09-09 04:34:34 +0000
@@ -0,0 +1,50 @@ 
+# The default device settings.
+
+# All device settings default to these values unless they are
+# overwritten by the specific device type file
+# (device-types/${TYPE}.conf) or the specific device file
+# (devices/${DEVICE}.conf).
+
+# The client_type.  Only 'conmux' (meaning we communicate with the
+# device over a serial line via conmux) is supported today but 'qemu'
+# and 'ssh' are coming.
+client_type = conmux
+
+# The bootloader commands to boot the device into the test image (we
+# assume that the device boots into the master image without bootloader
+# intervention).
+#
+# XXX should be called # boot_test_image_commands ?
+boot_cmds =
+
+# The bootloader commands to boot the device into an android-based test
+# image.
+#
+# XXX should be called # boot_android_test_image_commands ?
+boot_cmds_android =
+
+# The device type.  Settings in device-types/${TYPE}.conf override
+# settings in this file, but are overridden by the
+# devices/${DEVICE}.conf file.
+type =
+
+# The network interface that comes up by default
+default_network_interface = eth0
+
+# boot partition number, counting from 1
+#
+# This is used to divide up the image produced by linaro-media-create
+# into sections to write onto the device.
+boot_part = 1
+
+# root partition number, counting from 1
+#
+# This is used to divide up the image produced by linaro-media-create
+# into sections to write onto the device.
+root_part = 2
+
+# Master image recognization string
+MASTER_STR = root@master:
+
+# Test image recognization string
+TESTER_STR = root@linaro:

=== added directory 'lava_dispatcher/default-config/lava-dispatcher/device-types'
=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf	2011-09-07 03:46:15 +0000
@@ -0,0 +1,20 @@ 
+boot_cmds = mmc init, 
+	mmc part 0, 
+    setenv bootcmd "'fatload mmc 0:3 0x80000000 uImage; 
+    fatload mmc 0:3 0x81600000 uInitrd; 
+    bootm 0x80000000 0x81600000'",
+    setenv bootargs "' console=tty0 console=ttyO2,115200n8
+    root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache
+    vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60'",
+    boot
+
+boot_cmds_android = mmc init,
+	mmc part 0,
+	setenv bootcmd "'fatload mmc 0:3 0x80000000 uImage;
+	fatload mmc 0:3 0x81600000 uInitrd;
+	bootm 0x80000000 0x81600000'",
+	setenv bootargs "'console=tty0 console=ttyO2,115200n8 
+	rootwait rw earlyprintk fixrtc nocompcache 
+	vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 
+	init=/init androidboot.console=ttyO2'",
+	boot

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf	2011-09-07 03:46:15 +0000
@@ -0,0 +1,10 @@ 
+boot_part = 2
+root_part = 3
+boot_cmds = mmc init,
+    mmc part 0,
+    setenv bootcmd "'fatload mmc 0:5 0x90000000 uImage; fatload mmc 0:5 
+    0x92000000 uInitrd; fatload mmc 0:5 0x91ff0000 board.dtb; bootm 
+    0x90000000 0x92000000 0x91ff0000'",
+    setenv bootargs "' console=tty0 console=ttymxc0,115200n8 
+    root=LABEL=testrootfs rootwait ro'",
+    boot

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf	2011-09-07 03:46:15 +0000
@@ -0,0 +1,9 @@ 
+boot_part = 2
+root_part = 3
+boot_cmds = mmc init,
+    mmc part 0,
+    setenv bootcmd "'fatload mmc 0:5 0x70800000 uImage; fatload mmc 
+    0:5 0x71800000 uInitrd; bootm 0x70800000 0x71800000'",
+    setenv bootargs "' console=tty0 console=ttymxc0,115200n8 
+    root=LABEL=testrootfs rootwait ro'",
+    boot

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf	2011-09-07 03:46:15 +0000
@@ -0,0 +1,19 @@ 
+boot_cmds = mmc init,
+    mmc part 0,
+    setenv bootcmd "'fatload mmc 0:3 0x80200000 uImage; fatload mmc 
+    0:3 0x81600000 uInitrd; bootm 0x80200000 0x81600000'",
+    setenv bootargs "' console=tty0 console=ttyO2,115200n8 
+    root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache 
+    vram=48M omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000'",
+    boot
+boot_cmds_android = mmc init,
+    mmc part 0,
+    setenv bootcmd "'fatload mmc 0:3 0x80200000 uImage;
+    fatload mmc 0:3 0x81600000 uInitrd;
+    bootm 0x80200000 0x81600000'",
+    setenv bootargs "'console=tty0 console=ttyO2,115200n8 
+    rootwait rw earlyprintk fixrtc nocompcache vram=32M 
+    omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000 
+    omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 
+    init=/init androidboot.console=ttyO2'",
+    boot

=== added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf'
--- lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf	2011-09-07 03:46:15 +0000
@@ -0,0 +1,12 @@ 
+boot_cmds = mmc init,
+    mmc rescan 1,
+    setenv bootcmd "'fat load mmc 1:3 0x00100000 /uImage;
+    bootm 0x00100000'",
+    setenv bootargs "'console=tty0 console=ttyAMA2,115200n8 
+    root=LABEL=testrootfs rootwait ro earlyprintk rootdelay=1 
+    fixrtc nocompcache mem=96M@0 mem_modem=32M@96M mem=44M@128M 
+    pmem=22M@172M mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M 
+    hwmem=48M@302M mem=152M@360M'",
+    boot
+
+#boot_cmds_android = TBD
\ No newline at end of file

=== added directory 'lava_dispatcher/default-config/lava-dispatcher/devices'
=== added file 'lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf'
--- lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf	1970-01-01 00:00:00 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf	2011-09-08 23:35:05 +0000
@@ -0,0 +1,19 @@ 
+# General lava-dispatcher settings.
+
+# Main LAVA server IP in the lab.
+#
+# This is the IP the device downloads the image parts from.
+LAVA_SERVER_IP = 192.168.1.10
+
+# Location for rootfs/boot tarballs extracted from images
+LAVA_IMAGE_TMPDIR = /linaro/images/tmp
+
+# URL where LAVA_IMAGE_TMPDIR can be accessed remotely
+LAVA_IMAGE_URL = http://%(LAVA_SERVER_IP)s/images/tmp
+
+# Location on the device for storing test results.
+LAVA_RESULT_DIR = /lava/results
+
+# Location for caching downloaded artifacts such as hwpacks and images
+LAVA_CACHEDIR = /linaro/images/cache
+

=== modified file 'lava_dispatcher/tests/test_config.py'
--- lava_dispatcher/tests/test_config.py	2011-06-27 04:55:08 +0000
+++ lava_dispatcher/tests/test_config.py	2011-09-08 23:35:05 +0000
@@ -19,12 +19,15 @@ 
 
 from unittest import TestCase
 
-from lava_dispatcher.config import BOARDS, LAVA_SERVER_IP
+from lava_dispatcher.config import get_config, get_device_config
+from lava_dispatcher.utils import string_to_list
 
 class TestConfigData(TestCase):
     def test_beagle01_uboot_cmds(self):
+        beagle01_config = get_device_config("beaglexm01")
         expected = [
             "mmc init",
+            "mmc part 0",
             "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage; fatload mmc "
                 "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'",
             "setenv bootargs ' console=tty0 console=ttyO2,115200n8 "
@@ -32,11 +35,12 @@ 
                 "nocompcache vram=12M omapfb.debug=y "
                 "omapfb.mode=dvi:1280x720MR-16@60'",
             "boot"]
-        brd = BOARDS["beagle01"]
-        uboot_cmds = brd.uboot_cmds
-        self.assertEquals(expected, uboot_cmds)
+        uboot_cmds = beagle01_config.get("boot_cmds")
+        self.assertEquals(expected, string_to_list(uboot_cmds))
 
     def test_server_ip(self):
+        server_config = get_config("lava-dispatcher")
         expected = "192.168.1.10"
-        self.assertEqual(expected, LAVA_SERVER_IP)
+        lava_server_ip = server_config.get("LAVA_SERVER_IP")
+        self.assertEqual(expected, lava_server_ip)
 

=== modified file 'lava_dispatcher/utils.py'
--- lava_dispatcher/utils.py	2011-08-18 20:38:38 +0000
+++ lava_dispatcher/utils.py	2011-09-01 03:03:09 +0000
@@ -22,8 +22,7 @@ 
 import shutil
 import urllib2
 import urlparse
-
-from lava_dispatcher.config import LAVA_CACHEDIR
+from shlex import shlex
 
 def download(url, path=""):
     urlpath = urlparse.urlsplit(url).path
@@ -41,8 +40,8 @@ 
         raise RuntimeError("Could not retrieve %s" % url)
     return filename
 
-def download_with_cache(url, path=""):
-    cache_loc = url_to_cache(url)
+def download_with_cache(url, path="", cachedir=""):
+    cache_loc = url_to_cache(url, cachedir)
     if os.path.exists(cache_loc):
         filename = os.path.basename(cache_loc)
         file_location = os.path.join(path, filename)
@@ -61,8 +60,16 @@ 
             #so ignore
     return file_location
 
-def url_to_cache(url):
+def url_to_cache(url, cachedir):
     url_parts = urlparse.urlsplit(url)
-    path = os.path.join(LAVA_CACHEDIR, url_parts.netloc,
+    path = os.path.join(cachedir, url_parts.netloc,
         url_parts.path.lstrip(os.sep))
     return path
+
+def string_to_list(string):
+    splitter = shlex(string, posix=True)
+    splitter.whitespace = ","
+    splitter.whitespace_split = True
+    newlines_to_spaces = lambda x: x.replace('\n', ' ')
+    strip_newlines = lambda x: newlines_to_spaces(x).strip(' ')    
+    return map(strip_newlines, list(splitter))

=== modified file 'setup.py'
--- setup.py	2011-07-21 16:55:47 +0000
+++ setup.py	2011-09-09 00:52:17 +0000
@@ -12,6 +12,15 @@ 
     author='Linaro Validation Team',
     author_email='linaro-dev@lists.linaro.org',
     packages=find_packages(),
+    package_data= {
+        'lava_dispatcher': [
+            'default-config/lava-dispatcher/lava-dispatcher.conf',
+            'default-config/lava-dispatcher/lava-dispatcher.conf',
+            'default-config/lava-dispatcher/device-defaults.conf',
+            'default-config/lava-dispatcher/device-types/*.conf',
+            'default-config/lava-dispatcher/devices/*.conf',
+            ],
+        },
     scripts = [
         'lava-dispatch'
     ],