From patchwork Wed Jan 18 21:28:13 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zygmunt Krynicki X-Patchwork-Id: 6285 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 60B8623E0E for ; Wed, 18 Jan 2012 21:28:15 +0000 (UTC) Received: from mail-bk0-f52.google.com (mail-bk0-f52.google.com [209.85.214.52]) by fiordland.canonical.com (Postfix) with ESMTP id 40C92A1863A for ; Wed, 18 Jan 2012 21:28:15 +0000 (UTC) Received: by bkbzt4 with SMTP id zt4so3580214bkb.11 for ; Wed, 18 Jan 2012 13:28:15 -0800 (PST) Received: by 10.204.153.27 with SMTP id i27mr9227889bkw.81.1326922094918; Wed, 18 Jan 2012 13:28:14 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.205.82.144 with SMTP id ac16cs160683bkc; Wed, 18 Jan 2012 13:28:14 -0800 (PST) Received: by 10.180.99.232 with SMTP id et8mr34107796wib.8.1326922093535; Wed, 18 Jan 2012 13:28:13 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id k66si17069877weq.23.2012.01.18.13.28.13 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 18 Jan 2012 13:28:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Rnd3F-0004Ef-9l for ; Wed, 18 Jan 2012 21:28:13 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 3D2E3E0286 for ; Wed, 18 Jan 2012 21:28:13 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-server X-Launchpad-Branch: ~linaro-validation/lava-server/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 341 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-server/trunk] Rev 341: Merge lp:~zkrynicki/lava-server/data-tables-1 Message-Id: <20120118212813.5947.56898.launchpad@ackee.canonical.com> Date: Wed, 18 Jan 2012 21:28:13 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14681"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 586620c8a5c2da4590b23dbc0152b5a24dbdde7d Merge authors: Zygmunt Krynicki (zkrynicki) Related merge proposals: https://code.launchpad.net/~zkrynicki/lava-server/data-tables-1/+merge/88800 proposed by: Zygmunt Krynicki (zkrynicki) ------------------------------------------------------------ revno: 341 [merge] committer: Zygmunt Krynicki branch nick: trunk timestamp: Wed 2012-01-18 22:23:45 +0100 message: Merge lp:~zkrynicki/lava-server/data-tables-1 This adds initial support for data-tables server side code. Only the array backend is included. We are also using django 1.3 features for the first time so dependencies for lava-server have been update. added: lava/ lava/__init__.py lava/utils/ lava/utils/__init__.py lava/utils/data_tables/ lava/utils/data_tables/__init__.py lava/utils/data_tables/backends.py lava/utils/data_tables/interface.py lava/utils/data_tables/query.py lava/utils/data_tables/views.py lava/utils/interface/ lava/utils/interface/__init__.py modified: setup.py --- lp:lava-server https://code.launchpad.net/~linaro-validation/lava-server/trunk You are subscribed to branch lp:lava-server. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-server/trunk/+edit-subscription === added directory 'lava' === added file 'lava/__init__.py' --- lava/__init__.py 1970-01-01 00:00:00 +0000 +++ lava/__init__.py 2012-01-17 01:15:02 +0000 @@ -0,0 +1,3 @@ +__import__('pkg_resources').declare_namespace(__name__) +# DO NOT ADD ANYTHING TO THIS FILE! +# IT MUST STAY AS IS (empty apart from the two lines above) === added directory 'lava/utils' === added file 'lava/utils/__init__.py' --- lava/utils/__init__.py 1970-01-01 00:00:00 +0000 +++ lava/utils/__init__.py 2012-01-17 01:16:42 +0000 @@ -0,0 +1,3 @@ +__import__('pkg_resources').declare_namespace(__name__) +# DO NOT ADD ANYTHING TO THIS FILE! +# IT MUST STAY AS IS (empty apart from the two lines above) === added directory 'lava/utils/data_tables' === added file 'lava/utils/data_tables/__init__.py' === added file 'lava/utils/data_tables/backends.py' --- lava/utils/data_tables/backends.py 1970-01-01 00:00:00 +0000 +++ lava/utils/data_tables/backends.py 2012-01-18 21:15:42 +0000 @@ -0,0 +1,72 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Zygmunt Krynicki +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + +from django.core.exceptions import ImproperlyConfigured + +from lava.utils.data_tables.interface import IBackend + + +class _BackendBase(IBackend): + """ + Common code for data backends to data tables + """ + + def process(self, query): + return { + 'sEcho': query.sEcho, + 'sColumns': query.sColumns + } + + +class ArrayBackend(_BackendBase): + """ + Array backend to data tables + + Stores all data in a plain python list. All filtering is handled in the + running process. It is suitable for very small data sets only but has the + advantage of being unrelated to any databases. + """ + + def __init__(self, data): + self.data = data + + def process(self, query): + # Get the basic response structure + response = super(ArrayBackend, self).process(query) + # 0) Copy original data + # TODO: add support for lazy copy (only if really needed) + data = list(self.data) + response['iTotalRecords'] = len(data) + # 1) Apply search/filtering + if query.sSearch: + if query.bRegex: + raise NotImplementedError("Searching with regular expresions is not implemented") + else: + data = [row for row in data if any((query.sSearch in unicode(cell) for cell in row))] + # Remember how many records matched filtering + response['iTotalDisplayRecords'] = len(data) + # TODO: Support regex search + # TODO: Support per-column search + # 2) Apply sorting + for column_index, order in reversed(euery.sorting_columns): + data.sort(key=lambda row: row[column_index], reverse=order == 'desc') + # 3) Apply offset/limit + data = data[query.iDisplayStart:query.iDisplayStart + query.iDisplayLength] + # Remember the subset of the displayed data + response['aaData'] = data + return response === added file 'lava/utils/data_tables/interface.py' --- lava/utils/data_tables/interface.py 1970-01-01 00:00:00 +0000 +++ lava/utils/data_tables/interface.py 2012-01-17 01:26:01 +0000 @@ -0,0 +1,34 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Zygmunt Krynicki +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + +from lava.utils.interface import Interface, abstractmethod + + +class IBackend(Interface): + """ + Interface for data-store-and-compute backends to data-tables. + """ + + @abstractmethod + def process(self, query): + """ + Process data query. + + Process the query and return a JSON response object compatible with + data tables. + """ === added file 'lava/utils/data_tables/query.py' --- lava/utils/data_tables/query.py 1970-01-01 00:00:00 +0000 +++ lava/utils/data_tables/query.py 2012-01-17 01:24:32 +0000 @@ -0,0 +1,67 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Zygmunt Krynicki +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + + +class DataTableQuery(object): + """ + Query to data table server backend + """ + + DEFAULT_DISPLAY_LENGTH = 10 + + def __init__(self, request): + # Store the request object + self.request = request + + # Number of columns + self.iColumns = int(request.GET.get('iColumns', 0)) + self.sColumns = request.GET.get('sColumns', '') + + # Echo data (seems to be cache poison, needs follow up checks) + self.sEcho = request.GET.get('sEcho', 0) + + # Some unspecified value + # XXX: What is this for? + self._ = request.GET.get('_', '') + + # Data window parameters + self.iDisplayStart = int(request.GET.get('iDisplayStart', 0)) + self.iDisplayLength = int(request.GET.get('iDisplayLength', self.DEFAULT_DISPLAY_LENGTH)) + + # Sorting/ordering parameters + self.sorting_columns = [] + self.iSortingCols = int(request.GET.get('iSortingCols', 0)) + for i in range(self.iSortingCols): + column_index = int(request.GET.get('iSortCol_{0}'.format(i), 0)) + sortable = request.GET.get('bSortable_{0}'.format(i), 'false') == 'true' + if sortable: + sorting_direction = request.GET.get('sSortDir_{0}'.format(i), 'asc') + self.sorting_columns.append((column_index, sorting_direction)) + + # Global search parameters + self.sSearch = request.GET.get('sSearch', '') + self.bRegex = request.GET.get('bRegex', 'false') == 'true' + + # Per-column search parameters + self.search_columns = [] + for i in range(self.iColumns): + searchable = request.GET.get('bSearchable_{0}'.format(i), 'false') == 'true' + if searchable: + regex = request.GET.get('bRegex_{0}'.format(i), 'false') == 'true' + term = request.GET.get('sSearch_{0}'.format(i), '') + self.search_columns.append((i, regex, term)) === added file 'lava/utils/data_tables/views.py' --- lava/utils/data_tables/views.py 1970-01-01 00:00:00 +0000 +++ lava/utils/data_tables/views.py 2012-01-17 01:43:59 +0000 @@ -0,0 +1,42 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Zygmunt Krynicki +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + +from django.core.exceptions import ImproperlyConfigured +from django.http import HttpResponse +from django.utils import simplejson +from django.views.generic import View + +from lava.utils.data_tables.query import DataTableQuery + + +class DataTableView(View): + """ + View for processing data table requests + """ + + backend = None + + def get(self, request, *args, **kwargs): + if self.backend is None: + raise ImproperlyConfigured( + "DataTableView requires a definition of a backend") + query = DataTableQuery(request) + result = self.backend.process(query) + return HttpResponse( + simplejson.dumps(result), + mimetype='application/json') === added directory 'lava/utils/interface' === added file 'lava/utils/interface/__init__.py' --- lava/utils/interface/__init__.py 1970-01-01 00:00:00 +0000 +++ lava/utils/interface/__init__.py 2012-01-17 01:18:34 +0000 @@ -0,0 +1,27 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Zygmunt Krynicki +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + +from abc import ABCMeta, abstractmethod + + +class Interface(object): + """ + Interface class + """ + + __metaclass__ = ABCMeta === modified file 'setup.py' --- setup.py 2011-11-29 19:59:48 +0000 +++ setup.py 2012-01-17 01:29:55 +0000 @@ -26,6 +26,7 @@ version=":versiontools:lava_server:__version__", author="Zygmunt Krynicki", author_email="zygmunt.krynicki@linaro.org", + namespace_packages=['lava', 'lava.utils'], packages=find_packages(), entry_points=""" [console_scripts] @@ -57,7 +58,7 @@ "Topic :: Software Development :: Testing", ], install_requires=[ - 'django >= 1.2', + 'django >= 1.3', 'django-debian >= 0.10', 'django-openid-auth >= 0.2', 'django-restricted-resource >= 0.2.6',