From patchwork Thu Aug 9 21:13:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Barker X-Patchwork-Id: 10662 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 4EC0523E28 for ; Thu, 9 Aug 2012 21:13:14 +0000 (UTC) Received: from mail-gg0-f180.google.com (mail-gg0-f180.google.com [209.85.161.180]) by fiordland.canonical.com (Postfix) with ESMTP id 019EEA18AC2 for ; Thu, 9 Aug 2012 21:13:13 +0000 (UTC) Received: by ggnf1 with SMTP id f1so966728ggn.11 for ; Thu, 09 Aug 2012 14:13:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=36vfiHi4DZVpj2SrmDL3SCw2Xqp+6Y+++qu6Vjh+AY0=; b=RBMOO/ugcjjrP/oeBtAUsrzDPOJQLuH5WH2MmR4UDQQVwtCCoDGit0B+JSXlGXey2J pdDDPB3DoRWg7syf93ixbrVB2Vu69FKFcLPRz8s8gr/+TeVTfa/tbVDsB3+L+zbnuTCa 3fKPUyEkpeWh9VO6UIbS2mrdS3ZLRWzOUOu5Sxt75ZQW0CdOC6VgM25AKyAL3yeGeZo/ e7Kjmr2mZJoSLUfxFwalla1/nGEFMBzvX34xSw5adZn30U/KRlGBJlwjCIg3p3LnfYIS yRg6WncP03uWUhBzo5pNF0FBc+D+peHa4N1s41UbXkHO+czjwMneSn8t4U0TTWgGWgT7 OK7A== Received: by 10.50.94.133 with SMTP id dc5mr2574260igb.16.1344546793162; Thu, 09 Aug 2012 14:13:13 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.200 with SMTP id ew8csp29101igc; Thu, 9 Aug 2012 14:13:12 -0700 (PDT) Received: by 10.216.29.17 with SMTP id h17mr258723wea.221.1344546791781; Thu, 09 Aug 2012 14:13:11 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id c62si3024124wef.110.2012.08.09.14.13.11 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 09 Aug 2012 14:13:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Sza2Z-0000Np-8Z for ; Thu, 09 Aug 2012 21:13:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 32D6CE01FF for ; Thu, 9 Aug 2012 21:13:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: libmatrix X-Launchpad-Branch: ~jesse-barker/libmatrix/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 39 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~jesse-barker/libmatrix/trunk] Rev 39: Merge of lp:~linaro-graphics-wg/libmatrix/split-refactor Message-Id: <20120809211311.21705.71214.launchpad@ackee.canonical.com> Date: Thu, 09 Aug 2012 21:13:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15761"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: fe32a91493a25e334aa4307f9902d06d79fbbd13 X-Gm-Message-State: ALoCoQkk+7Hy0rLA4vX6Kiq+rtERCgjUvW5jBNADSil9dyiw1mdOzsGf7SRuShl3BVfKitowOsWN Merge authors: Alexandros Frantzis (afrantzis) Related merge proposals: https://code.launchpad.net/~linaro-graphics-wg/libmatrix/split-refactor/+merge/118059 proposed by: Alexandros Frantzis (afrantzis) review: Approve - Jesse Barker (jesse-barker) ------------------------------------------------------------ revno: 39 [merge] committer: Jesse Barker branch nick: trunk timestamp: Thu 2012-08-09 14:11:02 -0700 message: Merge of lp:~linaro-graphics-wg/libmatrix/split-refactor Adds bash-style escaping for Util::split and cleans up the API to handle all of the modes we now support. added: test/util_split_test.cc test/util_split_test.h modified: Makefile shader-source.cc test/libmatrix_test.cc util.cc util.h --- lp:libmatrix https://code.launchpad.net/~jesse-barker/libmatrix/trunk You are subscribed to branch lp:libmatrix. To unsubscribe from this branch go to https://code.launchpad.net/~jesse-barker/libmatrix/trunk/+edit-subscription === modified file 'Makefile' --- Makefile 2012-05-02 21:36:04 +0000 +++ Makefile 2012-08-03 09:51:22 +0000 @@ -9,6 +9,7 @@ $(TESTDIR)/inverse_test.cc \ $(TESTDIR)/transpose_test.cc \ $(TESTDIR)/shader_source_test.cc \ + $(TESTDIR)/util_split_test.cc \ $(TESTDIR)/libmatrix_test.cc TESTOBJS = $(TESTSRCS:.cc=.o) @@ -32,6 +33,7 @@ $(TESTDIR)/inverse_test.o: $(TESTDIR)/inverse_test.cc $(TESTDIR)/inverse_test.h $(TESTDIR)/libmatrix_test.h mat.h $(TESTDIR)/transpose_test.o: $(TESTDIR)/transpose_test.cc $(TESTDIR)/transpose_test.h $(TESTDIR)/libmatrix_test.h mat.h $(TESTDIR)/shader_source_test.o: $(TESTDIR)/shader_source_test.cc $(TESTDIR)/shader_source_test.h $(TESTDIR)/libmatrix_test.h shader-source.h +$(TESTDIR)/util_split_test.o: $(TESTDIR)/util_split_test.cc $(TESTDIR)/util_split_test.h $(TESTDIR)/libmatrix_test.h util.h $(TESTDIR)/libmatrix_test: $(TESTOBJS) libmatrix.a $(CXX) -o $@ $^ run_tests: $(LIBMATRIX_TESTS) === modified file 'shader-source.cc' --- shader-source.cc 2012-01-26 16:12:35 +0000 +++ shader-source.cc 2012-08-03 09:46:28 +0000 @@ -589,7 +589,7 @@ { std::vector elems; - Util::split(precision_values, ',', elems); + Util::split(precision_values, ',', elems, Util::SplitModeNormal); for (size_t i = 0; i < elems.size() && i < 4; i++) { const std::string& pstr(elems[i]); === modified file 'test/libmatrix_test.cc' --- test/libmatrix_test.cc 2012-01-23 19:18:34 +0000 +++ test/libmatrix_test.cc 2012-08-06 15:04:36 +0000 @@ -8,6 +8,7 @@ // // Contributors: // Jesse Barker - original implementation. +// Alexandros Frantzis - Util::split tests // #include #include @@ -17,6 +18,7 @@ #include "transpose_test.h" #include "const_vec_test.h" #include "shader_source_test.h" +#include "util_split_test.h" using std::cerr; using std::cout; @@ -42,6 +44,8 @@ testVec.push_back(new MatrixTest3x3Transpose()); testVec.push_back(new MatrixTest4x4Transpose()); testVec.push_back(new ShaderSourceBasic()); + testVec.push_back(new UtilSplitTestNormal()); + testVec.push_back(new UtilSplitTestQuoted()); for (vector::iterator testIt = testVec.begin(); testIt != testVec.end(); === added file 'test/util_split_test.cc' --- test/util_split_test.cc 1970-01-01 00:00:00 +0000 +++ test/util_split_test.cc 2012-08-06 15:04:36 +0000 @@ -0,0 +1,180 @@ +// +// Copyright (c) 2012 Linaro Limited +// +// All rights reserved. This program and the accompanying materials +// are made available under the terms of the MIT License which accompanies +// this distribution, and is available at +// http://www.opensource.org/licenses/mit-license.php +// +// Contributors: +// Alexandros Frantzis - original implementation. +// +#include +#include +#include +#include "libmatrix_test.h" +#include "util_split_test.h" +#include "../util.h" + +using std::cout; +using std::endl; +using std::string; +using std::vector; + +template static bool +areVectorsEqual(vector& vec1, vector& vec2) +{ + if (vec1.size() != vec2.size()) + return false; + + for (unsigned int i = 0; i < vec1.size(); i++) + { + if (vec1[i] != vec2[i]) + return false; + } + + return true; +} + +template static void +printVector(vector& vec) +{ + cout << "["; + for (unsigned int i = 0; i < vec.size(); i++) + { + cout << '"' << vec[i] << '"'; + if (i < vec.size() - 1) + cout << ", "; + } + cout << "]"; +} + +void +UtilSplitTestNormal::run(const Options& options) +{ + const string test1("abc def ghi"); + const string test2(" abc: def :ghi "); + vector expected1; + vector expected2; + vector results; + + expected1.push_back("abc"); + expected1.push_back("def"); + expected1.push_back("ghi"); + + expected2.push_back(" abc"); + expected2.push_back(" def "); + expected2.push_back("ghi "); + + if (options.beVerbose()) + { + cout << "Testing string \"" << test1 << "\"" << endl; + } + + Util::split(test1, ' ', results, Util::SplitModeNormal); + + if (options.beVerbose()) + { + cout << "Split result: "; + printVector(results); + cout << endl << "Expected: "; + printVector(expected1); + cout << endl; + } + + if (!areVectorsEqual(results, expected1)) + { + return; + } + + results.clear(); + + if (options.beVerbose()) + { + cout << "Testing string \"" << test2 << "\"" << endl; + } + + Util::split(test2, ':', results, Util::SplitModeNormal); + + if (options.beVerbose()) + { + cout << "Split result: "; + printVector(results); + cout << endl << "Expected: "; + printVector(expected2); + cout << endl; + } + + if (!areVectorsEqual(results, expected2)) + { + return; + } + + pass_ = true; +} + +void +UtilSplitTestQuoted::run(const Options& options) +{ + const string test1("abc \"def' ghi\" klm\\ nop -b qr:title='123 \"456'"); + const string test2("abc: def='1:2:3:'ghi : \":jk\""); + vector expected1; + vector expected2; + vector results; + + expected1.push_back("abc"); + expected1.push_back("def' ghi"); + expected1.push_back("klm nop"); + expected1.push_back("-b"); + expected1.push_back("qr:title=123 \"456"); + + expected2.push_back("abc"); + expected2.push_back(" def=1:2:3:ghi "); + expected2.push_back(" :jk"); + + if (options.beVerbose()) + { + cout << "Testing string \"" << test1 << "\"" << endl; + } + + Util::split(test1, ' ', results, Util::SplitModeQuoted); + + if (options.beVerbose()) + { + cout << "Split result: "; + printVector(results); + cout << endl << "Expected: "; + printVector(expected1); + cout << endl; + } + + if (!areVectorsEqual(results, expected1)) + { + return; + } + + results.clear(); + + if (options.beVerbose()) + { + cout << "Testing string \"" << test2 << "\"" << endl; + } + + Util::split(test2, ':', results, Util::SplitModeQuoted); + + if (options.beVerbose()) + { + cout << "Split result: "; + printVector(results); + cout << endl << "Expected: "; + printVector(expected2); + cout << endl; + } + + if (!areVectorsEqual(results, expected2)) + { + return; + } + + pass_ = true; +} === added file 'test/util_split_test.h' --- test/util_split_test.h 1970-01-01 00:00:00 +0000 +++ test/util_split_test.h 2012-08-06 15:04:36 +0000 @@ -0,0 +1,31 @@ +// +// Copyright (c) 2012 Linaro Limited +// +// All rights reserved. This program and the accompanying materials +// are made available under the terms of the MIT License which accompanies +// this distribution, and is available at +// http://www.opensource.org/licenses/mit-license.php +// +// Contributors: +// Alexandros Frantzis - original implementation. +// +#ifndef UTIL_SPLIT_TEST_H_ +#define UTIL_SPLIT_TEST_H_ + +class MatrixTest; +class Options; + +class UtilSplitTestNormal : public MatrixTest +{ +public: + UtilSplitTestNormal() : MatrixTest("Util::split::normal") {} + virtual void run(const Options& options); +}; + +class UtilSplitTestQuoted : public MatrixTest +{ +public: + UtilSplitTestQuoted() : MatrixTest("Util::split::quoted") {} + virtual void run(const Options& options); +}; +#endif // UTIL_SPLIT_TEST_H_ === modified file 'util.cc' --- util.cc 2012-05-02 21:36:04 +0000 +++ util.cc 2012-08-03 10:08:36 +0000 @@ -25,25 +25,102 @@ using std::string; using std::vector; -void -Util::split(const string& src, char delim, vector& elementVec, bool fuzzy) -{ - // Trivial rejection - if (src.empty()) - { - return; - } - - // Simple case: we want to enforce the value of 'delim' strictly - if (!fuzzy) - { - std::stringstream ss(src); - string item; - while(std::getline(ss, item, delim)) - elementVec.push_back(item); - return; - } - +/* + * State machine for bash-like quoted string escaping: + * + * \ + * -----------> +---------+ + * | ---------- | Escaped | + * | | *,ESC +---------+ + * | | + * | v ' + * +--------+ ---> +--------------+ ----- + * | Normal | <--- | SingleQuoted | | *, ESC + * +--------+ ' +--------------+ <---- + * | ^ + * | | + * | | " +--------------+ ---- + * | ---------- | DoubleQuoted | | *, ESC + * -----------> +--------------+ <--- + * " | ^ + * \ | | *, ESC + * v | + * +---------------------+ + * | DoubleQuotedEscaped | + * +---------------------+ + * + * ESC: Mark character as Escaped + */ +static void +fill_escape_vector(const string &str, vector &esc_vec) +{ + enum State { + StateNormal, + StateEscaped, + StateDoubleQuoted, + StateDoubleQuotedEscaped, + StateSingleQuoted + }; + + State state = StateNormal; + + for (string::const_iterator iter = str.begin(); + iter != str.end(); + iter++) + { + const char c(*iter); + bool esc = false; + + switch (state) { + case StateNormal: + if (c == '"') + state = StateDoubleQuoted; + else if (c == '\\') + state = StateEscaped; + else if (c == '\'') + state = StateSingleQuoted; + break; + case StateEscaped: + esc = true; + state = StateNormal; + break; + case StateDoubleQuoted: + if (c == '"') + state = StateNormal; + else if (c == '\\') + state = StateDoubleQuotedEscaped; + else + esc = true; + break; + case StateDoubleQuotedEscaped: + esc = true; + state = StateDoubleQuoted; + break; + case StateSingleQuoted: + if (c == '\'') + state = StateNormal; + else + esc = true; + default: + break; + } + + esc_vec.push_back(esc); + } +} + +static void +split_normal(const string& src, char delim, vector& elementVec) +{ + std::stringstream ss(src); + string item; + while(std::getline(ss, item, delim)) + elementVec.push_back(item); +} + +static void +split_fuzzy(const string& src, char delim, vector& elementVec) +{ // Fuzzy case: Initialize our delimiter string based upon the caller's plus // a space to allow for more flexibility. string delimiter(" "); @@ -76,6 +153,70 @@ elementVec.push_back(str); } +static void +split_quoted(const string& src, char delim, vector& elementVec) +{ + std::stringstream ss; + vector escVec; + + /* Mark characters in the string as escaped or not */ + fill_escape_vector(src, escVec); + + /* Sanity check... */ + if (src.length() != escVec.size()) + return; + + for (vector::const_iterator iter = escVec.begin(); + iter != escVec.end(); + iter++) + { + bool escaped = static_cast(*iter); + char c = src[iter - escVec.begin()]; + + /* Output all characters, except unescaped ",\,' */ + if ((c != '"' && c != '\\' && c != '\'') || escaped) { + /* If we reach an unescaped delimiter character, do a split */ + if (c == delim && !escaped) { + elementVec.push_back(ss.str()); + ss.str(""); + ss.clear(); + } + else { + ss << c; + } + } + + } + + /* Handle final element, delimited by end of string */ + const string &finalElement(ss.str()); + if (!finalElement.empty()) + elementVec.push_back(finalElement); +} + +void +Util::split(const string& src, char delim, vector& elementVec, + Util::SplitMode mode) +{ + // Trivial rejection + if (src.empty()) + { + return; + } + + switch (mode) + { + case Util::SplitModeNormal: + return split_normal(src, delim, elementVec); + case Util::SplitModeFuzzy: + return split_fuzzy(src, delim, elementVec); + case Util::SplitModeQuoted: + return split_quoted(src, delim, elementVec); + default: + break; + } +} + uint64_t Util::get_timestamp_us() { === modified file 'util.h' --- util.h 2012-05-02 21:36:04 +0000 +++ util.h 2012-08-03 10:08:36 +0000 @@ -25,21 +25,33 @@ struct Util { /** + * How to perform the split() operation + */ + enum SplitMode { + /** Normal split operation */ + SplitModeNormal, + /** Allow for spaces and multiple consecutive occurences of the delimiter */ + SplitModeFuzzy, + /** Take into account bash-like quoting and escaping rules */ + SplitModeQuoted + }; + + /** * split() - Splits a string into elements using a provided delimiter * * @s: the string to split * @delim: the delimiter to use * @elems: the string vector to populate - * @fuzzy: (optional) enable/disable strict handling of @delim + * @mode: the SplitMode to use * * Using @delim to determine field boundaries, splits @s into separate * string elements. These elements are returned in the string vector - * @elems. If @fuzzy is true, then the handling of @delim allows for - * spaces and multiple consecutive occurences of @delim in determining - * field boundaries. As long as @s is non-empty, there will be at least - * one element in @elems. + * @elems. As long as @s is non-empty, there will be at least one + * element in @elems. */ - static void split(const std::string &s, char delim, std::vector &elems, bool fuzzy = false); + static void split(const std::string& src, char delim, + std::vector& elems, + Util::SplitMode mode); /** * get_timestamp_us() - Returns the current time in microseconds */