From patchwork Mon Dec 3 10:26:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 152685 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp6639256ljp; Mon, 3 Dec 2018 02:26:41 -0800 (PST) X-Google-Smtp-Source: AFSGD/XgI2JhF9xwuSmO0T11HL6Mbmy5T52LniONBW/1lLoOF/zecA6gjOX43RS6N02Y5nSwdJpU X-Received: by 2002:a63:d005:: with SMTP id z5mr12795610pgf.64.1543832801403; Mon, 03 Dec 2018 02:26:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543832801; cv=none; d=google.com; s=arc-20160816; b=cjNEiP8xNLZ7or2ZVTbca/scxzUSxLTdjVVmgK8aYYr33N2WejXqYbP6P0wHd/OOkm 4zJvgwWBpcoqUdHHmi8FYVgV/JGI1oHthz8qEQzEOQza/1VdoH/RHsSRwbP9hXtEvMeM gkI82ssMx05mG9DKfspZPHnVmL3Ya2u3LhkSboISZaXCON510pQVi7AetFGetFbAy1Gp r/CL68iOA1MGtZrq96kvBlu9/RCBw3QKN0XtHnX8p/UjXEjqIiytq9oofMQQPzFlOSug FvXuzWjBBJRBUQMjLewQJ+2YJdGKowqXu6x0ukRtPJMEeF3rrz07IXrV3KHDipgtwQmX ZmLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=Ez9Dbl5SS4V0L3K/huvVXA2vaD4DNxNJQxXuHwn5pRw=; b=RbbIS5cCxWmZJDUEuSGBUaWrDB3gbIEaxgXZOiW2z+JafESuNJnWzDinhxw7wlxIMo hKp8kMGVCNyldVy35WxW+SJBIHAq/5rG5AdLCtdY7iQMyOM9BMoVTb/30HLuijCwqDSg IhXFiVow9hNtEiT5CHvq9xTNT0vIEaJsKzA5tmH0kB6RYZzx0FU5ZvdOOo6HwXEYVLP0 gDhn0xoM3G2YOXQP9G0dsC5ypv5oKUO2On06AmNAEcThI6Q+9Zdw0LO0MPcARXmTn3fF mTIbyp0ns23hLkRc8v9WE/Vi51z7cheB6mZvxopz2MLnGe0bl8vWUqv0dpbvbn2LDFyb ZGEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=1zcc1xlV; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id x186si12294592pgb.33.2018.12.03.02.26.40; Mon, 03 Dec 2018 02:26:41 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=1zcc1xlV; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from 165.28.230.35.bc.googleusercontent.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 7D54E6BBF8; Mon, 3 Dec 2018 10:26:37 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by mail.openembedded.org (Postfix) with ESMTP id 209CB6C52A for ; Mon, 3 Dec 2018 10:26:35 +0000 (UTC) Received: by mail-wr1-f54.google.com with SMTP id 96so11555547wrb.2 for ; Mon, 03 Dec 2018 02:26:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id; bh=/IYQQ7XsnB8lyeDIgYgjZqs4T8lWIxYwfhB2SifaZGs=; b=1zcc1xlVp1GdSR4pGWmVF2uJWDt1dsbZIEXsvTAS7zHf82TaBddopsSF+aIFDpcNU1 G8zi44L/G4FuG7+648z1evhbc2iwYiGWDWFV4FxY+tZzDa8IZ10aaY9GxqFquE7eokju ZA6cJJ5Ik3dFZaGPFgQFBWlMBx8p/0Oi3z5nWi/PHEpGWFYJIkp8qYYQX6WAG62826M4 ZetEvLC0sPFKXAJ2roFSswOHEKZlKoLuvQtavHGg+K1OChOJ7h40WiztjA8KE3uv5NwA YiS4QGMb/5l07zcgsPcXhp1mlrqfzPcFmGiDoN7Fgt0rYIrOWCyHaQknUkW/u0ir4dMd JedQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=/IYQQ7XsnB8lyeDIgYgjZqs4T8lWIxYwfhB2SifaZGs=; b=kBKDXU2HLdyTA1fQJx7P0WW3AtRZQP2B7lzuItgAepXTynQHbfXHQ1n2r0jo+iAvwb BNCAJjuTNpE3mTJhTUFwn7FjYr1t55mvnVhlimx9aE0Su/RWSYUivUY/ZKyL4qscwAtV wHYlFQueHjh0XrrP700J0N1qFpyEx5bkL7hUcy8YMv1/vQcyMEu7Ud8zAkEPjkAs+0Nh ecb/nGOHHfzGVrS0j/EXZXVj07gdv8IY9sXUsiDe1U1MhNnbaCBLCRUdh6KYVaYAWfRK qasfkCOgpEzQJUknUjQyYHxp3/VXmZr2UbKPgI39bji5g5LrvMkwTm/crK7DkqKhxvZg HJrA== X-Gm-Message-State: AA+aEWaPkdYWgYEMrcdlibJi6lqPF1BWb6b5VcBO4GMD9wAAPTMG4DcS LSAvLro2tt/ncjKB8KVcbo4WxWMbXTw= X-Received: by 2002:a5d:4c8a:: with SMTP id z10mr13131058wrs.75.1543832796397; Mon, 03 Dec 2018 02:26:36 -0800 (PST) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id k26sm2977930wmi.28.2018.12.03.02.26.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Dec 2018 02:26:35 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 3 Dec 2018 10:26:33 +0000 Message-Id: <20181203102634.4850-1-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 Subject: [OE-core] [PATCH 1/2] patchreview: various fixes X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org Signed-off-by: Ross Burton --- scripts/contrib/patchreview.py | 46 ++++++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 11 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/scripts/contrib/patchreview.py b/scripts/contrib/patchreview.py index a4d5ba68676..072166504da 100755 --- a/scripts/contrib/patchreview.py +++ b/scripts/contrib/patchreview.py @@ -5,6 +5,7 @@ # - test suite # - validate signed-off-by +status_values = ("accepted", "pending", "inappropriate", "backport", "submitted", "denied") class Result: # Whether the patch has an Upstream-Status or not @@ -35,25 +36,26 @@ def blame_patch(patch): "--format=%s (%aN <%aE>)", "--", patch)).decode("utf-8").splitlines() -def patchreview(patches): - import re +def patchreview(path, patches): + import re, os.path # General pattern: start of line, optional whitespace, tag with optional # hyphen or spaces, maybe a colon, some whitespace, then the value, all case # insensitive. sob_re = re.compile(r"^[\t ]*(Signed[-_ ]off[-_ ]by:?)[\t ]*(.+)", re.IGNORECASE | re.MULTILINE) status_re = re.compile(r"^[\t ]*(Upstream[-_ ]Status:?)[\t ]*(\w*)", re.IGNORECASE | re.MULTILINE) - status_values = ("accepted", "pending", "inappropriate", "backport", "submitted", "denied") cve_tag_re = re.compile(r"^[\t ]*(CVE:)[\t ]*(.*)", re.IGNORECASE | re.MULTILINE) cve_re = re.compile(r"cve-[0-9]{4}-[0-9]{4,6}", re.IGNORECASE) results = {} for patch in patches: + + fullpath = os.path.join(path, patch) result = Result() - results[patch] = result + results[fullpath] = result - content = open(patch, encoding='ascii', errors='ignore').read() + content = open(fullpath, encoding='ascii', errors='ignore').read() # Find the Signed-off-by tag match = sob_re.search(content) @@ -122,6 +124,8 @@ def analyse(results, want_blame=False, verbose=True): missing_status += 1 if r.malformed_upstream_status or r.unknown_upstream_status: malformed_status += 1 + # Count patches with no status as pending + pending_patches +=1 if r.missing_cve: missing_cve += 1 if r.upstream_status == "pending": @@ -132,7 +136,6 @@ def analyse(results, want_blame=False, verbose=True): need_blame = True if verbose: print("Missing Signed-off-by tag (%s)" % patch) - if r.malformed_sob: need_blame = True if verbose: @@ -198,14 +201,35 @@ if __name__ == "__main__": args.add_argument("-b", "--blame", action="store_true", help="show blame for malformed patches") args.add_argument("-v", "--verbose", action="store_true", help="show per-patch results") args.add_argument("-g", "--histogram", action="store_true", help="show patch histogram") - args.add_argument("directory", nargs="?", help="directory to scan") + args.add_argument("-j", "--json", help="update JSON") + args.add_argument("directory", help="directory to scan") args = args.parse_args() - if args.directory: - os.chdir(args.directory) - patches = subprocess.check_output(("git", "ls-files", "recipes-*/**/*.patch", "recipes-*/**/*.diff")).decode("utf-8").split() - results = patchreview(patches) + patches = subprocess.check_output(("git", "-C", args.directory, "ls-files", "recipes-*/**/*.patch", "recipes-*/**/*.diff")).decode("utf-8").split() + results = patchreview(args.directory, patches) analyse(results, want_blame=args.blame, verbose=args.verbose) + + if args.json: + import json, os.path, collections + if os.path.isfile(args.json): + data = json.load(open(args.json)) + else: + data = [] + + row = collections.Counter() + row["total"] = len(results) + row["date"] = subprocess.check_output(["git", "-C", args.directory, "show", "-s", "--pretty=format:%cd", "--date=format:%s"]).decode("utf-8").strip() + for r in results.values(): + if r.upstream_status in status_values: + row[r.upstream_status] += 1 + if r.malformed_upstream_status or r.missing_upstream_status: + row['malformed-upstream-status'] += 1 + if r.malformed_sob or r.missing_sob: + row['malformed-sob'] += 1 + + data.append(row) + json.dump(data, open(args.json, "w")) + if args.histogram: print() histogram(results)