From patchwork Tue Dec 10 12:08:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 181116 Delivered-To: patch@linaro.org Received: by 2002:a92:3001:0:0:0:0:0 with SMTP id x1csp5694839ile; Tue, 10 Dec 2019 04:08:41 -0800 (PST) X-Google-Smtp-Source: APXvYqwrbi1yhcp9ZkcT9WEtGGa23HMbfSz0kBGouSj+RZjk1FXYEs63l/yxvnitBqPBRx44KX/W X-Received: by 2002:aa7:979a:: with SMTP id o26mr35006531pfp.0.1575979721270; Tue, 10 Dec 2019 04:08:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1575979721; cv=none; d=google.com; s=arc-20160816; b=UtEuR/SPJOeZ4i11X/+sONrJ632Bvfp10cJ1OyloQgToss8wATKRbDQJGEnSTEwkgg VzESsfsyuswQc7lke14BtXQbfHy/OKGHP6dEtUQTkLHhC4o5v9jiFhJyBQzh9dbiJ/d8 GFjShdqqILljCZalSMym1R5EvEKK7ulqUkrhod46bSMmLEoTyMEjbKv+Bbm4dk5WmGKQ dS2YG/CtNxSD+4B+SFQrny7NF0lUUUJ7XmF8MTbX6sWiIAmbMLGh2DSVW9Hq+Vv/n7pg wq/W3KwssjbbUDIGnTSzuvPW0NzVTAtGb/AjuQi9IPjUR9IBJwKECXw/BGf4jDRzpkY9 uPPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:message-id:date:to:from:dkim-signature:delivered-to; bh=t885EJ7uFFwk3ciVXOLnYBMHOzQ/Aay2uimCSMCC9fs=; b=WHlhZiKPFAzKPePsNpb2/V4WSVQoQe1UjlccchqBv+WQ/GhLDeGEwcpUdGROwYsMkB r4dvp/njkjK44CJ7TogjQAfpl6ff6jDYiQkevhAoQaYyvPlWchGNT/4Ji0MANCk7SAyh 8RvAJZx9YvPUKIEfTDwj0As4l3pulBC2CXFMUNf8vEKa9+F29Qz51zWx22sAZAwpGbaf JzX2NZV8QtPMvv+NUyVu3hFm9hs5PjJColoP0LtY//nv6EKRXzoboMLLtBlpaImloJUp EfUamAq3svW06M4QukoblO9vaTQJHHgjICP+UAgBIlX19ESiypR8vXYU9fVgNJtVq9SV QfJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=yV4j7cS0; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id m4si2730492pgs.238.2019.12.10.04.08.40; Tue, 10 Dec 2019 04:08:41 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=yV4j7cS0; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ec2-34-214-78-129.us-west-2.compute.amazonaws.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 2298D7FCA3; Tue, 10 Dec 2019 12:08:38 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by mail.openembedded.org (Postfix) with ESMTP id 659037FCAC for ; Tue, 10 Dec 2019 12:08:22 +0000 (UTC) Received: by mail-wr1-f48.google.com with SMTP id g17so19793699wro.2 for ; Tue, 10 Dec 2019 04:08:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=UlrTGtCR5KQYOvp+RTmzIWjTqJ1e85b6PTzsUrJ0NGw=; b=yV4j7cS0hx/Dvgf1+8itj0AoFCwhZb5/Rab0zUCT0yjRsvqxI0jy3y+2a0lHN2UgTQ Z6sBaOZpmtXu14uUw1PTDER+gAD92hotKivR3obwZm/BejueLLCeOqmx3MvG7c2CPKXd uXsyEpjZdCsHBloVXGFoEvbraxO4+PT8r8eZ3jL8mGMSHgCU8vSXvUrDoYoaBcqX2V4M m8rcBv/3M99F93Gzp/QQDVgYb3+oZs5hZtZaQYjXLBjxxMXplssnLf/28XJWWTlsxxQt 6NBqRNvMxuEDt32u//yEEyDZ/PxSUvSZwltszkEcBAt/h9ymKkzMsYN1Z2pFlMjZJBqD aW8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=UlrTGtCR5KQYOvp+RTmzIWjTqJ1e85b6PTzsUrJ0NGw=; b=kz4mb6gxoqn9PXWCyjSkCFjJ/8rRPfxle+T+2BGWGp1R1fxonZUh1nLwyVmAqteD6v rz9MLFwG+d8xXjdbGh8AjAvJhVGuZmdXeX5H2WI33wgoFfywUyURsbAoWWc4ty9ELgxM MRCkHUTsd4H7PqllcFHo2cAZcPcE16NSsnCEx7v3sWloMLy02KkAh/qNEEET4GhcTzy3 YepSH9MtVb0vrO6j+oACayM/HRJcHqc/zq74j3oXDG1MQ2Gj+6zmRIdUOHS9lTGRdoHV aPnth1jcguh6zum0DkDWlCsbTNtX8Al/EYude50RUwneqY9X25SpjKJVU7/2a3jSMll0 OpoA== X-Gm-Message-State: APjAAAXG0qprcnSaT3vPKTwgaEIXjc/0d3foAZAagyh57rnNabsp/Aef 6k+WgVg0q2WAWPyGfP5wb1yR/caw3h0= X-Received: by 2002:adf:bc87:: with SMTP id g7mr2989875wrh.121.1575979702312; Tue, 10 Dec 2019 04:08:22 -0800 (PST) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id n16sm2961022wro.88.2019.12.10.04.08.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2019 04:08:20 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Tue, 10 Dec 2019 12:08:17 +0000 Message-Id: <20191210120817.8374-1-ross.burton@intel.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [OE-core] [PATCH] scripts: add pkgdataui X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org pkgdataui is a Python 3/GObject Introspection/GTK+ 3 tool to browse the pkgdata database at your leisure. By being graphical it is easier to explore and can follow links between packages. This is very much a work in progress, so be gentle and patches are welcome. Signed-off-by: Ross Burton --- scripts/pkgdataui.glade | 335 ++++++++++++++++++++++++++++++++++++++++ scripts/pkgdataui.py | 241 +++++++++++++++++++++++++++++ 2 files changed, 576 insertions(+) create mode 100644 scripts/pkgdataui.glade create mode 100755 scripts/pkgdataui.py -- 2.20.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/scripts/pkgdataui.glade b/scripts/pkgdataui.glade new file mode 100644 index 00000000000..04e987b975e --- /dev/null +++ b/scripts/pkgdataui.glade @@ -0,0 +1,335 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + False + Package Data Browser + accessories-dictionary + True + + + True + False + 4 + 4 + 4 + 4 + vertical + 4 + + + False + pkgdata_store + 1 + + + + 0 + + + + + False + True + 0 + + + + + True + True + 200 + True + + + True + True + in + 100 + + + True + True + recipe_store + 0 + True + False + + + + + + fixed + Recipe + + + + 0 + + + + + + + + + False + True + + + + + True + True + 200 + True + + + True + True + in + 100 + + + True + True + package_store + 0 + False + + + + + + True + autosize + Package + 0 + + + + 0 + + + + + + + True + autosize + Size + 1 + + + + + + + + + + False + True + + + + + True + False + 4 + vertical + 4 + + + True + False + 0 + label + + + False + True + 0 + + + + + True + False + 0 + depends_label + True + False + + + False + True + 1 + + + + + True + False + 0 + recs_label + True + False + + + False + True + 2 + + + + + True + False + 0 + suggests_label + True + False + + + False + True + 3 + + + + + True + False + 0 + provides_label + True + False + + + False + True + 4 + + + + + True + False + 0 + files_label + end + + + False + True + 5 + + + + + True + True + in + + + True + True + file_store + True + 0 + False + + + + + + Name + True + 0 + + + rgba(0,0,0,0) + + + 0 + + + + + + + Size + True + 1 + + + + 1 + + + + + + + + + True + True + 6 + + + + + True + True + + + + + True + True + + + + + True + True + 1 + + + + + + diff --git a/scripts/pkgdataui.py b/scripts/pkgdataui.py new file mode 100755 index 00000000000..f6b23fc0be6 --- /dev/null +++ b/scripts/pkgdataui.py @@ -0,0 +1,241 @@ +#! /usr/bin/env python3 + +import os, sys, enum, ast + +scripts_path = os.path.dirname(os.path.realpath(__file__)) +lib_path = scripts_path + '/lib' +sys.path = sys.path + [lib_path] + +import scriptpath +bitbakepath = scriptpath.add_bitbake_lib_path() +if not bitbakepath: + print("Unable to find bitbake by searching parent directory of this script or PATH") + sys.exit(1) +import bb + +import gi +gi.require_version('Gtk', '3.0') +from gi.repository import Gtk, Gdk, GObject + +RecipeColumns = enum.IntEnum("RecipeColumns", {"Recipe": 0}) +PackageColumns = enum.IntEnum("PackageColumns", {"Package": 0, "Size": 1}) +FileColumns = enum.IntEnum("FileColumns", {"Filename": 0, "Size": 1}) + +import time +def timeit(f): + def timed(*args, **kw): + ts = time.time() + print ("func:%r calling" % f.__name__) + result = f(*args, **kw) + te = time.time() + print ('func:%r args:[%r, %r] took: %2.4f sec' % \ + (f.__name__, args, kw, te-ts)) + return result + return timed + +def human_size(nbytes): + import math + suffixes = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'] + human = nbytes + rank = 0 + if nbytes != 0: + rank = int((math.log10(nbytes)) / 3) + rank = min(rank, len(suffixes) - 1) + human = nbytes / (1000.0 ** rank) + f = ('%.2f' % human).rstrip('0').rstrip('.') + return '%s %s' % (f, suffixes[rank]) + +def load(filename, suffix=None): + from configparser import ConfigParser + from itertools import chain + + parser = ConfigParser() + if suffix: + parser.optionxform = lambda option: option.replace("_" + suffix, "") + with open(filename) as lines: + lines = chain(("[fake]",), lines) + parser.read_file(lines) + + # TODO extract the data and put it into a real dict so we can transform some + # values to ints? + return parser["fake"] + +def find_pkgdata(): + import subprocess + output = subprocess.check_output(("bitbake", "-e"), universal_newlines=True) + for line in output.splitlines(): + if line.startswith("PKGDATA_DIR="): + return line.split("=", 1)[1].strip("\'\"") + # TODO exception or something + return None + +def packages_in_recipe(pkgdata, recipe): + """ + Load the recipe pkgdata to determine the list of runtime packages. + """ + data = load(os.path.join(pkgdata, recipe)) + packages = data["PACKAGES"].split() + return packages + +def load_runtime_package(pkgdata, package): + return load(os.path.join(pkgdata, "runtime", package), suffix=package) + +def recipe_from_package(pkgdata, package): + data = load(os.path.join(pkgdata, "runtime", package), suffix=package) + return data["PN"] + +def summary(data): + s = "" + s += "{0[PKG]} {0[PKGV]}-{0[PKGR]}\n{0[LICENSE]}\n{0[SUMMARY]}\n".format(data) + + return s + + +class PkgUi(): + def __init__(self, pkgdata): + self.pkgdata = pkgdata + self.current_recipe = None + self.recipe_iters = {} + self.package_iters = {} + + builder = Gtk.Builder() + builder.add_from_file(os.path.join(os.path.dirname(__file__), "pkgdataui.glade")) + + self.window = builder.get_object("window") + self.window.connect("delete-event", Gtk.main_quit) + + self.recipe_store = builder.get_object("recipe_store") + self.recipe_view = builder.get_object("recipe_view") + self.package_store = builder.get_object("package_store") + self.package_view = builder.get_object("package_view") + + # Somehow resizable does not get set via builder xml + package_name_column = builder.get_object("package_name_column") + package_name_column.set_resizable(True) + file_name_column = builder.get_object("file_name_column") + file_name_column.set_resizable(True) + + self.recipe_view.get_selection().connect("changed", self.on_recipe_changed) + self.package_view.get_selection().connect("changed", self.on_package_changed) + + self.package_store.set_sort_column_id(PackageColumns.Package, Gtk.SortType.ASCENDING) + builder.get_object("package_size_column").set_cell_data_func(builder.get_object("package_size_cell"), lambda column, cell, model, iter, data: cell.set_property("text", human_size(model[iter][PackageColumns.Size]))) + + self.label = builder.get_object("label1") + self.depends_label = builder.get_object("depends_label") + self.recommends_label = builder.get_object("recommends_label") + self.suggests_label = builder.get_object("suggests_label") + self.provides_label = builder.get_object("provides_label") + + self.depends_label.connect("activate-link", self.on_link_activate) + self.recommends_label.connect("activate-link", self.on_link_activate) + self.suggests_label.connect("activate-link", self.on_link_activate) + + self.file_store = builder.get_object("file_store") + self.file_store.set_sort_column_id(FileColumns.Filename, Gtk.SortType.ASCENDING) + self.files_view = builder.get_object("files_scrollview") + self.files_label = builder.get_object("files_label") + + self.load_recipes() + + self.recipe_view.set_cursor(Gtk.TreePath.new_first()) + + self.window.show() + + def on_link_activate(self, label, url_string): + from urllib.parse import urlparse + url = urlparse(url_string) + if url.scheme == "package": + package = url.path + recipe = recipe_from_package(self.pkgdata, package) + + it = self.recipe_iters[recipe] + path = self.recipe_store.get_path(it) + self.recipe_view.set_cursor(path) + self.recipe_view.scroll_to_cell(path) + + self.on_recipe_changed(self.recipe_view.get_selection()) + + it = self.package_iters[package] + path = self.package_store.get_path(it) + self.package_view.set_cursor(path) + self.package_view.scroll_to_cell(path) + + return True + else: + return False + + def on_recipe_changed(self, selection): + self.package_store.clear() + self.package_iters = {} + + (model, it) = selection.get_selected() + if not it: + return + + recipe = model[it][RecipeColumns.Recipe] + for package in packages_in_recipe(self.pkgdata, recipe): + # TODO also show PKG after debian-renaming? + data = load_runtime_package(self.pkgdata, package) + # TODO stash data to avoid reading in on_package_changed + self.package_iters[package] = self.package_store.append([package, int(data["PKGSIZE"])]) + + def on_package_changed(self, selection): + self.label.set_text("") + self.file_store.clear() + self.depends_label.hide() + self.recommends_label.hide() + self.suggests_label.hide() + + (model, it) = selection.get_selected() + if it is None: + return + + package = model[it][PackageColumns.Package] + data = load_runtime_package(self.pkgdata, package) + + self.label.set_text(summary(data)) + + files = ast.literal_eval(data["FILES_INFO"]) + if files: + self.files_label.set_text("{0} files take {1}.".format(len(files), human_size(int(data["PKGSIZE"])))) + self.files_view.show() + for filename, size in files.items(): + self.file_store.append([filename, size]) + else: + self.files_view.hide() + self.files_label.set_text("This package has no files.") + + def update_deps(field, prefix, label, clickable=True): + if field in data: + l = [] + for name, version in bb.utils.explode_dep_versions2(data[field]).items(): + if clickable: + l.append("{0} {1}".format(name, " ".join(version))) + else: + l.append("{0} {1}".format(name, " ".join(version))) + label.set_markup(prefix + ", ".join(l)) + label.show() + else: + label.hide() + update_deps("RDEPENDS", "Depends: ", self.depends_label) + update_deps("RRECOMMENDS", "Recommends: ", self.recommends_label) + update_deps("RSUGGESTS", "Suggests: ", self.suggests_label) + update_deps("RPROVIDES", "Provides: ", self.provides_label, clickable=False) + + def load_recipes(self): + for recipe in sorted(os.listdir(pkgdata)): + if os.path.isfile(os.path.join(pkgdata, recipe)): + self.recipe_iters[recipe] = self.recipe_store.append([recipe]) + +if __name__ == "__main__": + import argparse + + parser = argparse.ArgumentParser(description='pkgdata browser') + parser.add_argument('-p', '--pkgdata', help="Optional location of pkgdata") + + args = parser.parse_args() + pkgdata = args.pkgdata if args.pkgdata else find_pkgdata() + # TODO assert pkgdata is a directory + window = PkgUi(pkgdata) + Gtk.main()