From patchwork Thu Jul 9 14:13:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 235158 Delivered-To: patch@linaro.org Received: by 2002:a92:d244:0:0:0:0:0 with SMTP id v4csp1458981ilg; Thu, 9 Jul 2020 07:21:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxHUs8ZqKCF2o49YoVnHNmWJDsiFSjoRFLV46ZDohsftUcVBA20xAGR/CZCYxaszY+f2TOl X-Received: by 2002:a25:8808:: with SMTP id c8mr15554339ybl.422.1594304497508; Thu, 09 Jul 2020 07:21:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594304497; cv=none; d=google.com; s=arc-20160816; b=uXv7cnA05+jN3+/pxrm7wRiejwNYubvxUcddT4jKO6j0cBhuKfVvqqDV2AlJg6Zr3u W8lfOkmyHs3MjBnKoGCabAH0v+zuOD6ec587YFoXlu4sdOZWW8IxLtjXs/CaYfjEYF/1 wHaCidZE0nZ1lvE+63kc2oOv6x0SP9A6dyf0dpZ/Bc4wpGR2HN8RgUtWi0i6ZWMFwb5T wBT0nHJdYQIKX693fnHFu2uXihKOYZ2amI1mDdNlM4AZRLRdqP/2wj2Y8an1cwS4Djgk qaEs+MIBuSY1A5LohGAEpzxF0+PTNtgjEPgYpYxlBZ5B+SoZpE2kNG1yKElbK9i1+fs/ dQOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ijru7vfuo0rF0FQiZmJ4xzr85Qw/JDmcUhGplkCDPx8=; b=fPj5O3Z9F/5j8liQONYEnNhG0VvxYMfO5mrj6n0+t2U/rb4fNNahVQDbtrRJsR1Lz6 Q4xcfRjjLS36nYLQnn/PbbZ9Q4wkOJgeFs8pcj6zUtovAzFOfBjWlo4RbwBwJ0tZGC7E Lg3k993USHjI45lAquv0rWmeDz/GDqHEK4JAsFEb9oRA/EM46qagv4qZVvtaSCpcPV5H hZgBFMM+TdGdDwL0w2NSYaE/THR2TgGZh5E+6oP7GwcrmlBRbQdDv82g2tXmtVwilm3D npuvnopn9nqffprlB8wdabtqATEKkTW/XFIsigI+zpv/7ZPI6ieaw+zHfGLyU3Q3Tj7H xJ3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=S3iVwUZj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r9si2781068ybm.175.2020.07.09.07.21.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Jul 2020 07:21:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=S3iVwUZj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtXQe-0002ik-S4 for patch@linaro.org; Thu, 09 Jul 2020 10:21:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtXJ6-0000YO-Kr for qemu-devel@nongnu.org; Thu, 09 Jul 2020 10:13:48 -0400 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:36065) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jtXJ4-0001Lc-7s for qemu-devel@nongnu.org; Thu, 09 Jul 2020 10:13:48 -0400 Received: by mail-wr1-x42c.google.com with SMTP id k6so2562595wrn.3 for ; Thu, 09 Jul 2020 07:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ijru7vfuo0rF0FQiZmJ4xzr85Qw/JDmcUhGplkCDPx8=; b=S3iVwUZjHDJZDZuyhj+yTgH4bbYUQr9euBhlOv9yMhNuK22V43NImYbZjebxqI2hd3 fYaHQgt05+FYw2DBZ3QLoRkMlFOwXAJ3xbA3gusj/zZx3vCsCULV+3xWtXca2KSWaqEl noyDJ1BSMzffHSHHDgj0uOeZXiBX2/Kl9+4TfwY9Y/cio+9JP15+5HUYHDCFhYljgjqY JuRMEDsBa6W6K/4wU1v8YfiaWkkboxI3nH4FOhKH7qYgFm3iNnuwgx6j24qGlNhA2HLB Q5cyGV85wbCVJwaI1lIjLZRiFf6l/YVkijeFGXscxxNU/JfahBts78jVih+4+IDHprqL dHjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ijru7vfuo0rF0FQiZmJ4xzr85Qw/JDmcUhGplkCDPx8=; b=lW0ge80xQ0EUGG2Omk6Sj/3Acu65I8bHGmYFPcICi5/4rSw5NA+uLzmDBbMMMVAMHp g0v6JCcqiTmFOb/gJjA5Bto98BLNfMQnB7JUJbnF8oz7F4ijPC40yZSq98mU0uVCGUhU CQ4Tlqbiv+7X/3aBtRvnDZWrFaVNyiYx98kPTe7AQb9KI2//ISHtVH7EenTy0ThGqQkK RojATaGzkadWzQburCgJvhAS2xEqZGUm4AEazzhkFiXWxNBId5l2KuantgCHMV2Ntl4o OM4JccjLF/us27hjpvYrsYi/lX/8mXPuFFJXsAFg0v6Yw7kMdDxn65l652YFzhdFwgEw /+Hw== X-Gm-Message-State: AOAM531q/vLNEjcsMk+QkYluQCYDPeGIGBMlouvpYI0W+JdcLwrHaaxZ 8c5QLxQMzOZ7rqMg+sD2PAtDcg== X-Received: by 2002:a5d:5686:: with SMTP id f6mr62620207wrv.15.1594304020957; Thu, 09 Jul 2020 07:13:40 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id u8sm5431717wrt.28.2020.07.09.07.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jul 2020 07:13:34 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 107A11FF93; Thu, 9 Jul 2020 15:13:28 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v1 07/13] plugins: new hwprofile plugin Date: Thu, 9 Jul 2020 15:13:21 +0100 Message-Id: <20200709141327.14631-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200709141327.14631-1-alex.bennee@linaro.org> References: <20200709141327.14631-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42c; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x42c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, robert.foley@linaro.org, =?utf-8?q?Alex_Benn=C3=A9e?= , richard.henderson@linaro.org, f4bug@amsat.org, robhenry@microsoft.com, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , aaron@os.amperecomputing.com, cota@braap.org, kuhn.chenqun@huawei.com, peter.puhov@linaro.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is a plugin intended to help with profiling access to various bits of system hardware. It only really makes sense for system emulation. It takes advantage of the recently exposed helper API that allows us to see the device name (memory region name) associated with a device. You can specify arg=read or arg=write to limit the tracking to just reads or writes (by default it does both). The pattern option: -plugin ./tests/plugin/libhwprofile.so,arg=pattern will allow you to see the access pattern to devices, eg: gic_cpu @ 0xffffffc010040000 off:00000000, 8, 1, 8, 1 off:00000000, 4, 1, 4, 1 off:00000000, 2, 1, 2, 1 off:00000000, 1, 1, 1, 1 The source option: -plugin ./tests/plugin/libhwprofile.so,arg=source will track the virtual source address of the instruction making the access: pl011 @ 0xffffffc010031000 pc:ffffffc0104c785c, 1, 4, 0, 0 pc:ffffffc0104c7898, 1, 4, 0, 0 pc:ffffffc010512bcc, 2, 1867, 0, 0 You cannot mix source and pattern. Finally the match option allow you to limit the tracking to just the devices you care about. Signed-off-by: Alex Bennée Reviewed-by: Robert Foley Tested-by: Robert Foley --- v2 - use 64 bit cpu masks - warn if we will exceed cpu mask capability - don't run in linux-user mode - skip in check-tcg for linux-user test v3 - update device name API - re-factor and clean-up - add source tracking - use direct hash for source/pattern - add match function - expand the commit message - checkpatch cleanups --- tests/plugin/hwprofile.c | 305 ++++++++++++++++++++++++++++++++++++++ tests/plugin/Makefile | 1 + tests/tcg/Makefile.target | 12 +- 3 files changed, 316 insertions(+), 2 deletions(-) create mode 100644 tests/plugin/hwprofile.c -- 2.20.1 diff --git a/tests/plugin/hwprofile.c b/tests/plugin/hwprofile.c new file mode 100644 index 000000000000..6dac1d5f8541 --- /dev/null +++ b/tests/plugin/hwprofile.c @@ -0,0 +1,305 @@ +/* + * Copyright (C) 2020, Alex Bennée + * + * HW Profile - breakdown access patterns for IO to devices + * + * License: GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION; + +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) + +typedef struct { + uint64_t cpu_read; + uint64_t cpu_write; + uint64_t reads; + uint64_t writes; +} IOCounts; + +typedef struct { + uint64_t off_or_pc; + IOCounts counts; +} IOLocationCounts; + +typedef struct { + const char *name; + uint64_t base; + IOCounts totals; + GHashTable *detail; +} DeviceCounts; + +static GMutex lock; +static GHashTable *devices; + +/* track the access pattern to a piece of HW */ +static bool pattern; +/* track the source address of access to HW */ +static bool source; +/* track only matched regions of HW */ +static bool check_match; +static gchar **matches; + +static enum qemu_plugin_mem_rw rw = QEMU_PLUGIN_MEM_RW; + +static inline bool track_reads(void) +{ + return rw == QEMU_PLUGIN_MEM_RW || rw == QEMU_PLUGIN_MEM_R; +} + +static inline bool track_writes(void) +{ + return rw == QEMU_PLUGIN_MEM_RW || rw == QEMU_PLUGIN_MEM_W; +} + +static void plugin_init(void) +{ + devices = g_hash_table_new(NULL, NULL); +} + +static gint sort_cmp(gconstpointer a, gconstpointer b) +{ + DeviceCounts *ea = (DeviceCounts *) a; + DeviceCounts *eb = (DeviceCounts *) b; + return ea->totals.reads + ea->totals.writes > + eb->totals.reads + eb->totals.writes ? -1 : 1; +} + +static gint sort_loc(gconstpointer a, gconstpointer b) +{ + IOLocationCounts *ea = (IOLocationCounts *) a; + IOLocationCounts *eb = (IOLocationCounts *) b; + return ea->off_or_pc > eb->off_or_pc; +} + +static void fmt_iocount_record(GString *s, IOCounts *rec) +{ + if (track_reads()) { + g_string_append_printf(s, ", %"PRIx64", %"PRId64, + rec->cpu_read, rec->reads); + } + if (track_writes()) { + g_string_append_printf(s, ", %"PRIx64", %"PRId64, + rec->cpu_write, rec->writes); + } +} + +static void fmt_dev_record(GString *s, DeviceCounts *rec) +{ + g_string_append_printf(s, "%s, 0x%"PRIx64, + rec->name, rec->base); + fmt_iocount_record(s, &rec->totals); + g_string_append_c(s, '\n'); +} + +static void plugin_exit(qemu_plugin_id_t id, void *p) +{ + g_autoptr(GString) report = g_string_new(""); + GList *counts; + + if (!(pattern || source)) { + g_string_printf(report, "Device, Address"); + if (track_reads()) { + g_string_append_printf(report, ", RCPUs, Reads"); + } + if (track_writes()) { + g_string_append_printf(report, ", WCPUs, Writes"); + } + g_string_append_c(report, '\n'); + } + + counts = g_hash_table_get_values(devices); + if (counts && g_list_next(counts)) { + GList *it; + + it = g_list_sort(counts, sort_cmp); + + while (it) { + DeviceCounts *rec = (DeviceCounts *) it->data; + if (rec->detail) { + GList *accesses = g_hash_table_get_values(rec->detail); + GList *io_it = g_list_sort(accesses, sort_loc); + const char *prefix = pattern ? "off" : "pc"; + g_string_append_printf(report, "%s @ 0x%"PRIx64"\n", + rec->name, rec->base); + while (io_it) { + IOLocationCounts *loc = (IOLocationCounts *) io_it->data; + g_string_append_printf(report, " %s:%08"PRIx64, + prefix, loc->off_or_pc); + fmt_iocount_record(report, &loc->counts); + g_string_append_c(report, '\n'); + io_it = io_it->next; + } + } else { + fmt_dev_record(report, rec); + } + it = it->next; + }; + g_list_free(it); + } + + qemu_plugin_outs(report->str); +} + +static DeviceCounts *new_count(const char *name, uint64_t base) +{ + DeviceCounts *count = g_new0(DeviceCounts, 1); + count->name = name; + count->base = base; + if (pattern || source) { + count->detail = g_hash_table_new(NULL, NULL); + } + g_hash_table_insert(devices, (gpointer) name, count); + return count; +} + +static IOLocationCounts *new_location(GHashTable *table, uint64_t off_or_pc) +{ + IOLocationCounts *loc = g_new0(IOLocationCounts, 1); + loc->off_or_pc = off_or_pc; + g_hash_table_insert(table, (gpointer) off_or_pc, loc); + return loc; +} + +static void hwprofile_match_hit(DeviceCounts *rec, uint64_t off) +{ + g_autoptr(GString) report = g_string_new("hwprofile: match @ offset"); + g_string_append_printf(report, "%"PRIx64", previous hits\n", off); + fmt_dev_record(report, rec); + qemu_plugin_outs(report->str); +} + +static void inc_count(IOCounts *count, bool is_write, unsigned int cpu_index) +{ + if (is_write) { + count->writes++; + count->cpu_write |= (1 << cpu_index); + } else { + count->reads++; + count->cpu_read |= (1 << cpu_index); + } +} + +static void vcpu_haddr(unsigned int cpu_index, qemu_plugin_meminfo_t meminfo, + uint64_t vaddr, void *udata) +{ + struct qemu_plugin_hwaddr *hwaddr = qemu_plugin_get_hwaddr(meminfo, vaddr); + + if (!hwaddr || !qemu_plugin_hwaddr_is_io(hwaddr)) { + return; + } else { + const char *name = qemu_plugin_hwaddr_device_name(hwaddr); + uint64_t off = qemu_plugin_hwaddr_device_offset(hwaddr); + bool is_write = qemu_plugin_mem_is_store(meminfo); + DeviceCounts *counts; + + g_mutex_lock(&lock); + counts = (DeviceCounts *) g_hash_table_lookup(devices, name); + + if (!counts) { + uint64_t base = vaddr - off; + counts = new_count(name, base); + } + + if (check_match) { + if (g_strv_contains((const char * const *)matches, counts->name)) { + hwprofile_match_hit(counts, off); + inc_count(&counts->totals, is_write, cpu_index); + } + } else { + inc_count(&counts->totals, is_write, cpu_index); + } + + /* either track offsets or source of access */ + if (source) { + off = (uint64_t) udata; + } + + if (pattern || source) { + IOLocationCounts *io_count = g_hash_table_lookup(counts->detail, + (gpointer) off); + if (!io_count) { + io_count = new_location(counts->detail, off); + } + inc_count(&io_count->counts, is_write, cpu_index); + } + + g_mutex_unlock(&lock); + } +} + +static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb) +{ + size_t n = qemu_plugin_tb_n_insns(tb); + size_t i; + + for (i = 0; i < n; i++) { + struct qemu_plugin_insn *insn = qemu_plugin_tb_get_insn(tb, i); + gpointer udata = (gpointer) (source ? qemu_plugin_insn_vaddr(insn) : 0); + qemu_plugin_register_vcpu_mem_cb(insn, vcpu_haddr, + QEMU_PLUGIN_CB_NO_REGS, + rw, udata); + } +} + +QEMU_PLUGIN_EXPORT +int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info, + int argc, char **argv) +{ + int i; + + for (i = 0; i < argc; i++) { + char *opt = argv[i]; + if (g_strcmp0(opt, "read") == 0) { + rw = QEMU_PLUGIN_MEM_R; + } else if (g_strcmp0(opt, "write") == 0) { + rw = QEMU_PLUGIN_MEM_W; + } else if (g_strcmp0(opt, "pattern") == 0) { + pattern = true; + } else if (g_strcmp0(opt, "source") == 0) { + source = true; + } else if (g_str_has_prefix(opt, "match")) { + gchar **parts = g_strsplit(opt, "=", 2); + check_match = true; + matches = g_strsplit(parts[1], ",", -1); + g_strfreev(parts); + } else { + fprintf(stderr, "option parsing failed: %s\n", opt); + return -1; + } + } + + if (source && pattern) { + fprintf(stderr, "can only currently track either source or pattern.\n"); + return -1; + } + + if (!info->system_emulation) { + fprintf(stderr, "hwprofile: plugin only useful for system emulation\n"); + return -1; + } + + /* Just warn about overflow */ + if (info->system.smp_vcpus > 64 || + info->system.max_vcpus > 64) { + fprintf(stderr, "hwprofile: can only track up to 64 CPUs\n"); + } + + plugin_init(); + + qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans); + qemu_plugin_register_atexit_cb(id, plugin_exit, NULL); + return 0; +} diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile index 3a50451428b4..0cb8e35ae407 100644 --- a/tests/plugin/Makefile +++ b/tests/plugin/Makefile @@ -14,6 +14,7 @@ NAMES += hotblocks NAMES += howvec NAMES += hotpages NAMES += lockstep +NAMES += hwprofile SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES))) diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target index 4b2b696fcee3..bc595b058845 100644 --- a/tests/tcg/Makefile.target +++ b/tests/tcg/Makefile.target @@ -129,8 +129,16 @@ ifeq ($(CONFIG_PLUGIN),y) PLUGIN_SRC=$(SRC_PATH)/tests/plugin PLUGIN_LIB=../../plugin VPATH+=$(PLUGIN_LIB) -PLUGINS=$(filter-out liblockstep.so,\ - $(patsubst %.c, lib%.so, $(notdir $(wildcard $(PLUGIN_SRC)/*.c)))) + +# Some plugins aren't testable automatically +SKIP_PLUGINS=liblockstep.so +ifdef CONFIG_USER_ONLY +SKIP_PLUGINS+=libhwprofile.so +endif + +PLUGINS=$(filter-out $(SKIP_PLUGINS),\ + $(patsubst %.c, lib%.so, \ + $(notdir $(wildcard $(PLUGIN_SRC)/*.c)))) # We need to ensure expand the run-plugin-TEST-with-PLUGIN # pre-requistes manually here as we can't use stems to handle it. We