From patchwork Thu Nov 19 17:53:03 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arnaldo Carvalho de Melo X-Patchwork-Id: 57033 Delivered-To: patch@linaro.org Received: by 10.112.180.98 with SMTP id dn2csp1754lbc; Thu, 19 Nov 2015 09:56:53 -0800 (PST) X-Received: by 10.68.203.233 with SMTP id kt9mr12634627pbc.39.1447955813409; Thu, 19 Nov 2015 09:56:53 -0800 (PST) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id ov1si13502938pbb.110.2015.11.19.09.56.53; Thu, 19 Nov 2015 09:56:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759216AbbKSR4t (ORCPT + 28 others); Thu, 19 Nov 2015 12:56:49 -0500 Received: from bombadil.infradead.org ([198.137.202.9]:52424 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758804AbbKSRxt (ORCPT ); Thu, 19 Nov 2015 12:53:49 -0500 Received: from 191-247-226-49.3g.claro.net.br ([191.247.226.49] helo=zoo.infradead.org) by bombadil.infradead.org with esmtpsa (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZzTOv-0004hz-VH; Thu, 19 Nov 2015 17:53:42 +0000 Received: by zoo.infradead.org (Postfix, from userid 1000) id AC4622202CC; Thu, 19 Nov 2015 14:53:28 -0300 (BRT) From: Arnaldo Carvalho de Melo To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Wang Nan , Alexei Starovoitov , Masami Hiramatsu , Zefan Li , pi3orama@163.com, Arnaldo Carvalho de Melo Subject: [PATCH 17/37] perf test: Print result for each LLVM subtest Date: Thu, 19 Nov 2015 14:53:03 -0300 Message-Id: <1447955603-24895-18-git-send-email-acme@kernel.org> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1447955603-24895-1-git-send-email-acme@kernel.org> References: <1447955603-24895-1-git-send-email-acme@kernel.org> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Wang Nan Currently 'perf test llvm' and 'perf test BPF' have multiple sub-tests, but the result is provided in only one line: # perf test LLVM 35: Test LLVM searching and compiling : Ok This patch introduces sub-tests support, allowing 'perf test' to report result for each sub-tests: # perf test LLVM 35: Test LLVM searching and compiling : 35.1: Basic BPF llvm compiling test : Ok 35.2: Test kbuild searching : Ok 35.3: Compile source for BPF prologue generation test : Ok When a failure happens: # cat ~/.perfconfig [llvm] clang-path = "/bin/false" # perf test LLVM 35: Test LLVM searching and compiling : 35.1: Basic BPF llvm compiling test : FAILED! 35.2: Test kbuild searching : Skip 35.3: Compile source for BPF prologue generation test : Skip And: # rm ~/.perfconfig # ./perf test LLVM 35: Test LLVM searching and compiling : 35.1: Basic BPF llvm compiling test : Skip 35.2: Test kbuild searching : Skip 35.3: Compile source for BPF prologue generation test : Skip Skip by user: # ./perf test -s 1,`seq -s , 3 42` 1: vmlinux symtab matches kallsyms : Skip (user override) 2: detect openat syscall event : Ok ... 35: Test LLVM searching and compiling : Skip (user override) ... Suggested-and-Tested-by: Arnaldo Carvalho de Melo Signed-off-by: Wang Nan Cc: Alexei Starovoitov Cc: Masami Hiramatsu Cc: Zefan Li Cc: pi3orama@163.com Link: http://lkml.kernel.org/r/1447749170-175898-4-git-send-email-wangnan0@huawei.com [ Changed so that func is not on an anonymous union ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/builtin-test.c | 91 ++++++++++++++++++++++++++++++++++------- tools/perf/tests/llvm.c | 65 ++++++++++++++--------------- tools/perf/tests/tests.h | 9 ++++ 3 files changed, 115 insertions(+), 50 deletions(-) -- 2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 9cf4892c061d..813660976217 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -160,6 +160,11 @@ static struct test generic_tests[] = { { .desc = "Test LLVM searching and compiling", .func = test__llvm, + .subtest = { + .skip_if_fail = true, + .get_nr = test__llvm_subtest_get_nr, + .get_desc = test__llvm_subtest_get_desc, + }, }, { .desc = "Test topology in session", @@ -237,6 +242,40 @@ static int run_test(struct test *test, int subtest) for (j = 0; j < ARRAY_SIZE(tests); j++) \ for (t = &tests[j][0]; t->func; t++) +static int test_and_print(struct test *t, bool force_skip, int subtest) +{ + int err; + + if (!force_skip) { + pr_debug("\n--- start ---\n"); + err = run_test(t, subtest); + pr_debug("---- end ----\n"); + } else { + pr_debug("\n--- force skipped ---\n"); + err = TEST_SKIP; + } + + if (!t->subtest.get_nr) + pr_debug("%s:", t->desc); + else + pr_debug("%s subtest %d:", t->desc, subtest); + + switch (err) { + case TEST_OK: + pr_info(" Ok\n"); + break; + case TEST_SKIP: + color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip\n"); + break; + case TEST_FAIL: + default: + color_fprintf(stderr, PERF_COLOR_RED, " FAILED!\n"); + break; + } + + return err; +} + static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist) { struct test *t; @@ -264,21 +303,43 @@ static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist) continue; } - pr_debug("\n--- start ---\n"); - err = run_test(t, i); - pr_debug("---- end ----\n%s:", t->desc); - - switch (err) { - case TEST_OK: - pr_info(" Ok\n"); - break; - case TEST_SKIP: - color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip\n"); - break; - case TEST_FAIL: - default: - color_fprintf(stderr, PERF_COLOR_RED, " FAILED!\n"); - break; + if (!t->subtest.get_nr) { + test_and_print(t, false, -1); + } else { + int subn = t->subtest.get_nr(); + /* + * minus 2 to align with normal testcases. + * For subtest we print additional '.x' in number. + * for example: + * + * 35: Test LLVM searching and compiling : + * 35.1: Basic BPF llvm compiling test : Ok + */ + int subw = width > 2 ? width - 2 : width; + bool skip = false; + int subi; + + if (subn <= 0) { + color_fprintf(stderr, PERF_COLOR_YELLOW, + " Skip (not compiled in)\n"); + continue; + } + pr_info("\n"); + + for (subi = 0; subi < subn; subi++) { + int len = strlen(t->subtest.get_desc(subi)); + + if (subw < len) + subw = len; + } + + for (subi = 0; subi < subn; subi++) { + pr_info("%2d.%1d: %-*s:", i, subi + 1, subw, + t->subtest.get_desc(subi)); + err = test_and_print(t, skip, subi); + if (err != TEST_OK && t->subtest.skip_if_fail) + skip = true; + } } } diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c index 4350c455d06c..06f45c1d4256 100644 --- a/tools/perf/tests/llvm.c +++ b/tools/perf/tests/llvm.c @@ -46,7 +46,7 @@ static struct { }, [LLVM_TESTCASE_BPF_PROLOGUE] = { .source = test_llvm__bpf_test_prologue_prog, - .desc = "Test BPF prologue generation", + .desc = "Compile source for BPF prologue generation test", }, }; @@ -131,44 +131,39 @@ out: return ret; } -int test__llvm(int subtest __maybe_unused) +int test__llvm(int subtest) { - enum test_llvm__testcase i; + int ret; + void *obj_buf = NULL; + size_t obj_buf_sz = 0; - for (i = 0; i < __LLVM_TESTCASE_MAX; i++) { - int ret; - void *obj_buf = NULL; - size_t obj_buf_sz = 0; + if ((subtest < 0) || (subtest >= __LLVM_TESTCASE_MAX)) + return TEST_FAIL; - ret = test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, - i, false); + ret = test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, + subtest, false); - if (ret == TEST_OK) { - ret = test__bpf_parsing(obj_buf, obj_buf_sz); - if (ret != TEST_OK) - pr_debug("Failed to parse test case '%s'\n", - bpf_source_table[i].desc); - } - free(obj_buf); - - switch (ret) { - case TEST_SKIP: - return TEST_SKIP; - case TEST_OK: - break; - default: - /* - * Test 0 is the basic LLVM test. If test 0 - * fail, the basic LLVM support not functional - * so the whole test should fail. If other test - * case fail, it can be fixed by adjusting - * config so don't report error. - */ - if (i == 0) - return TEST_FAIL; - else - return TEST_SKIP; + if (ret == TEST_OK) { + ret = test__bpf_parsing(obj_buf, obj_buf_sz); + if (ret != TEST_OK) { + pr_debug("Failed to parse test case '%s'\n", + bpf_source_table[subtest].desc); } } - return TEST_OK; + free(obj_buf); + + return ret; +} + +int test__llvm_subtest_get_nr(void) +{ + return __LLVM_TESTCASE_MAX; +} + +const char *test__llvm_subtest_get_desc(int subtest) +{ + if ((subtest < 0) || (subtest >= __LLVM_TESTCASE_MAX)) + return NULL; + + return bpf_source_table[subtest].desc; } diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 204e4eeadea2..f92af527f080 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -1,6 +1,8 @@ #ifndef TESTS_H #define TESTS_H +#include + #define TEST_ASSERT_VAL(text, cond) \ do { \ if (!(cond)) { \ @@ -27,6 +29,11 @@ enum { struct test { const char *desc; int (*func)(int subtest); + struct { + bool skip_if_fail; + int (*get_nr)(void); + const char *(*get_desc)(int subtest); + } subtest; }; /* Tests */ @@ -66,6 +73,8 @@ int test__fdarray__add(int subtest); int test__kmod_path__parse(int subtest); int test__thread_map(int subtest); int test__llvm(int subtest); +const char *test__llvm_subtest_get_desc(int subtest); +int test__llvm_subtest_get_nr(void); int test__bpf(int subtest); int test_session_topology(int subtest);