From patchwork Fri Jul 30 22:18:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63E27C19F33 for ; Fri, 30 Jul 2021 22:18:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4698B610A7 for ; Fri, 30 Jul 2021 22:18:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233809AbhG3WSu (ORCPT ); Fri, 30 Jul 2021 18:18:50 -0400 Received: from mail.kernel.org ([198.145.29.99]:51318 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233072AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 99F6660FED; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V16-CI; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 01/17] libtracefs: Added new API tracefs_sql() Date: Fri, 30 Jul 2021 18:18:08 -0400 Message-Id: <20210730221824.595597-2-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" This adds the API tracefs_sql() that takes a tep_handle handler, a name, and a SQL string and parses it to produce a tracefs_synth synthetic event handler. Currently it only supports simple SQL of the type: SELECT start.common_pid AS pid, end.common_timestamp.usecs AS usecs FROM sched_waking AS start on sched_switch AS end ON start.pid = end.next_pid Signed-off-by: Steven Rostedt (VMware) --- include/tracefs.h | 3 + src/Makefile | 11 + src/sqlhist-parse.h | 69 +++++ src/sqlhist.l | 88 ++++++ src/sqlhist.y | 143 +++++++++ src/tracefs-sqlhist.c | 691 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1005 insertions(+) create mode 100644 src/sqlhist-parse.h create mode 100644 src/sqlhist.l create mode 100644 src/sqlhist.y create mode 100644 src/tracefs-sqlhist.c diff --git a/include/tracefs.h b/include/tracefs.h index 246647f6496d..3d83604031b1 100644 --- a/include/tracefs.h +++ b/include/tracefs.h @@ -385,4 +385,7 @@ void tracefs_synth_free(struct tracefs_synth *synth); int tracefs_synth_show(struct trace_seq *seq, struct tracefs_instance *instance, struct tracefs_synth *synth); +struct tracefs_synth *tracefs_sql(struct tep_handle *tep, const char *name, + const char *sql_buffer, char **err); + #endif /* _TRACE_FS_H */ diff --git a/src/Makefile b/src/Makefile index 767af49034ee..eac404a1a147 100644 --- a/src/Makefile +++ b/src/Makefile @@ -12,6 +12,11 @@ OBJS += tracefs-kprobes.o OBJS += tracefs-hist.o OBJS += tracefs-filter.o +# Order matters for the the three below +OBJS += sqlhist-lex.o +OBJS += sqlhist.tab.o +OBJS += tracefs-sqlhist.o + OBJS := $(OBJS:%.o=$(bdir)/%.o) DEPS := $(OBJS:$(bdir)/%.o=$(bdir)/.%.d) @@ -32,6 +37,12 @@ $(LIBTRACEFS_SHARED_SO): $(LIBTRACEFS_SHARED_VERSION) libtracefs.so: $(LIBTRACEFS_SHARED_SO) +sqlhist.tab.c: sqlhist.y + bison --debug -v --report-file=bison.report -d -o $@ $^ + +sqlhist-lex.c: sqlhist.l sqlhist.tab.c + flex -o $@ $< + $(bdir)/%.o: %.c $(Q)$(call do_fpic_compile) diff --git a/src/sqlhist-parse.h b/src/sqlhist-parse.h new file mode 100644 index 000000000000..aa5232eea451 --- /dev/null +++ b/src/sqlhist-parse.h @@ -0,0 +1,69 @@ +#ifndef __SQLHIST_PARSE_H +#define __SQLHIST_PARSE_H + +#include +#include + +struct str_hash; +#define HASH_BITS 10 + +struct sql_table; + +struct sqlhist_bison { + const char *buffer; + size_t buffer_size; + size_t buffer_idx; + int line_no; + int line_idx; + struct sql_table *table; + char *parse_error_str; + struct str_hash *str_hash[1 << HASH_BITS]; +}; + +extern struct sqlhist_bison *sb; + +#include "sqlhist.tab.h" + +enum filter_type { + FILTER_GROUP, + FILTER_NOT_GROUP, + FILTER_EQ, + FILTER_NE, + FILTER_LE, + FILTER_LT, + FILTER_GE, + FILTER_GT, + FILTER_BIN_AND, + FILTER_STR_CMP, + FILTER_AND, + FILTER_OR, +}; + +enum compare_type { + COMPARE_GROUP, + COMPARE_ADD, + COMPARE_SUB, + COMPARE_MUL, + COMPARE_DIV, + COMPARE_BIN_AND, + COMPARE_BIN_OR, + COMPARE_AND, + COMPARE_OR, +}; + +char * store_str(struct sqlhist_bison *sb, const char *str); + +int table_start(struct sqlhist_bison *sb); + +void *add_field(struct sqlhist_bison *sb, const char *field, const char *label); + +int add_match(struct sqlhist_bison *sb, void *A, void *B); + +int add_selection(struct sqlhist_bison *sb, void *item, const char *label); +int add_from(struct sqlhist_bison *sb, void *item); +int add_to(struct sqlhist_bison *sb, void *item); + +extern void sql_parse_error(struct sqlhist_bison *sb, const char *text, + const char *fmt, va_list ap); + +#endif diff --git a/src/sqlhist.l b/src/sqlhist.l new file mode 100644 index 000000000000..3f394f37b738 --- /dev/null +++ b/src/sqlhist.l @@ -0,0 +1,88 @@ +%{ +/* code here */ + +#include +#include "sqlhist-parse.h" + +extern int my_yyinput(char *buf, int max); + +#undef YY_INPUT +#define YY_INPUT(b, r, m) ({r = my_yyinput(b, m);}) + +#define YY_NO_INPUT +#define YY_NO_UNPUT + +#define YY_EXTRA_TYPE struct sqlhist_bison * + +#define HANDLE_COLUMN do { sb->line_idx += strlen(yytext); } while (0) + +%} + +%option caseless + +field [a-z_][a-z0-9_\.]* +qstring \"[^\"]*\" +hexnum 0x[0-9a-z]+ +number [0-9a-z]+ + +%% + +select { HANDLE_COLUMN; return SELECT; } +as { HANDLE_COLUMN; return AS; } +from { HANDLE_COLUMN; return FROM; } +join { HANDLE_COLUMN; return JOIN; } +on { HANDLE_COLUMN; return ON; } + +{qstring} { + HANDLE_COLUMN; + yylval.string = store_str(sb, yytext); + return STRING; +} + +{field} { + HANDLE_COLUMN; + yylval.string = store_str(sb, yytext); + return FIELD; +} + +{hexnum} { + HANDLE_COLUMN; + yylval.number = strtol(yytext, NULL, 0); + return NUMBER; +} + +{number} { + HANDLE_COLUMN; + yylval.number = strtol(yytext, NULL, 0); + return NUMBER; +} + +\!= { HANDLE_COLUMN; return NEQ; } +\<= { HANDLE_COLUMN; return LE; } +\>= { HANDLE_COLUMN; return GE; } +== { HANDLE_COLUMN; return EQ; } +&& { HANDLE_COLUMN; return AND; } +"||" { HANDLE_COLUMN; return OR; } +[<>&~] { HANDLE_COLUMN; return yytext[0]; } + +[\!()\-\+\*/,=] { HANDLE_COLUMN; return yytext[0]; } + +[ \t] { HANDLE_COLUMN; } +\n { sb->line_idx = 0; sb->line_no++; } + +. { HANDLE_COLUMN; return PARSE_ERROR; } +%% + +int yywrap(void) +{ + return 1; +} + +void yyerror(const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + sql_parse_error(sb, yytext, fmt, ap); + va_end(ap); +} diff --git a/src/sqlhist.y b/src/sqlhist.y new file mode 100644 index 000000000000..ecb0a0ed44b3 --- /dev/null +++ b/src/sqlhist.y @@ -0,0 +1,143 @@ +%{ +#include +#include +#include +#include + +#include "sqlhist-parse.h" + +extern int yylex(void); +extern void yyerror(char *fmt, ...); + +#define CHECK_RETURN_PTR(x) \ + do { \ + if (!(x)) { \ + printf("FAILED MEMORY: %s\n", #x); \ + return -ENOMEM; \ + } \ + } while (0) + +#define CHECK_RETURN_VAL(x) \ + do { \ + if ((x) < 0) { \ + printf("FAILED MEMORY: %s\n", #x); \ + return -ENOMEM; \ + } \ + } while (0) + +%} + +%union { + int s32; + char *string; + long number; + void *expr; +} + +%token AS SELECT FROM JOIN ON PARSE_ERROR +%token NUMBER +%token STRING +%token FIELD +%token LE GE EQ NEQ AND OR + +%left '+' '-' +%left '*' '/' +%left '<' '>' +%left AND OR + +%type name label + +%type selection_expr field item named_field join_clause + +%% + +start : + select_statement + ; + +label : AS name { CHECK_RETURN_PTR($$ = store_str(sb, $2)); } + | name { CHECK_RETURN_PTR($$ = store_str(sb, $1)); } + ; + +select : SELECT { table_start(sb); } + ; + +select_statement : + select selection_list table_exp + ; + +selection_list : + selection + | selection ',' selection_list + ; + +selection : + selection_expr + { + CHECK_RETURN_VAL(add_selection(sb, $1, NULL)); + } + | selection_expr label + { + CHECK_RETURN_VAL(add_selection(sb, $1, $2)); + } + ; + +selection_expr : + field + | '(' field ')' { $$ = $2; } + ; + +item : + named_field + | field + ; + +field : + FIELD { $$ = add_field(sb, $1, NULL); CHECK_RETURN_PTR($$); } + ; + +named_field : + FIELD label { $$ = add_field(sb, $1, $2); CHECK_RETURN_PTR($$); } + ; + +name : + FIELD + ; + +table_exp : + from_clause join_clause + ; + +from_clause : + FROM item { CHECK_RETURN_VAL(add_from(sb, $2)); } + +/* + * Select from a from clause confuses the variable parsing. + * disable it for now. + + | FROM '(' select_statement ')' label + { + from_table_end($5); + $$ = store_printf("FROM (%s) AS %s", $3, $5); + } +*/ + ; + +join_clause : + JOIN item ON match_clause { add_to(sb, $2); } + ; + +match : + item '=' item { CHECK_RETURN_VAL(add_match(sb, $1, $3)); } + | item EQ item { CHECK_RETURN_VAL(add_match(sb, $1, $3)); } + + ; + +match_clause : + match + | match ',' match_clause + ; + +%% + + diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c new file mode 100644 index 000000000000..7ec0c11b902e --- /dev/null +++ b/src/tracefs-sqlhist.c @@ -0,0 +1,691 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021 VMware Inc, Steven Rostedt + * + * Updates: + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov + * + */ +#include +#include +#include + +#include "tracefs.h" +#include "tracefs-local.h" +#include "sqlhist-parse.h" + +struct sqlhist_bison *sb; + +extern int yylex_init(void* ptr_yy_globals); +extern int yylex_init_extra(struct sqlhist_bison *sb, void* ptr_yy_globals); +extern int yylex_destroy (void * yyscanner ); + +struct str_hash { + struct str_hash *next; + char *str; +}; + +enum alias_type { + ALIAS_EVENT, + ALIAS_FIELD, +}; + +#define for_each_field(expr, field, table) \ + for (expr = (table)->fields; expr; expr = (field)->next) + +struct field { + struct expr *next; /* private link list */ + const char *system; + const char *event; + const char *raw; + const char *label; + const char *field; +}; + +struct match { + struct match *next; + struct expr *lval; + struct expr *rval; +}; + +enum expr_type +{ + EXPR_NUMBER, + EXPR_STRING, + EXPR_FIELD, +}; + +struct expr { + struct expr *free_list; + struct expr *next; + enum expr_type type; + union { + struct field field; + const char *string; + long number; + }; +}; + +struct sql_table { + struct sqlhist_bison *sb; + const char *name; + struct expr *exprs; + struct expr *fields; + struct expr *from; + struct expr *to; + struct match *matches; + struct match **next_match; + struct expr *selections; + struct expr **next_selection; +}; + +__hidden int my_yyinput(char *buf, int max) +{ + if (!sb || !sb->buffer) + return -1; + + if (sb->buffer_idx + max > sb->buffer_size) + max = sb->buffer_size - sb->buffer_idx; + + if (max) + memcpy(buf, sb->buffer + sb->buffer_idx, max); + + sb->buffer_idx += max; + + return max; +} + +__hidden void sql_parse_error(struct sqlhist_bison *sb, const char *text, + const char *fmt, va_list ap) +{ + const char *buffer = sb->buffer; + struct trace_seq s; + int line = sb->line_no; + int idx = sb->line_idx - strlen(text); + int i; + + if (!buffer) + return; + + trace_seq_init(&s); + if (!s.buffer) { + fprintf(stderr, "Error allocating internal buffer\n"); + return; + } + + for (i = 0; line && buffer[i]; i++) { + if (buffer[i] == '\n') + line--; + } + for (; buffer[i] && buffer[i] != '\n'; i++) + trace_seq_putc(&s, buffer[i]); + trace_seq_putc(&s, '\n'); + for (i = idx; i > 0; i--) + trace_seq_putc(&s, ' '); + trace_seq_puts(&s, "^\n"); + trace_seq_printf(&s, "ERROR: '%s'\n", text); + trace_seq_vprintf(&s, fmt, ap); + + trace_seq_terminate(&s); + + sb->parse_error_str = strdup(s.buffer); + trace_seq_destroy(&s); +} + +static inline unsigned int quick_hash(const char *str) +{ + unsigned int val = 0; + int len = strlen(str); + + for (; len >= 4; str += 4, len -= 4) { + val += str[0]; + val += str[1] << 8; + val += str[2] << 16; + val += str[3] << 24; + } + for (; len > 0; str++, len--) + val += str[0] << (len * 8); + + val *= 2654435761; + + return val & ((1 << HASH_BITS) - 1); +} + + +static struct str_hash *find_string(struct sqlhist_bison *sb, const char *str) +{ + unsigned int key = quick_hash(str); + struct str_hash *hash = sb->str_hash[key]; + + for (; hash; hash = hash->next) { + if (!strcmp(hash->str, str)) + return hash; + } + return NULL; +} + +/* + * If @str is found, then return the hash string. + * This lets store_str() know to free str. + */ +static char **add_hash(struct sqlhist_bison *sb, const char *str) +{ + struct str_hash *hash; + unsigned int key; + + if ((hash = find_string(sb, str))) { + return &hash->str; + } + + hash = malloc(sizeof(*hash)); + if (!hash) + return NULL; + key = quick_hash(str); + hash->next = sb->str_hash[key]; + sb->str_hash[key] = hash; + hash->str = NULL; + return &hash->str; +} + +__hidden char *store_str(struct sqlhist_bison *sb, const char *str) +{ + char **pstr = add_hash(sb, str); + + if (!pstr) + return NULL; + + if (!(*pstr)) + *pstr = strdup(str); + + return *pstr; +} + +__hidden int add_selection(struct sqlhist_bison *sb, void *select, + const char *name) +{ + struct sql_table *table = sb->table; + struct expr *expr = select; + + switch (expr->type) { + case EXPR_FIELD: + break; + case EXPR_NUMBER: + case EXPR_STRING: + default: + return -1; + } + + if (expr->next) + return -1; + + *table->next_selection = expr; + table->next_selection = &expr->next; + + return 0; +} + +static struct expr *find_field(struct sqlhist_bison *sb, + const char *raw, const char *label) +{ + struct field *field; + struct expr *expr; + + for_each_field(expr, field, sb->table) { + field = &expr->field; + + if (!strcmp(field->raw, raw)) { + if (label && !field->label) + field->label = label; + return expr; + } + + if (label && !strcmp(field->raw, label)) { + if (!field->label) { + field->label = label; + field->raw = raw; + } + return expr; + } + + if (!field->label) + continue; + + if (!strcmp(field->label, raw)) + return expr; + + if (label && !strcmp(field->label, label)) + return expr; + } + return NULL; +} + +static void *create_expr(enum expr_type type, struct expr **expr_p) +{ + struct expr *expr; + + expr = calloc(1, sizeof(*expr)); + if (!expr) + return NULL; + + if (expr_p) + *expr_p = expr; + + expr->free_list = sb->table->exprs; + sb->table->exprs = expr; + + expr->type = type; + + switch (type) { + case EXPR_FIELD: return &expr->field; + case EXPR_NUMBER: return &expr->number; + case EXPR_STRING: return &expr->string; + } + + return NULL; +} + +#define __create_expr(var, type, ENUM, expr) \ + do { \ + var = (type *)create_expr(EXPR_##ENUM, expr); \ + } while(0) + +#define create_field(var, expr) \ + __create_expr(var, struct field, FIELD, expr) + +__hidden void *add_field(struct sqlhist_bison *sb, + const char *field_name, const char *label) +{ + struct sql_table *table = sb->table; + struct expr *expr; + struct field *field; + + expr = find_field(sb, field_name, label); + if (expr) + return expr; + + create_field(field, &expr); + + field->next = table->fields; + table->fields = expr; + + field->raw = field_name; + field->label = label; + + return expr; +} + +__hidden int add_match(struct sqlhist_bison *sb, void *A, void *B) +{ + struct sql_table *table = sb->table; + struct match *match; + + match = calloc(1, sizeof(*match)); + if (!match) + return -1; + + match->lval = A; + match->rval = B; + + *table->next_match = match; + table->next_match = &match->next; + + return 0; +} + +__hidden int add_from(struct sqlhist_bison *sb, void *item) +{ + struct expr *expr = item; + + if (expr->type != EXPR_FIELD) + return -1; + + sb->table->from = expr; + + return 0; +} + +__hidden int add_to(struct sqlhist_bison *sb, void *item) +{ + struct expr *expr = item; + + if (expr->type != EXPR_FIELD) + return -1; + + sb->table->to = expr; + + return 0; +} + +__hidden int table_start(struct sqlhist_bison *sb) +{ + struct sql_table *table; + + table = calloc(1, sizeof(*table)); + if (!table) + return -ENOMEM; + + table->sb = sb; + sb->table = table; + + table->next_match = &table->matches; + table->next_selection = &table->selections; + + return 0; +} + +static int update_vars(struct sql_table *table, struct field *event) +{ + struct sqlhist_bison *sb = table->sb; + struct expr *expr; + struct field *field; + const char *label; + const char *p, *r; + char *system; + int len; + + p = strchr(event->raw, '.'); + if (p) { + system = strndup(event->raw, p - event->raw); + if (!system) + return -1; + event->system = store_str(sb, system); + free(system); + if (!event->system) + return -1; + p++; + } else { + p = event->raw; + } + + event->event = store_str(sb, p); + if (!event->event) + return -1; + + if (!event->label) + event->label = event->event; + + label = event->label; + len = strlen(label); + + for_each_field(expr, field, table) { + field = &expr->field; + + if (field->event) + continue; + + p = strchr(field->raw, '.'); + if (p) { + /* Does this field have a system */ + r = strchr(p + 1, '.'); + if (r) { + /* This has a system, and is not a alias */ + system = strndup(field->raw, p - field->raw); + if (!system) + return -1; + field->system = store_str(sb, system); + free(system); + if (!field->system) + return -1; + + /* save the event as well */ + p++; + system = strndup(p, r - p); + if (!system) + return -1; + field->event = store_str(sb, system); + free(system); + if (!field->event) + return -1; + r++; + field->field = store_str(sb, r); + goto check_timestamps; + } + } + + if (strncmp(field->raw, label, len)) + continue; + + if (field->raw[len] != '.') + continue; + + field->system = event->system; + field->event = event->event; + field->field = field->raw + len + 1; + check_timestamps: + if (!strcmp(field->field, "TIMESTAMP")) + field->field = store_str(sb, TRACEFS_TIMESTAMP); + else if (!strcmp(field->field, "TIMESTAMP_USECS")) + field->field = store_str(sb, TRACEFS_TIMESTAMP_USECS); + } + + return 0; +} + +static int test_match(struct sql_table *table, struct match *match) +{ + struct field *lval, *rval; + struct field *to, *from; + + if (!match->lval || !match->rval) + return -1; + + if (match->lval->type != EXPR_FIELD || match->rval->type != EXPR_FIELD) + return -1; + + to = &table->to->field; + from = &table->from->field; + + lval = &match->lval->field; + rval = &match->rval->field; + + /* + * Note, strings are stored in the string store, so all + * duplicate strings are the same value, and we can use + * normal "==" and "!=" instead of strcmp(). + * + * Either lval == to and rval == from + * or lval == from and rval == to. + */ + if ((lval->system != to->system) || + (lval->event != to->event)) { + if ((rval->system != to->system) || + (rval->event != to->event) || + (lval->system != from->system) || + (lval->event != from->event)) + return -1; + } else { + if ((rval->system != from->system) || + (rval->event != from->event) || + (lval->system != to->system) || + (lval->event != to->event)) + return -1; + } + return 0; +} + +static void assign_match(const char *system, const char *event, + struct match *match, + const char **start_match, const char **end_match) +{ + struct field *lval, *rval; + + lval = &match->lval->field; + rval = &match->rval->field; + + if (lval->system == system && + lval->event == event) { + *start_match = lval->field; + *end_match = rval->field; + } else { + *start_match = rval->field; + *end_match = lval->field; + } +} + +static struct tracefs_synth *build_synth(struct tep_handle *tep, + const char *name, + struct sql_table *table) +{ + struct tracefs_synth *synth; + struct field *field; + struct match *match; + struct expr *expr; + const char *start_system; + const char *start_event; + const char *end_system; + const char *end_event; + const char *start_match; + const char *end_match; + int ret; + + if (!table->to || !table->from) + return NULL; + + ret = update_vars(table, &table->to->field); + if (ret < 0) + return NULL; + + ret = update_vars(table, &table->from->field); + if (ret < 0) + return NULL; + + match = table->matches; + if (!match) + return NULL; + + ret = test_match(table, match); + if (ret < 0) + return NULL; + + start_system = table->from->field.system; + start_event = table->from->field.event; + + end_system = table->to->field.system; + end_event = table->to->field.event; + + assign_match(start_system, start_event, match, + &start_match, &end_match); + + synth = tracefs_synth_init(tep, name, start_system, + start_event, end_system, end_event, + start_match, end_match, NULL); + if (!synth) + return NULL; + + for (match = match->next; match; match = match->next) { + ret = test_match(table, match); + if (ret < 0) + goto free; + + assign_match(start_system, start_event, match, + &start_match, &end_match); + + ret = tracefs_synth_add_match_field(synth, + start_match, + end_match, NULL); + if (ret < 0) + goto free; + } + + for (expr = table->selections; expr; expr = expr->next) { + if (expr->type == EXPR_FIELD) { + field = &expr->field; + if (field->system == start_system && + field->event == start_event) { + ret = tracefs_synth_add_start_field(synth, + field->field, field->label); + } else { + ret = tracefs_synth_add_end_field(synth, + field->field, field->label); + } + if (ret < 0) + goto free; + continue; + } + goto free; + } + + return synth; + free: + tracefs_synth_free(synth); + return NULL; +} + +static void free_sql_table(struct sql_table *table) +{ + struct match *match; + struct expr *expr; + + if (!table) + return; + + while ((expr = table->exprs)) { + table->exprs = expr->next; + free(expr); + } + + while ((match = table->matches)) { + table->matches = match->next; + free(match); + } + + free(table); +} + +static void free_str_hash(struct str_hash **hash) +{ + struct str_hash *item; + int i; + + for (i = 0; i < 1 << HASH_BITS; i++) { + while ((item = hash[i])) { + hash[i] = item->next; + free(item->str); + free(item); + } + } +} + +static void free_sb(struct sqlhist_bison *sb) +{ + free_sql_table(sb->table); + free_str_hash(sb->str_hash); + free(sb->parse_error_str); +} + +struct tracefs_synth *tracefs_sql(struct tep_handle *tep, const char *name, + const char *sql_buffer, char **err) +{ + struct sqlhist_bison local_sb; + struct tracefs_synth *synth = NULL; + int ret; + + if (!tep || !sql_buffer) { + errno = EINVAL; + return NULL; + } + + memset(&local_sb, 0, sizeof(local_sb)); + + local_sb.buffer = sql_buffer; + local_sb.buffer_size = strlen(sql_buffer); + local_sb.buffer_idx = 0; + + sb = &local_sb; + ret = yyparse(); + + if (ret) + goto free; + + synth = build_synth(tep, name, sb->table); + + free: + if (!synth) { + if (sb->parse_error_str && err) { + *err = sb->parse_error_str; + sb->parse_error_str = NULL; + } + } + free_sb(sb); + return synth; +} From patchwork Fri Jul 30 22:18:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23531C19F33 for ; Fri, 30 Jul 2021 22:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09A94610A0 for ; Fri, 30 Jul 2021 22:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233247AbhG3WSl (ORCPT ); Fri, 30 Jul 2021 18:18:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:51308 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232746AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A0FD561052; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V18-DX; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 02/17] tracefs: Add unit tests for tracefs_sql() Date: Fri, 30 Jul 2021 18:18:09 -0400 Message-Id: <20210730221824.595597-3-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add basic unit test for tracefs_sql()... Signed-off-by: Steven Rostedt (VMware) --- utest/tracefs-utest.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/utest/tracefs-utest.c b/utest/tracefs-utest.c index 2a99217b938a..89bb1cce61f7 100644 --- a/utest/tracefs-utest.c +++ b/utest/tracefs-utest.c @@ -44,6 +44,12 @@ #define KRETPROBE_ADDR "do_sys_openat2" #define KRETPROBE_FMT "ret=$retval" +#define SQL_1_EVENT "wakeup_1" +#define SQL_1_SQL "select sched_switch.next_pid as woke_pid, sched_waking.common_pid as waking_pid from sched_waking join sched_switch on sched_switch.next_pid = sched_waking.pid" + +#define SQL_2_EVENT "wakeup_2" +#define SQL_2_SQL "select woke.next_pid as woke_pid, wake.common_pid as waking_pid from sched_waking as wake join sched_switch as woke on woke.next_pid = wake.pid" + static struct tracefs_instance *test_instance; static struct tep_handle *test_tep; struct test_sample { @@ -322,6 +328,41 @@ static void test_ftrace_marker(void) test_instance_ftrace_marker(test_instance); } +static void test_instance_trace_sql(struct tracefs_instance *instance) +{ + struct tracefs_synth *synth; + struct trace_seq seq; + struct tep_handle *tep; + int ret; + + tep = tracefs_local_events(NULL); + CU_TEST(tep != NULL); + + trace_seq_init(&seq); + + synth = tracefs_sql(tep, SQL_1_EVENT, SQL_1_SQL, NULL); + CU_TEST(synth != NULL); + ret = tracefs_synth_show(&seq, instance, synth); + CU_TEST(ret == 0); + tracefs_synth_free(synth); + trace_seq_reset(&seq); + + synth = tracefs_sql(tep, SQL_2_EVENT, SQL_2_SQL, NULL); + CU_TEST(synth != NULL); + ret = tracefs_synth_show(&seq, instance, synth); + CU_TEST(ret == 0); + tracefs_synth_free(synth); + trace_seq_reset(&seq); + + tep_free(tep); + trace_seq_destroy(&seq); +} + +static void test_trace_sql(void) +{ + test_instance_trace_sql(test_instance); +} + static void test_trace_file(void) { const char *tmp = get_rand_str(); @@ -1338,6 +1379,8 @@ void test_tracefs_lib(void) fprintf(stderr, "Suite \"%s\" cannot be ceated\n", TRACEFS_SUITE); return; } + CU_add_test(suite, "trace sql", + test_trace_sql); CU_add_test(suite, "tracing file / directory APIs", test_trace_file); CU_add_test(suite, "instance file / directory APIs", From patchwork Fri Jul 30 22:18:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE910C00143 for ; Fri, 30 Jul 2021 22:18:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D34F4610A6 for ; Fri, 30 Jul 2021 22:18:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233487AbhG3WSp (ORCPT ); Fri, 30 Jul 2021 18:18:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:51318 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232817AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9A0F561019; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1A-EH; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 03/17] libtracefs: Add comparing start and end fields in tracefs_sql() Date: Fri, 30 Jul 2021 18:18:10 -0400 Message-Id: <20210730221824.595597-4-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add comparing a field and showing the differences between start and end for tracefs_sql(). For example: SELECT (end.common_timestamp.usecs - start.common_timestamp.usecs) AS lat FROM sched_waking AS start JOIN sched_switch AS end ON start.pid = stop.next_pid Signed-off-by: Steven Rostedt (VMware) --- src/sqlhist-parse.h | 4 ++ src/sqlhist.y | 19 ++++++++++ src/tracefs-sqlhist.c | 85 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 107 insertions(+), 1 deletion(-) diff --git a/src/sqlhist-parse.h b/src/sqlhist-parse.h index aa5232eea451..ebf4f61b5461 100644 --- a/src/sqlhist-parse.h +++ b/src/sqlhist-parse.h @@ -57,7 +57,11 @@ int table_start(struct sqlhist_bison *sb); void *add_field(struct sqlhist_bison *sb, const char *field, const char *label); +void *add_filter(struct sqlhist_bison *sb, void *A, void *B, enum filter_type op); + int add_match(struct sqlhist_bison *sb, void *A, void *B); +void *add_compare(struct sqlhist_bison *sb, void *A, void *B, enum compare_type type); +int add_where(struct sqlhist_bison *sb, void *expr); int add_selection(struct sqlhist_bison *sb, void *item, const char *label); int add_from(struct sqlhist_bison *sb, void *item); diff --git a/src/sqlhist.y b/src/sqlhist.y index ecb0a0ed44b3..f92c93ed5ecd 100644 --- a/src/sqlhist.y +++ b/src/sqlhist.y @@ -48,6 +48,10 @@ extern void yyerror(char *fmt, ...); %type name label %type selection_expr field item named_field join_clause +%type selection_addition +%type compare compare_list compare_cmds compare_items +%type compare_and_or +%type str_val val %% @@ -85,6 +89,21 @@ selection : selection_expr : field | '(' field ')' { $$ = $2; } + | selection_addition + | '(' selection_addition ')' { $$ = $2; } + ; + +selection_addition : + field '+' field + { + $$ = add_compare(sb, $1, $3, COMPARE_ADD); + CHECK_RETURN_PTR($$); + } + | field '-' field + { + $$ = add_compare(sb, $1, $3, COMPARE_SUB); + CHECK_RETURN_PTR($$); + } ; item : diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index 7ec0c11b902e..cf2661773679 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -42,17 +42,32 @@ struct field { const char *field; }; +struct filter { + enum filter_type type; + struct expr *lval; + struct expr *rval; +}; + struct match { struct match *next; struct expr *lval; struct expr *rval; }; +struct compare { + enum compare_type type; + struct expr *lval; + struct expr *rval; + const char *name; +}; + enum expr_type { EXPR_NUMBER, EXPR_STRING, EXPR_FIELD, + EXPR_FILTER, + EXPR_COMPARE, }; struct expr { @@ -61,6 +76,8 @@ struct expr { enum expr_type type; union { struct field field; + struct filter filter; + struct compare compare; const char *string; long number; }; @@ -209,8 +226,14 @@ __hidden int add_selection(struct sqlhist_bison *sb, void *select, switch (expr->type) { case EXPR_FIELD: break; + case EXPR_COMPARE: + if (!name) + return -1; + expr->compare.name = name; + break; case EXPR_NUMBER: case EXPR_STRING: + case EXPR_FILTER: default: return -1; } @@ -277,8 +300,10 @@ static void *create_expr(enum expr_type type, struct expr **expr_p) switch (type) { case EXPR_FIELD: return &expr->field; + case EXPR_COMPARE: return &expr->compare; case EXPR_NUMBER: return &expr->number; case EXPR_STRING: return &expr->string; + case EXPR_FILTER: return &expr->filter; } return NULL; @@ -292,6 +317,9 @@ static void *create_expr(enum expr_type type, struct expr **expr_p) #define create_field(var, expr) \ __create_expr(var, struct field, FIELD, expr) +#define create_compare(var, expr) \ + __create_expr(var, struct compare, COMPARE, expr) + __hidden void *add_field(struct sqlhist_bison *sb, const char *field_name, const char *label) { @@ -331,6 +359,21 @@ __hidden int add_match(struct sqlhist_bison *sb, void *A, void *B) return 0; } +__hidden void *add_compare(struct sqlhist_bison *sb, + void *A, void *B, enum compare_type type) +{ + struct compare *compare; + struct expr *expr; + + create_compare(compare, &expr); + + compare = &expr->compare; + compare->lval = A; + compare->rval = B; + compare->type = type; + + return expr; +} __hidden int add_from(struct sqlhist_bison *sb, void *item) { @@ -522,6 +565,39 @@ static void assign_match(const char *system, const char *event, } } +static int build_compare(struct tracefs_synth *synth, + const char *system, const char *event, + struct compare *compare) +{ + const char *start_field; + const char *end_field; + struct field *lval, *rval; + enum tracefs_synth_calc calc; + int ret; + + lval = &compare->lval->field; + rval = &compare->rval->field; + + if (lval->system == system && + lval->event == event) { + start_field = lval->field; + end_field = rval->field; + calc = TRACEFS_SYNTH_DELTA_START; + } else { + start_field = rval->field; + end_field = lval->field; + calc = TRACEFS_SYNTH_DELTA_END; + } + + if (compare->type == COMPARE_ADD) + calc = TRACEFS_SYNTH_ADD; + + ret = tracefs_synth_add_compare_field(synth, start_field, + end_field, calc, + compare->name); + return ret; +} + static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *name, struct sql_table *table) @@ -602,7 +678,14 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, goto free; continue; } - goto free; + + if (expr->type != EXPR_COMPARE) + goto free; + + ret = build_compare(synth, start_system, end_system, + &expr->compare); + if (ret < 0) + goto free; } return synth; From patchwork Fri Jul 30 22:18:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B720DC4320A for ; Fri, 30 Jul 2021 22:19:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A127D610A7 for ; Fri, 30 Jul 2021 22:19:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233187AbhG3WTY (ORCPT ); Fri, 30 Jul 2021 18:19:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:51328 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232956AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B436361077; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1C-F5; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 04/17] libtracefs: Add unit test to test tracefs_sql() compare Date: Fri, 30 Jul 2021 18:18:11 -0400 Message-Id: <20210730221824.595597-5-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add a test to test passing time from sched_waking to sched_switch to show wake up latency. Signed-off-by: Steven Rostedt (VMware) --- utest/tracefs-utest.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/utest/tracefs-utest.c b/utest/tracefs-utest.c index 89bb1cce61f7..f4e1a6b90a46 100644 --- a/utest/tracefs-utest.c +++ b/utest/tracefs-utest.c @@ -50,6 +50,9 @@ #define SQL_2_EVENT "wakeup_2" #define SQL_2_SQL "select woke.next_pid as woke_pid, wake.common_pid as waking_pid from sched_waking as wake join sched_switch as woke on woke.next_pid = wake.pid" +#define SQL_3_EVENT "wakeup_lat" +#define SQL_3_SQL "select start.pid, end.next_prio as prio, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start join sched_switch as end on start.pid = end.next_pid" + static struct tracefs_instance *test_instance; static struct tep_handle *test_tep; struct test_sample { @@ -354,6 +357,13 @@ static void test_instance_trace_sql(struct tracefs_instance *instance) tracefs_synth_free(synth); trace_seq_reset(&seq); + synth = tracefs_sql(tep, SQL_3_EVENT, SQL_3_SQL, NULL); + CU_TEST(synth != NULL); + ret = tracefs_synth_show(&seq, instance, synth); + CU_TEST(ret == 0); + tracefs_synth_free(synth); + trace_seq_reset(&seq); + tep_free(tep); trace_seq_destroy(&seq); } From patchwork Fri Jul 30 22:18:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE134C00143 for ; Fri, 30 Jul 2021 22:18:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8BB9610A6 for ; Fri, 30 Jul 2021 22:18:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234012AbhG3WSy (ORCPT ); Fri, 30 Jul 2021 18:18:54 -0400 Received: from mail.kernel.org ([198.145.29.99]:51340 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233028AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B215661059; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1I-Fp; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 05/17] libtracefs: Add filtering for start and end events in tracefs_sql() Date: Fri, 30 Jul 2021 18:18:12 -0400 Message-Id: <20210730221824.595597-6-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Allow the start and end events to have filters with the "WHERE" clause. For example: SELECT (end.common_timestamp.usecs - start.common_timestamp.usecs) AS lat FROM sched_waking AS start JOIN sched_switch AS end ON start.pid = stop.next_pid WHERE start.prio < 100 && end.prev_prio < 100 Signed-off-by: Steven Rostedt (VMware) --- src/sqlhist-parse.h | 3 + src/sqlhist.l | 1 + src/sqlhist.y | 67 ++++++++++- src/tracefs-sqlhist.c | 260 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 329 insertions(+), 2 deletions(-) diff --git a/src/sqlhist-parse.h b/src/sqlhist-parse.h index ebf4f61b5461..0933bfe9a574 100644 --- a/src/sqlhist-parse.h +++ b/src/sqlhist-parse.h @@ -67,6 +67,9 @@ int add_selection(struct sqlhist_bison *sb, void *item, const char *label); int add_from(struct sqlhist_bison *sb, void *item); int add_to(struct sqlhist_bison *sb, void *item); +void *add_string(struct sqlhist_bison *sb, const char *str); +void *add_number(struct sqlhist_bison *sb, long val); + extern void sql_parse_error(struct sqlhist_bison *sb, const char *text, const char *fmt, va_list ap); diff --git a/src/sqlhist.l b/src/sqlhist.l index 3f394f37b738..476f7fd1f1ec 100644 --- a/src/sqlhist.l +++ b/src/sqlhist.l @@ -32,6 +32,7 @@ as { HANDLE_COLUMN; return AS; } from { HANDLE_COLUMN; return FROM; } join { HANDLE_COLUMN; return JOIN; } on { HANDLE_COLUMN; return ON; } +where { HANDLE_COLUMN; return WHERE; } {qstring} { HANDLE_COLUMN; diff --git a/src/sqlhist.y b/src/sqlhist.y index f92c93ed5ecd..8dcc824bb9f1 100644 --- a/src/sqlhist.y +++ b/src/sqlhist.y @@ -34,7 +34,7 @@ extern void yyerror(char *fmt, ...); void *expr; } -%token AS SELECT FROM JOIN ON PARSE_ERROR +%token AS SELECT FROM JOIN ON WHERE PARSE_ERROR %token NUMBER %token STRING %token FIELD @@ -123,8 +123,71 @@ name : FIELD ; +str_val : + STRING { $$ = add_string(sb, $1); CHECK_RETURN_PTR($$); } + ; + +val : + str_val + | NUMBER { $$ = add_number(sb, $1); CHECK_RETURN_PTR($$); } + ; + + +compare : + field '<' val { $$ = add_filter(sb, $1, $3, FILTER_LT); CHECK_RETURN_PTR($$); } + | field '>' val { $$ = add_filter(sb, $1, $3, FILTER_GT); CHECK_RETURN_PTR($$); } + | field LE val { $$ = add_filter(sb, $1, $3, FILTER_LE); CHECK_RETURN_PTR($$); } + | field GE val { $$ = add_filter(sb, $1, $3, FILTER_GE); CHECK_RETURN_PTR($$); } + | field '=' val { $$ = add_filter(sb, $1, $3, FILTER_EQ); CHECK_RETURN_PTR($$); } + | field EQ val { $$ = add_filter(sb, $1, $3, FILTER_EQ); CHECK_RETURN_PTR($$); } + | field NEQ val { $$ = add_filter(sb, $1, $3, FILTER_NE); CHECK_RETURN_PTR($$); } + | field "!=" val { $$ = add_filter(sb, $1, $3, FILTER_NE); CHECK_RETURN_PTR($$); } + | field '&' val { $$ = add_filter(sb, $1, $3, FILTER_BIN_AND); CHECK_RETURN_PTR($$); } + | field '~' str_val { $$ = add_filter(sb, $1, $3, FILTER_STR_CMP); CHECK_RETURN_PTR($$); } +; + +compare_and_or : + compare_and_or OR compare_and_or { $$ = add_filter(sb, $1, $3, FILTER_OR); CHECK_RETURN_PTR($$); } + | compare_and_or AND compare_and_or { $$ = add_filter(sb, $1, $3, FILTER_AND); CHECK_RETURN_PTR($$); } + | '!' '(' compare_and_or ')' { $$ = add_filter(sb, $3, NULL, FILTER_NOT_GROUP); CHECK_RETURN_PTR($$); } + | '!' compare { $$ = add_filter(sb, $2, NULL, FILTER_NOT_GROUP); CHECK_RETURN_PTR($$); } + | compare + ; + +compare_items : + compare_items OR compare_items { $$ = add_filter(sb, $1, $3, FILTER_OR); CHECK_RETURN_PTR($$); } + | '(' compare_and_or ')' { $$ = add_filter(sb, $2, NULL, FILTER_GROUP); CHECK_RETURN_PTR($$); } + | '!' '(' compare_and_or ')' { $$ = add_filter(sb, $3, NULL, FILTER_NOT_GROUP); CHECK_RETURN_PTR($$); } + | '!' compare { $$ = add_filter(sb, $2, NULL, FILTER_NOT_GROUP); CHECK_RETURN_PTR($$); } + | compare + ; + +compare_cmds : + compare_items { CHECK_RETURN_VAL(add_where(sb, $1)); } + ; + +/* + * Top level AND is equal to ',' but the compare_cmds in them must + * all be of for the same event (start or end exclusive). + * That is, OR is not to be used between start and end events. + */ +compare_list : + compare_cmds + | compare_cmds ',' compare_list + | compare_cmds AND compare_list + ; + +where_clause : + WHERE compare_list + ; + +opt_where_clause : + /* empty */ + | where_clause +; + table_exp : - from_clause join_clause + from_clause join_clause opt_where_clause ; from_clause : diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index cf2661773679..e47bc57c5add 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -90,6 +90,8 @@ struct sql_table { struct expr *fields; struct expr *from; struct expr *to; + struct expr *where; + struct expr **next_where; struct match *matches; struct match **next_match; struct expr *selections; @@ -317,9 +319,18 @@ static void *create_expr(enum expr_type type, struct expr **expr_p) #define create_field(var, expr) \ __create_expr(var, struct field, FIELD, expr) +#define create_filter(var, expr) \ + __create_expr(var, struct filter, FILTER, expr) + #define create_compare(var, expr) \ __create_expr(var, struct compare, COMPARE, expr) +#define create_string(var, expr) \ + __create_expr(var, const char *, STRING, expr) + +#define create_number(var, expr) \ + __create_expr(var, long, NUMBER, expr) + __hidden void *add_field(struct sqlhist_bison *sb, const char *field_name, const char *label) { @@ -342,6 +353,22 @@ __hidden void *add_field(struct sqlhist_bison *sb, return expr; } +__hidden void *add_filter(struct sqlhist_bison *sb, + void *A, void *B, enum filter_type op) +{ + struct filter *filter; + struct expr *expr; + + create_filter(filter, &expr); + + filter->lval = A; + filter->rval = B; + + filter->type = op; + + return expr; +} + __hidden int add_match(struct sqlhist_bison *sb, void *A, void *B) { struct sql_table *table = sb->table; @@ -375,6 +402,23 @@ __hidden void *add_compare(struct sqlhist_bison *sb, return expr; } +__hidden int add_where(struct sqlhist_bison *sb, void *item) +{ + struct expr *expr = item; + struct sql_table *table = sb->table; + + if (expr->type != EXPR_FILTER) + return -1; + + *table->next_where = expr; + table->next_where = &expr->next; + + if (expr->next) + return -1; + + return 0; +} + __hidden int add_from(struct sqlhist_bison *sb, void *item) { struct expr *expr = item; @@ -399,6 +443,34 @@ __hidden int add_to(struct sqlhist_bison *sb, void *item) return 0; } +__hidden void *add_string(struct sqlhist_bison *sb, const char *str) +{ + struct expr *expr; + const char **str_p; + + create_string(str_p, &expr); + *str_p = str; + return expr; +} + +__hidden void *add_number(struct sqlhist_bison *sb, long val) +{ + struct expr *expr; + long *num; + + create_number(num, &expr); + *num = val; + return expr; + + expr = calloc(1, sizeof(expr)); + if (!expr) + return NULL; + + expr->type = EXPR_NUMBER; + expr->number = val; + return expr; +} + __hidden int table_start(struct sqlhist_bison *sb) { struct sql_table *table; @@ -410,6 +482,7 @@ __hidden int table_start(struct sqlhist_bison *sb) table->sb = sb; sb->table = table; + table->next_where = &table->where; table->next_match = &table->matches; table->next_selection = &table->selections; @@ -598,6 +671,167 @@ static int build_compare(struct tracefs_synth *synth, return ret; } +static int do_verify_filter(struct filter *filter, + const char **system, const char **event) +{ + int ret; + + if (filter->type == FILTER_OR || + filter->type == FILTER_AND) { + ret = do_verify_filter(&filter->lval->filter, system, event); + if (ret) + return ret; + return do_verify_filter(&filter->rval->filter, system, event); + } + if (filter->type == FILTER_GROUP || + filter->type == FILTER_NOT_GROUP) { + return do_verify_filter(&filter->lval->filter, system, event); + } + + /* + * system and event will be NULL until we find the left most + * node. Then assign it, and compare on the way back up. + */ + if (!*system && !*event) { + *system = filter->lval->field.system; + *event = filter->lval->field.event; + return 0; + } + + if (filter->lval->field.system != *system || + filter->lval->field.event != *event) + return -1; + + return 0; +} + +static int verify_filter(struct filter *filter, + const char **system, const char **event) +{ + int ret; + + switch (filter->type) { + case FILTER_OR: + case FILTER_AND: + case FILTER_GROUP: + case FILTER_NOT_GROUP: + break; + default: + return do_verify_filter(filter, system, event); + } + + ret = do_verify_filter(&filter->lval->filter, system, event); + if (ret) + return ret; + + switch (filter->type) { + case FILTER_OR: + case FILTER_AND: + return do_verify_filter(&filter->rval->filter, system, event); + default: + return 0; + } +} + +static int build_filter(struct tracefs_synth *synth, + bool start, struct filter *filter, bool *started) +{ + int (*append_filter)(struct tracefs_synth *synth, + enum tracefs_filter type, + const char *field, + enum tracefs_compare compare, + const char *val); + enum tracefs_compare cmp; + const char *val; + int and_or = TRACEFS_FILTER_AND; + char num[64]; + int ret; + + if (start) + append_filter = tracefs_synth_append_start_filter; + else + append_filter = tracefs_synth_append_end_filter; + + if (started && *started) { + ret = append_filter(synth, and_or, NULL, 0, NULL); + ret = append_filter(synth, TRACEFS_FILTER_OPEN_PAREN, + NULL, 0, NULL); + } + + switch (filter->type) { + case FILTER_NOT_GROUP: + ret = append_filter(synth, TRACEFS_FILTER_NOT, + NULL, 0, NULL); + if (ret < 0) + goto out; + /* Fall through */ + case FILTER_GROUP: + ret = append_filter(synth, TRACEFS_FILTER_OPEN_PAREN, + NULL, 0, NULL); + if (ret < 0) + goto out; + ret = build_filter(synth, start, &filter->lval->filter, NULL); + if (ret < 0) + goto out; + ret = append_filter(synth, TRACEFS_FILTER_CLOSE_PAREN, + NULL, 0, NULL); + goto out; + + case FILTER_OR: + and_or = TRACEFS_FILTER_OR; + /* Fall through */ + case FILTER_AND: + ret = build_filter(synth, start, &filter->lval->filter, NULL); + if (ret < 0) + goto out; + ret = append_filter(synth, and_or, NULL, 0, NULL); + + if (ret) + goto out; + ret = build_filter(synth, start, &filter->rval->filter, NULL); + goto out; + default: + break; + } + + switch (filter->rval->type) { + case EXPR_NUMBER: + sprintf(num, "%ld", filter->rval->number); + val = num; + break; + case EXPR_STRING: + val = filter->rval->string; + break; + default: + break; + } + + switch (filter->type) { + case FILTER_EQ: cmp = TRACEFS_COMPARE_EQ; break; + case FILTER_NE: cmp = TRACEFS_COMPARE_NE; break; + case FILTER_LE: cmp = TRACEFS_COMPARE_LE; break; + case FILTER_LT: cmp = TRACEFS_COMPARE_LT; break; + case FILTER_GE: cmp = TRACEFS_COMPARE_GE; break; + case FILTER_GT: cmp = TRACEFS_COMPARE_GT; break; + case FILTER_BIN_AND: cmp = TRACEFS_COMPARE_AND; break; + case FILTER_STR_CMP: cmp = TRACEFS_COMPARE_RE; break; + default: + break; + } + + ret = append_filter(synth, TRACEFS_FILTER_COMPARE, + filter->lval->field.field, cmp, val); + + out: + if (!ret && started) { + if (*started) + ret = append_filter(synth, TRACEFS_FILTER_CLOSE_PAREN, + NULL, 0, NULL); + *started = true; + } + return ret; +} + static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *name, struct sql_table *table) @@ -612,6 +846,8 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *end_event; const char *start_match; const char *end_match; + bool started_start = false; + bool started_end = false; int ret; if (!table->to || !table->from) @@ -688,6 +924,30 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, goto free; } + for (expr = table->where; expr; expr = expr->next) { + const char *filter_system = NULL; + const char *filter_event = NULL; + bool *started; + bool start; + + ret = verify_filter(&expr->filter, &filter_system, + &filter_event); + if (ret < 0) + goto free; + + start = filter_system == start_system && + filter_event == start_event; + + if (start) + started = &started_start; + else + started = &started_end; + + ret = build_filter(synth, start, &expr->filter, started); + if (ret < 0) + goto free; + } + return synth; free: tracefs_synth_free(synth); From patchwork Fri Jul 30 22:18:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C965C4320A for ; Fri, 30 Jul 2021 22:18:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6848E60FE7 for ; Fri, 30 Jul 2021 22:18:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233053AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:51268 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232376AbhG3WSj (ORCPT ); Fri, 30 Jul 2021 18:18:39 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9498B60F94; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1L-Gs; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 06/17] libtracefs: Add unit test to test tracefs_sql() where clause Date: Fri, 30 Jul 2021 18:18:13 -0400 Message-Id: <20210730221824.595597-7-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add a test to test filtering of events via the WHERE clause. Signed-off-by: Steven Rostedt (VMware) --- utest/tracefs-utest.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/utest/tracefs-utest.c b/utest/tracefs-utest.c index f4e1a6b90a46..645c84c31f0e 100644 --- a/utest/tracefs-utest.c +++ b/utest/tracefs-utest.c @@ -53,6 +53,13 @@ #define SQL_3_EVENT "wakeup_lat" #define SQL_3_SQL "select start.pid, end.next_prio as prio, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start join sched_switch as end on start.pid = end.next_pid" +#define SQL_4_EVENT "wakeup_lat_2" +#define SQL_4_SQL "select start.pid, end.next_prio as prio, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start join sched_switch as end on start.pid = end.next_pid where (start.prio >= 1 && start.prio < 100) || !(start.pid >= 0 && start.pid <= 1) && end.prev_pid != 0" + +#define SQL_5_EVENT "irq_lat" +#define SQL_5_SQL "select end.common_pid as pid, (end.common_timestamp.usecs - start.common_timestamp.usecs) as irq_lat from irq_disable as start join irq_enable as end on start.common_pid = end.common_pid, start.parent_offs == end.parent_offs where start.common_pid != 0" +#define SQL_5_START "irq_disable" + static struct tracefs_instance *test_instance; static struct tep_handle *test_tep; struct test_sample { @@ -336,6 +343,7 @@ static void test_instance_trace_sql(struct tracefs_instance *instance) struct tracefs_synth *synth; struct trace_seq seq; struct tep_handle *tep; + struct tep_event *event; int ret; tep = tracefs_local_events(NULL); @@ -364,6 +372,23 @@ static void test_instance_trace_sql(struct tracefs_instance *instance) tracefs_synth_free(synth); trace_seq_reset(&seq); + synth = tracefs_sql(tep, SQL_4_EVENT, SQL_4_SQL, NULL); + CU_TEST(synth != NULL); + ret = tracefs_synth_show(&seq, instance, synth); + CU_TEST(ret == 0); + tracefs_synth_free(synth); + trace_seq_reset(&seq); + + event = tep_find_event_by_name(tep, NULL, SQL_5_START); + if (event) { + synth = tracefs_sql(tep, SQL_5_EVENT, SQL_5_SQL, NULL); + CU_TEST(synth != NULL); + ret = tracefs_synth_show(&seq, instance, synth); + CU_TEST(ret == 0); + tracefs_synth_free(synth); + trace_seq_reset(&seq); + } + tep_free(tep); trace_seq_destroy(&seq); } From patchwork Fri Jul 30 22:18:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BAC0C4338F for ; Fri, 30 Jul 2021 22:18:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19747610A6 for ; Fri, 30 Jul 2021 22:18:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233634AbhG3WSz (ORCPT ); Fri, 30 Jul 2021 18:18:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:51334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233004AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B428061076; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1O-Hc; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 07/17] libtracefs: Make sqlhist parser reentrant Date: Fri, 30 Jul 2021 18:18:14 -0400 Message-Id: <20210730221824.595597-8-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Update bison and flex to be reentrant, and not depend on any global variables. Signed-off-by: Steven Rostedt (VMware) --- src/sqlhist-parse.h | 3 +-- src/sqlhist.l | 26 ++++++++++++++++---------- src/sqlhist.y | 10 ++++++++-- src/tracefs-sqlhist.c | 41 ++++++++++++++++++++++++----------------- 4 files changed, 49 insertions(+), 31 deletions(-) diff --git a/src/sqlhist-parse.h b/src/sqlhist-parse.h index 0933bfe9a574..7c1b97ca65af 100644 --- a/src/sqlhist-parse.h +++ b/src/sqlhist-parse.h @@ -10,6 +10,7 @@ struct str_hash; struct sql_table; struct sqlhist_bison { + void *scanner; const char *buffer; size_t buffer_size; size_t buffer_idx; @@ -20,8 +21,6 @@ struct sqlhist_bison { struct str_hash *str_hash[1 << HASH_BITS]; }; -extern struct sqlhist_bison *sb; - #include "sqlhist.tab.h" enum filter_type { diff --git a/src/sqlhist.l b/src/sqlhist.l index 476f7fd1f1ec..958c46051148 100644 --- a/src/sqlhist.l +++ b/src/sqlhist.l @@ -4,21 +4,26 @@ #include #include "sqlhist-parse.h" -extern int my_yyinput(char *buf, int max); +extern int my_yyinput(void *extra, char *buf, int max); #undef YY_INPUT -#define YY_INPUT(b, r, m) ({r = my_yyinput(b, m);}) +#define YY_INPUT(b, r, m) ({r = my_yyinput(yyextra, b, m);}) #define YY_NO_INPUT #define YY_NO_UNPUT #define YY_EXTRA_TYPE struct sqlhist_bison * -#define HANDLE_COLUMN do { sb->line_idx += strlen(yytext); } while (0) +#define yytext yyg->yytext_r + +#define TRACE_SB ((struct sqlhist_bison *)yyextra) +#define HANDLE_COLUMN do { TRACE_SB->line_idx += strlen(yytext); } while (0) %} %option caseless +%option reentrant +%option bison-bridge field [a-z_][a-z0-9_\.]* qstring \"[^\"]*\" @@ -36,25 +41,25 @@ where { HANDLE_COLUMN; return WHERE; } {qstring} { HANDLE_COLUMN; - yylval.string = store_str(sb, yytext); + yylval->string = store_str(TRACE_SB, yyg->yytext_r); return STRING; } {field} { HANDLE_COLUMN; - yylval.string = store_str(sb, yytext); + yylval->string = store_str(TRACE_SB, yyg->yytext_r); return FIELD; } {hexnum} { HANDLE_COLUMN; - yylval.number = strtol(yytext, NULL, 0); + yylval->number = strtol(yyg->yytext_r, NULL, 0); return NUMBER; } {number} { HANDLE_COLUMN; - yylval.number = strtol(yytext, NULL, 0); + yylval->number = strtol(yyg->yytext_r, NULL, 0); return NUMBER; } @@ -69,18 +74,19 @@ where { HANDLE_COLUMN; return WHERE; } [\!()\-\+\*/,=] { HANDLE_COLUMN; return yytext[0]; } [ \t] { HANDLE_COLUMN; } -\n { sb->line_idx = 0; sb->line_no++; } +\n { TRACE_SB->line_idx = 0; TRACE_SB->line_no++; } . { HANDLE_COLUMN; return PARSE_ERROR; } %% -int yywrap(void) +int yywrap(void *data) { return 1; } -void yyerror(const char *fmt, ...) +void yyerror(struct sqlhist_bison *sb, char *fmt, ...) { + struct yyguts_t * yyg = (struct yyguts_t*)sb->scanner; va_list ap; va_start(ap, fmt); diff --git a/src/sqlhist.y b/src/sqlhist.y index 8dcc824bb9f1..ce9cb58fb3a9 100644 --- a/src/sqlhist.y +++ b/src/sqlhist.y @@ -6,8 +6,10 @@ #include "sqlhist-parse.h" -extern int yylex(void); -extern void yyerror(char *fmt, ...); +#define scanner sb->scanner + +extern int yylex(YYSTYPE *yylval, void *); +extern void yyerror(struct sqlhist_bison *, char *fmt, ...); #define CHECK_RETURN_PTR(x) \ do { \ @@ -27,6 +29,10 @@ extern void yyerror(char *fmt, ...); %} +%define api.pure +%lex-param {void *scanner} +%parse-param {struct sqlhist_bison *sb} + %union { int s32; char *string; diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index e47bc57c5add..e8f77d60659f 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -14,8 +14,6 @@ #include "tracefs-local.h" #include "sqlhist-parse.h" -struct sqlhist_bison *sb; - extern int yylex_init(void* ptr_yy_globals); extern int yylex_init_extra(struct sqlhist_bison *sb, void* ptr_yy_globals); extern int yylex_destroy (void * yyscanner ); @@ -98,8 +96,10 @@ struct sql_table { struct expr **next_selection; }; -__hidden int my_yyinput(char *buf, int max) +__hidden int my_yyinput(void *extra, char *buf, int max) { + struct sqlhist_bison *sb = extra; + if (!sb || !sb->buffer) return -1; @@ -284,7 +284,8 @@ static struct expr *find_field(struct sqlhist_bison *sb, return NULL; } -static void *create_expr(enum expr_type type, struct expr **expr_p) +static void *create_expr(struct sqlhist_bison *sb, + enum expr_type type, struct expr **expr_p) { struct expr *expr; @@ -313,7 +314,7 @@ static void *create_expr(enum expr_type type, struct expr **expr_p) #define __create_expr(var, type, ENUM, expr) \ do { \ - var = (type *)create_expr(EXPR_##ENUM, expr); \ + var = (type *)create_expr(sb, EXPR_##ENUM, expr); \ } while(0) #define create_field(var, expr) \ @@ -999,8 +1000,8 @@ static void free_sb(struct sqlhist_bison *sb) struct tracefs_synth *tracefs_sql(struct tep_handle *tep, const char *name, const char *sql_buffer, char **err) { - struct sqlhist_bison local_sb; struct tracefs_synth *synth = NULL; + struct sqlhist_bison sb; int ret; if (!tep || !sql_buffer) { @@ -1008,27 +1009,33 @@ struct tracefs_synth *tracefs_sql(struct tep_handle *tep, const char *name, return NULL; } - memset(&local_sb, 0, sizeof(local_sb)); + memset(&sb, 0, sizeof(sb)); + + sb.buffer = sql_buffer; + sb.buffer_size = strlen(sql_buffer); + sb.buffer_idx = 0; - local_sb.buffer = sql_buffer; - local_sb.buffer_size = strlen(sql_buffer); - local_sb.buffer_idx = 0; + ret = yylex_init_extra(&sb, &sb.scanner); + if (ret < 0) { + yylex_destroy(sb.scanner); + return NULL; + } - sb = &local_sb; - ret = yyparse(); + ret = yyparse(&sb); + yylex_destroy(sb.scanner); if (ret) goto free; - synth = build_synth(tep, name, sb->table); + synth = build_synth(tep, name, sb.table); free: if (!synth) { - if (sb->parse_error_str && err) { - *err = sb->parse_error_str; - sb->parse_error_str = NULL; + if (sb.parse_error_str && err) { + *err = sb.parse_error_str; + sb.parse_error_str = NULL; } } - free_sb(sb); + free_sb(&sb); return synth; } From patchwork Fri Jul 30 22:18:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA45AC432BE for ; Fri, 30 Jul 2021 22:18:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C01B06101C for ; Fri, 30 Jul 2021 22:18:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233179AbhG3WSl (ORCPT ); Fri, 30 Jul 2021 18:18:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:51246 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229604AbhG3WSj (ORCPT ); Fri, 30 Jul 2021 18:18:39 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9080560F13; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1S-Ic; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 08/17] libtracefs: Make parser unique to libtracefs Date: Fri, 30 Jul 2021 18:18:15 -0400 Message-Id: <20210730221824.595597-9-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add %define api.prefix and defines to have the parser global variables use tracefs_* instead of yy*, as without this, if a tool that links to this library, and tries to use the synth sql parsing, it may end up using its own yyparse() and friends functions. Signed-off-by: Steven Rostedt (VMware) --- src/sqlhist.y | 10 ++++++++++ src/tracefs-sqlhist.c | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/sqlhist.y b/src/sqlhist.y index ce9cb58fb3a9..9d03a457ae84 100644 --- a/src/sqlhist.y +++ b/src/sqlhist.y @@ -30,6 +30,16 @@ extern void yyerror(struct sqlhist_bison *, char *fmt, ...); %} %define api.pure + +/* Change the globals to use tracefs_ prefix */ +%define api.prefix{tracefs_} +%code provides +{ + #define YYSTYPE TRACEFS_STYPE + #define yylex tracefs_lex + #define yyerror tracefs_error +} + %lex-param {void *scanner} %parse-param {struct sqlhist_bison *sb} diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index e8f77d60659f..933b3609733b 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -1021,7 +1021,7 @@ struct tracefs_synth *tracefs_sql(struct tep_handle *tep, const char *name, return NULL; } - ret = yyparse(&sb); + ret = tracefs_parse(&sb); yylex_destroy(sb.scanner); if (ret) From patchwork Fri Jul 30 22:18:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84878C43216 for ; Fri, 30 Jul 2021 22:18:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 66BCE610A6 for ; Fri, 30 Jul 2021 22:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233336AbhG3WSm (ORCPT ); Fri, 30 Jul 2021 18:18:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:51286 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232589AbhG3WSj (ORCPT ); Fri, 30 Jul 2021 18:18:39 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9DC6A61040; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1V-JQ; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 09/17] libtracefs: Add line number and index to expr structure Date: Fri, 30 Jul 2021 18:18:16 -0400 Message-Id: <20210730221824.595597-10-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" In order to have better error messages, record the line number and index when an expr structure is created. Then this can be used to show where in the SQL sequence a problem was found if the building of the synth event has issues. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index 933b3609733b..887c2441a39e 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -72,6 +72,8 @@ struct expr { struct expr *free_list; struct expr *next; enum expr_type type; + int line; + int idx; union { struct field field; struct filter filter; @@ -300,6 +302,8 @@ static void *create_expr(struct sqlhist_bison *sb, sb->table->exprs = expr; expr->type = type; + expr->line = sb->line_no; + expr->idx = sb->line_idx; switch (type) { case EXPR_FIELD: return &expr->field; From patchwork Fri Jul 30 22:18:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D424CC19F31 for ; Fri, 30 Jul 2021 22:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA3B361057 for ; Fri, 30 Jul 2021 22:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233262AbhG3WSm (ORCPT ); Fri, 30 Jul 2021 18:18:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:51306 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232680AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9DD6F61042; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1Y-KC; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 10/17] libtracefs: Add a parse_error() helper function to record errors Date: Fri, 30 Jul 2021 18:18:17 -0400 Message-Id: <20210730221824.595597-11-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add parse_error() that calls into sql_parse_error() with the appropriate ap argument. That is, parse_error() takes a normal printf() form and then translates it to the vprintf from of sql_parse_error. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index 887c2441a39e..c0875d630391 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -8,6 +8,7 @@ */ #include #include +#include #include #include "tracefs.h" @@ -153,6 +154,16 @@ __hidden void sql_parse_error(struct sqlhist_bison *sb, const char *text, trace_seq_destroy(&s); } +static void parse_error(struct sqlhist_bison *sb, const char *text, + const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + sql_parse_error(sb, text, fmt, ap); + va_end(ap); +} + static inline unsigned int quick_hash(const char *str) { unsigned int val = 0; From patchwork Fri Jul 30 22:18:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E4FFC4320E for ; Fri, 30 Jul 2021 22:18:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BC58610A7 for ; Fri, 30 Jul 2021 22:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233420AbhG3WSo (ORCPT ); Fri, 30 Jul 2021 18:18:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:51316 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232830AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AFC5661054; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1b-L1; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 11/17] libtracefs: Add error message when match fields are not FROM and JOIN events Date: Fri, 30 Jul 2021 18:18:18 -0400 Message-Id: <20210730221824.595597-12-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" It is required that the "match" content (the ON portion of the SQL sequence) has a field from the FROM event and a field from the JOIN event. If they do not, then give a better message about what went wrong. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index c0875d630391..0489657bea03 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -593,6 +593,34 @@ static int update_vars(struct sql_table *table, struct field *event) return 0; } +static int match_error(struct sqlhist_bison *sb, struct match *match, + struct field *lmatch, struct field *rmatch) +{ + struct field *lval = &match->lval->field; + struct field *rval = &match->rval->field; + struct field *field; + struct expr *expr; + + if (lval->system != lmatch->system || + lval->event != lmatch->event) { + expr = match->lval; + field = lval; + } else { + expr = match->rval; + field = rval; + } + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, field->raw, + "'%s' and '%s' must be a field for each event: '%s' and '%s'\n", + lval->raw, rval->raw, sb->table->to->field.raw, + sb->table->from->field.raw); + + return -1; +} + static int test_match(struct sql_table *table, struct match *match) { struct field *lval, *rval; @@ -624,13 +652,13 @@ static int test_match(struct sql_table *table, struct match *match) (rval->event != to->event) || (lval->system != from->system) || (lval->event != from->event)) - return -1; + return match_error(table->sb, match, from, to); } else { if ((rval->system != from->system) || (rval->event != from->event) || (lval->system != to->system) || (lval->event != to->event)) - return -1; + return match_error(table->sb, match, to, from); } return 0; } From patchwork Fri Jul 30 22:18:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9D16C19F35 for ; Fri, 30 Jul 2021 22:18:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8534461131 for ; Fri, 30 Jul 2021 22:18:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233739AbhG3WSt (ORCPT ); Fri, 30 Jul 2021 18:18:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:51330 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232985AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AC8FE61058; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1e-Lu; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 12/17] libtracefs: Add error message when match or init fails from bad events Date: Fri, 30 Jul 2021 18:18:19 -0400 Message-Id: <20210730221824.595597-13-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" If the tracefs_synth_init() or the matching fails due to events that do not exist, or if the match fields are not compatible with each other. Add an error message that reports this and where the problem is. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 101 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 99 insertions(+), 2 deletions(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index 0489657bea03..d39851845570 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -876,6 +876,101 @@ static int build_filter(struct tracefs_synth *synth, return ret; } +static int test_event_exists(struct tep_handle *tep, struct sqlhist_bison *sb, + struct expr *expr, struct tep_event **pevent) +{ + const char *system = expr->field.system; + const char *event_name = expr->field.event; + struct tep_event *event; + + event = tep_find_event_by_name(tep, system, event_name); + if (pevent) + *pevent = event; + if (event) + return 0; + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, expr->field.raw, "event not found\n"); + return -1; +} + +static int test_field_exists(struct tep_handle *tep, struct sqlhist_bison *sb, + struct expr *expr) +{ + struct tep_event *event; + + if (test_event_exists(tep, sb, expr, &event)) + return -1; + + if (trace_verify_event_field(event, expr->field.field, NULL)) + return 0; + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, expr->field.raw, "Field '%s' not part of event %s\n", + expr->field.field, expr->field.event); + return -1; +} + +static void *field_match_error(struct tep_handle *tep, struct sqlhist_bison *sb, + struct match *match) +{ + switch (errno) { + case ENODEV: + case EBADE: + break; + default: + /* System error */ + return NULL; + } + + /* ENODEV means that an event or field does not exist */ + if (errno == ENODEV) { + if (test_field_exists(tep, sb, match->lval)) + return NULL; + if (test_field_exists(tep, sb, match->rval)) + return NULL; + return NULL; + } + + /* fields exist, but values are not compatible */ + sb->line_no = match->lval->line; + sb->line_idx = match->lval->idx; + + parse_error(sb, match->lval->field.raw, + "Field '%s' is not compatible to match field '%s'\n", + match->lval->field.raw, match->rval->field.raw); + return NULL; +} + +static void *synth_init_error(struct tep_handle *tep, struct sql_table *table) +{ + struct sqlhist_bison *sb = table->sb; + struct match *match = table->matches; + + switch (errno) { + case ENODEV: + case EBADE: + break; + default: + /* System error */ + return NULL; + } + + /* ENODEV could mean that start or end events do not exist */ + if (errno == ENODEV) { + if (test_event_exists(tep, sb, table->from, NULL)) + return NULL; + if (test_event_exists(tep, sb, table->to, NULL)) + return NULL; + } + + return field_match_error(tep, sb, match); +} + static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *name, struct sql_table *table) @@ -926,7 +1021,7 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, start_event, end_system, end_event, start_match, end_match, NULL); if (!synth) - return NULL; + return synth_init_error(tep, table); for (match = match->next; match; match = match->next) { ret = test_match(table, match); @@ -939,8 +1034,10 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, ret = tracefs_synth_add_match_field(synth, start_match, end_match, NULL); - if (ret < 0) + if (ret < 0) { + field_match_error(tep, table->sb, match); goto free; + } } for (expr = table->selections; expr; expr = expr->next) { From patchwork Fri Jul 30 22:18:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9DBBC4320E for ; Fri, 30 Jul 2021 22:18:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91686610CD for ; Fri, 30 Jul 2021 22:18:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233601AbhG3WSs (ORCPT ); Fri, 30 Jul 2021 18:18:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:51322 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232948AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B609661078; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1h-Mq; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 13/17] libtracefs; Add error message for bad selections to SQL sequence Date: Fri, 30 Jul 2021 18:18:20 -0400 Message-Id: <20210730221824.595597-14-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" If the building of the synthetic event fails on creating the selection, report it properly. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index d39851845570..e542eb95c17f 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -971,6 +971,16 @@ static void *synth_init_error(struct tep_handle *tep, struct sql_table *table) return field_match_error(tep, sb, match); } +static void selection_error(struct tep_handle *tep, + struct sqlhist_bison *sb, struct expr *expr) +{ + /* We just care about event not existing */ + if (errno != ENODEV) + return; + + test_field_exists(tep, sb, expr); +} + static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *name, struct sql_table *table) @@ -1051,8 +1061,10 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, ret = tracefs_synth_add_end_field(synth, field->field, field->label); } - if (ret < 0) + if (ret < 0) { + selection_error(tep, table->sb, expr); goto free; + } continue; } From patchwork Fri Jul 30 22:18:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9006C4338F for ; Fri, 30 Jul 2021 22:19:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3E9E6108E for ; Fri, 30 Jul 2021 22:19:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233472AbhG3WSr (ORCPT ); Fri, 30 Jul 2021 18:18:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:51320 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232934AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B8EA761074; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1k-Nf; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 14/17] libtracefs: Add error message when compare fields fail Date: Fri, 30 Jul 2021 18:18:21 -0400 Message-Id: <20210730221824.595597-15-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" If the processing of comparing fields fail due to not existing or because they are not compatible to compare, report a proper error message. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index e542eb95c17f..ff0869232a9e 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -981,6 +981,38 @@ static void selection_error(struct tep_handle *tep, test_field_exists(tep, sb, expr); } +static void compare_error(struct tep_handle *tep, + struct sqlhist_bison *sb, struct expr *expr) +{ + struct compare *compare = &expr->compare; + + switch (errno) { + case ENODEV: + case EBADE: + break; + default: + /* System error */ + return; + } + + /* ENODEV means that an event or field does not exist */ + if (errno == ENODEV) { + if (test_field_exists(tep, sb, compare->lval)) + return; + if (test_field_exists(tep, sb, compare->rval)) + return; + return; + } + + /* fields exist, but values are not compatible */ + sb->line_no = compare->lval->line; + sb->line_idx = compare->lval->idx; + + parse_error(sb, compare->lval->field.raw, + "'%s' is not compatible to compare with '%s'\n", + compare->lval->field.raw, compare->rval->field.raw); +} + static struct tracefs_synth *build_synth(struct tep_handle *tep, const char *name, struct sql_table *table) @@ -1073,8 +1105,10 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, ret = build_compare(synth, start_system, end_system, &expr->compare); - if (ret < 0) + if (ret < 0) { + compare_error(tep, table->sb, expr); goto free; + } } for (expr = table->where; expr; expr = expr->next) { From patchwork Fri Jul 30 22:18:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8DB1C19F37 for ; Fri, 30 Jul 2021 22:18:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F1F2610CD for ; Fri, 30 Jul 2021 22:18:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232140AbhG3WSw (ORCPT ); Fri, 30 Jul 2021 18:18:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:51316 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233099AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C59956109F; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1n-OY; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 15/17] libtracefs: Add error message for grouping events in SQL filter Date: Fri, 30 Jul 2021 18:18:22 -0400 Message-Id: <20210730221824.595597-16-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" One requirement for the SQL filter in tracefs_sql() is that the WHERE clause (filter) can only filter the FROM and JOIN events with "&&". That is, you can not have: sched_switch.next_pid == 0 || sched_waking.pid == 0 As the filtering one event stops the synthetic event, having an || conjunction makes no sense. Add an error message that explains this when it is found. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index ff0869232a9e..1767c33d77d0 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -715,21 +715,36 @@ static int build_compare(struct tracefs_synth *synth, return ret; } -static int do_verify_filter(struct filter *filter, +static int verify_filter_error(struct sqlhist_bison *sb, struct expr *expr, + const char *event) +{ + struct field *field = &expr->field; + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, field->raw, + "event '%s' can not be grouped or '||' together with '%s'\n" + "All filters between '&&' must be for the same event\n", + field->event, event); + return -1; +} + +static int do_verify_filter(struct sqlhist_bison *sb, struct filter *filter, const char **system, const char **event) { int ret; if (filter->type == FILTER_OR || filter->type == FILTER_AND) { - ret = do_verify_filter(&filter->lval->filter, system, event); + ret = do_verify_filter(sb, &filter->lval->filter, system, event); if (ret) return ret; - return do_verify_filter(&filter->rval->filter, system, event); + return do_verify_filter(sb, &filter->rval->filter, system, event); } if (filter->type == FILTER_GROUP || filter->type == FILTER_NOT_GROUP) { - return do_verify_filter(&filter->lval->filter, system, event); + return do_verify_filter(sb, &filter->lval->filter, system, event); } /* @@ -744,12 +759,12 @@ static int do_verify_filter(struct filter *filter, if (filter->lval->field.system != *system || filter->lval->field.event != *event) - return -1; + return verify_filter_error(sb, filter->lval, *event); return 0; } -static int verify_filter(struct filter *filter, +static int verify_filter(struct sqlhist_bison *sb, struct filter *filter, const char **system, const char **event) { int ret; @@ -761,17 +776,17 @@ static int verify_filter(struct filter *filter, case FILTER_NOT_GROUP: break; default: - return do_verify_filter(filter, system, event); + return do_verify_filter(sb, filter, system, event); } - ret = do_verify_filter(&filter->lval->filter, system, event); + ret = do_verify_filter(sb, &filter->lval->filter, system, event); if (ret) return ret; switch (filter->type) { case FILTER_OR: case FILTER_AND: - return do_verify_filter(&filter->rval->filter, system, event); + return do_verify_filter(sb, &filter->rval->filter, system, event); default: return 0; } @@ -1117,7 +1132,7 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, bool *started; bool start; - ret = verify_filter(&expr->filter, &filter_system, + ret = verify_filter(table->sb, &expr->filter, &filter_system, &filter_event); if (ret < 0) goto free; From patchwork Fri Jul 30 22:18:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 490010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B230C19F3A for ; Fri, 30 Jul 2021 22:18:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33F45610A6 for ; Fri, 30 Jul 2021 22:18:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233975AbhG3WSx (ORCPT ); Fri, 30 Jul 2021 18:18:53 -0400 Received: from mail.kernel.org ([198.145.29.99]:51344 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233043AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CF6C56109D; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1q-PU; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 16/17] libtracefs: Add error message for bad filters in SQL statement Date: Fri, 30 Jul 2021 18:18:23 -0400 Message-Id: <20210730221824.595597-17-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" If a filter has a bad event, or incompatibility with the value assigned to it, have the error message display it. Signed-off-by: Steven Rostedt (VMware) --- src/tracefs-sqlhist.c | 80 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 74 insertions(+), 6 deletions(-) diff --git a/src/tracefs-sqlhist.c b/src/tracefs-sqlhist.c index 1767c33d77d0..f191661610c9 100644 --- a/src/tracefs-sqlhist.c +++ b/src/tracefs-sqlhist.c @@ -792,14 +792,80 @@ static int verify_filter(struct sqlhist_bison *sb, struct filter *filter, } } -static int build_filter(struct tracefs_synth *synth, - bool start, struct filter *filter, bool *started) +static int test_field_exists(struct tep_handle *tep, struct sqlhist_bison *sb, + struct expr *expr); + +static void filter_compare_error(struct tep_handle *tep, + struct sqlhist_bison *sb, + struct expr *expr) +{ + struct field *field = &expr->field; + + switch (errno) { + case ENODEV: + case EBADE: + break; + case EINVAL: + parse_error(sb, field->raw, "Invalid compare\n"); + break; + default: + parse_error(sb, field->raw, "System error?\n"); + return; + } + + /* ENODEV means that an event or field does not exist */ + if (errno == ENODEV) { + if (test_field_exists(tep, sb, expr)) + return; + if (test_field_exists(tep, sb, expr)) + return; + return; + } + + /* fields exist, but values are not compatible */ + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, field->raw, + "Field '%s' is not compatible to be compared with the given value\n", + field->field); +} + +static void filter_error(struct tep_handle *tep, + struct sqlhist_bison *sb, struct expr *expr) +{ + struct filter *filter = &expr->filter; + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + switch (filter->type) { + case FILTER_NOT_GROUP: + case FILTER_GROUP: + case FILTER_OR: + case FILTER_AND: + break; + default: + filter_compare_error(tep, sb, filter->lval); + return; + } + + sb->line_no = expr->line; + sb->line_idx = expr->idx; + + parse_error(sb, "", "Problem with filter entry?\n"); +} + +static int build_filter(struct tep_handle *tep, struct sqlhist_bison *sb, + struct tracefs_synth *synth, + bool start, struct expr *expr, bool *started) { int (*append_filter)(struct tracefs_synth *synth, enum tracefs_filter type, const char *field, enum tracefs_compare compare, const char *val); + struct filter *filter = &expr->filter; enum tracefs_compare cmp; const char *val; int and_or = TRACEFS_FILTER_AND; @@ -829,7 +895,7 @@ static int build_filter(struct tracefs_synth *synth, NULL, 0, NULL); if (ret < 0) goto out; - ret = build_filter(synth, start, &filter->lval->filter, NULL); + ret = build_filter(tep, sb, synth, start, filter->lval, NULL); if (ret < 0) goto out; ret = append_filter(synth, TRACEFS_FILTER_CLOSE_PAREN, @@ -840,14 +906,14 @@ static int build_filter(struct tracefs_synth *synth, and_or = TRACEFS_FILTER_OR; /* Fall through */ case FILTER_AND: - ret = build_filter(synth, start, &filter->lval->filter, NULL); + ret = build_filter(tep, sb, synth, start, filter->lval, NULL); if (ret < 0) goto out; ret = append_filter(synth, and_or, NULL, 0, NULL); if (ret) goto out; - ret = build_filter(synth, start, &filter->rval->filter, NULL); + ret = build_filter(tep, sb, synth, start, filter->rval, NULL); goto out; default: break; @@ -881,6 +947,8 @@ static int build_filter(struct tracefs_synth *synth, ret = append_filter(synth, TRACEFS_FILTER_COMPARE, filter->lval->field.field, cmp, val); + if (ret) + filter_error(tep, sb, expr); out: if (!ret && started) { if (*started) @@ -1145,7 +1213,7 @@ static struct tracefs_synth *build_synth(struct tep_handle *tep, else started = &started_end; - ret = build_filter(synth, start, &expr->filter, started); + ret = build_filter(tep, table->sb, synth, start, expr, started); if (ret < 0) goto free; } From patchwork Fri Jul 30 22:18:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 489623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 768F7C3F6A3 for ; Fri, 30 Jul 2021 22:18:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 582F9610A6 for ; Fri, 30 Jul 2021 22:18:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233848AbhG3WSv (ORCPT ); Fri, 30 Jul 2021 18:18:51 -0400 Received: from mail.kernel.org ([198.145.29.99]:51356 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233112AbhG3WSk (ORCPT ); Fri, 30 Jul 2021 18:18:40 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DBED9610E6; Fri, 30 Jul 2021 22:18:34 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.94.2) (envelope-from ) id 1m9apt-002V1t-QQ; Fri, 30 Jul 2021 18:18:33 -0400 From: Steven Rostedt To: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Tom Zanussi , Daniel Bristot de Oliveira , Masami Hiramatsu , Namhyung Kim , linux-rt-users , Clark Williams , "Steven Rostedt (VMware)" Subject: [PATCH 17/17] libtracefs: Add man page for tracefs_sql() Date: Fri, 30 Jul 2021 18:18:24 -0400 Message-Id: <20210730221824.595597-18-rostedt@goodmis.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210730221824.595597-1-rostedt@goodmis.org> References: <20210730221824.595597-1-rostedt@goodmis.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org From: "Steven Rostedt (VMware)" Add a man page for tracefs_sql(). Included in that man page is a full working sql parser example program that can allow you to create synthetic events from writing SQL on the command line. Signed-off-by: Steven Rostedt (VMware) --- Documentation/libtracefs-sql.txt | 367 +++++++++++++++++++++++++++++++ 1 file changed, 367 insertions(+) create mode 100644 Documentation/libtracefs-sql.txt diff --git a/Documentation/libtracefs-sql.txt b/Documentation/libtracefs-sql.txt new file mode 100644 index 000000000000..e9b3d1ddbc22 --- /dev/null +++ b/Documentation/libtracefs-sql.txt @@ -0,0 +1,367 @@ +libtracefs(3) +============= + +NAME +---- +tracefs_sql - Create a synthetitc event via an SQL statement + +SYNOPSIS +-------- +[verse] +-- +*#include * + +struct tracefs_synth *tracefs_sql(struct tep_handle pass:[*]tep, const char pass:[*]name, + const char pass:[*]sql_buffer, char pass:[**]err); +-- + +DESCRIPTION +----------- +Synthetic events are dynamically created events that attach two existing events +together via one or more matching fields between the two events. It can be used +to find the latency between the events, or to simply pass fields of the first event +on to the second event to display as one event. + +The Linux kernel interface to create synthetic events is complex, and there needs +to be a better way to create synthetic events that is easy and can be understood +via existing technology. + +If you think of each event as a table, where the fields are the column of the table +and each instance of the event as a row, you can understand how SQL can be used +to attach two events together and for another event (table). Utilizing the +SQL *SELECT* *FROM* *JOIN* *ON* [ *WHERE* ] syntax, a synthetic event can easily +be created from two different events. + + +*tracefs_sql*() takes in a *tep* handler (See _tep_local_events_(3)) that is used to +verify the events within the _sql_buffer_ expression. The _name_ is the name of the +synthetic event to create. If _err_ points to an address of a string, it will be filled +with a detailed message on any type of parsing error, including fields that do not belong +to an event, or if the events or fields are not properly compared. + +The example program below is a fully functional parser where it will create a synthetic +event from a SQL syntax passed in via the command line or a file. + +The SQL format is as follows: + +*SELECT* FROM JOIN ON WHERE + +Note, although the examples show the SQL commands in uppercase, they are not required to +be so. That is, you can use "SELECT" or "select" or "sElEct". + +For example: +[source,c] +-- +SELECT syscalls.sys_enter_read.fd, syscalls.sys_exit_read.ret FROM syscalls.sys_enter_read + JOIN syscalls.sys_exit_read + ON syscalls.sys_enter_read.common_pid = syscalls.sys_exit_write.common_pid +-- + +Will create a synthetic event that with the fields: + + u64 fd; s64 ret; + +Because the function takes a _tep_ handle, and usually all event names are unique, you can +leave off the system (group) name of the event, and *tracefs_sql*() will discover the +system for you. + +That is, the above statement would work with: + +[source,c] +-- +SELECT sys_enter_read.fd, sys_exit_read.ret FROM sys_enter_read JOIN sys_exit_read + ON sys_enter_read.common_pid = sys_exit_write.common_pid +-- + +The *AS* keyword can be used to name the fields as well as to give an alias to the +events, such that the above can be simplified even more as: + +[source,c] +-- +SELECT start.fd, end.ret FROM sys_enter_read AS start JOIN sys_exit_read AS end ON start.common_pid = end.common_pid +-- + +The above aliases _sys_enter_read_ as *start* and _sys_exit_read_ as *end* and uses +those aliases to reference the event throughout the statement. + +Using the *AS* keyword in the selection portion of the SQL statement will define what +those fields will be called in the synthetic event. + +[source,c] +-- +SELECT start.fd AS filed, end.ret AS return FROM sys_enter_read AS start JOIN sys_exit_read AS end + ON start.common_pid = end.common_pid +-- + +The above labels the _fd_ of _start_ as *filed* and the _ret_ of _end_ as *return* where +the synthetic event that is created will now have the fields: + + u64 filed; s64 return; + +The fields can also be calculated with results passed to the synthetic event: + +[source,c] +-- +select start.truesize, end.len, (start.truesize - end.len) as diff from napi_gro_receive_entry as start + JOIN netif_receive_skb as end ON start.skbaddr = end.skbaddr +-- + +Which would show the *truesize" of the _napi_gro_receive_entry_ event, the actual +_len_ of the content, shown by the _netif_receive_skb_, and alse the delta between +the two and expressed by the field *diff*. + +The code also supports recording the timestamps at either event, and performing calculations +on them. For wakeup latency, you have: + +[source,c] +-- +select start.pid, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start + JOIN sched_switch as end ON start.pid = end.next_pid +-- + +The above will create a synthetic event that records the _pid_ of the task being woken up, +and the time difference between the _sched_waking_ event and the _sched_switch_ event. +The *TIMESTAMP_USECS* will truncate the time down to microseconds as the timestamp usually +recorded in the tracing buffer has nanosecond resolution. If you do not want that +truncation, use *TIMESTAMP* instead of *TIMESTAMP_USECS*. + +Finally, the *WHERE* clause can be added, that will let you add filters on either or both events. + +[source,c] +-- +select start.pid, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start + JOIN sched_switch as end ON start.pid = end.next_pid + WHERE start.prio < 100 && (!(end.prev_pid < 1 || end.prev_prio > 100) || end.prev_pid == 0) +-- + +*NOTE* + +Although both events can be used together in the *WHERE* clause, they must not be mixed outside +the top most "&&" statements. You can not OR (||) the events together, where a filter of one +event is OR'd to a filter of the other event. This does not make sense, as the synthetic event +requires both events to take place to be recorded. If one is filtered out, then the synthetic +event does not execute. + +[source,c] +-- +select start.pid, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start + JOIN sched_switch as end ON start.pid = end.next_pid + WHERE start.prio < 100 && end.prev_prio < 100 +-- + +The above is valid. + +Where as the below is not. + +[source,c] +-- +select start.pid, (end.TIMESTAMP_USECS - start.TIMESTAMP_USECS) as lat from sched_waking as start + JOIN sched_switch as end ON start.pid = end.next_pid + WHERE start.prio < 100 || end.prev_prio < 100 +-- + + +RETURN VALUE +------------ +Returns 0 on success and -1 on failure. On failure, if _err_ is defined, it will be +allocated to hold a detailed description of what went wrong if it the error was caused +by a parsing error, or that an event, field does not exist or is not compatible with +what it was combined with. + +EXAMPLE +------- +[source,c] +-- +#include +#include +#include +#include +#include +#include +#include + +static void usage(char **argv) +{ + fprintf(stderr, "usage: %s [-ed][-n name][-t dir][-f file | sql-command-line]\n" + " -n name - name of synthetic event 'Anonymous' if left off\n" + " -t dir - use dir instead of /sys/kernel/tracing\n" + " -e - execute the commands to create the synthetic event\n" + " -d - delete the synthetic event that would be created\n" + " -f file - read sql lines from file otherwise from the command line\n" + " if file is '-' then read from standard input.\n", + argv[0]); + exit(-1); +} + +static int do_sql(const char *buffer, const char *name, + const char *trace_dir, bool execute) +{ + struct tracefs_synth *synth; + struct tep_handle *tep; + struct trace_seq seq; + char *err; + + if (!name) + name = "Anonymous"; + + trace_seq_init(&seq); + tep = tracefs_local_events(trace_dir); + if (!tep) { + if (!trace_dir) + trace_dir = "tracefs directory"; + perror(trace_dir); + exit(-1); + } + + synth = tracefs_sql(tep, name, buffer, &err); + if (!synth) { + perror("Failed creating synthetic event!"); + if (err) + fprintf(stderr, "%s", err); + exit(-1); + } + + tracefs_synth_show(&seq, NULL, synth); + if (execute) + tracefs_synth_create(NULL, synth); + tracefs_synth_free(synth); + + trace_seq_do_printf(&seq); + trace_seq_destroy(&seq); + return 0; +} + +int main (int argc, char **argv) +{ + char *trace_dir = NULL; + char *buffer = NULL; + char buf[BUFSIZ]; + int buffer_size = 0; + const char *file = NULL; + bool execute = false; + const char *name; + FILE *fp; + size_t r; + int c; + int i; + + for (;;) { + c = getopt(argc, argv, "ht:f:edn:"); + if (c == -1) + break; + + switch(c) { + case 'h': + usage(argv); + case 't': + trace_dir = optarg; + break; + case 'f': + file = optarg; + break; + case 'e': + execute = true; + break; + case 'n': + name = optarg; + break; + } + } + + if (file) { + if (!strcmp(file, "-")) + fp = stdin; + else + fp = fopen(file, "r"); + if (!fp) { + perror(file); + exit(-1); + } + while ((r = fread(buf, 1, BUFSIZ, fp)) > 0) { + buffer = realloc(buffer, buffer_size + r + 1); + strncpy(buffer + buffer_size, buf, r); + buffer_size += r; + } + fclose(fp); + if (buffer_size) + buffer[buffer_size] = '\0'; + } else if (argc == optind) { + usage(argv); + } else { + for (i = optind; i < argc; i++) { + r = strlen(argv[i]); + buffer = realloc(buffer, buffer_size + r + 2); + if (i != optind) + buffer[buffer_size++] = ' '; + strcpy(buffer + buffer_size, argv[i]); + buffer_size += r; + } + } + + do_sql(buffer, name, trace_dir, execute); + free(buffer); + + return 0; +} +-- + +FILES +----- +[verse] +-- +*tracefs.h* + Header file to include in order to have access to the library APIs. +*-ltracefs* + Linker switch to add when building a program that uses the library. +-- + +SEE ALSO +-------- +_libtracefs(3)_, +_libtraceevent(3)_, +_trace-cmd(1)_, +_tracefs_synth_init(3)_, +_tracefs_synth_add_match_field(3)_, +_tracefs_synth_add_compare_field(3)_, +_tracefs_synth_add_start_field(3)_, +_tracefs_synth_add_end_field(3)_, +_tracefs_synth_append_start_filter(3)_, +_tracefs_synth_append_end_filter(3)_, +_tracefs_synth_create(3)_, +_tracefs_synth_destroy(3)_, +_tracefs_synth_free(3)_, +_tracefs_synth_show(3)_, +_tracefs_hist_alloc(3)_, +_tracefs_hist_free(3)_, +_tracefs_hist_add_key(3)_, +_tracefs_hist_add_value(3)_, +_tracefs_hist_add_name(3)_, +_tracefs_hist_start(3)_, +_tracefs_hist_destory(3)_, +_tracefs_hist_add_sort_key(3)_, +_tracefs_hist_sort_key_direction(3)_ + +AUTHOR +------ +[verse] +-- +*Steven Rostedt* +*Tzvetomir Stoyanov* +*sameeruddin shaik* +-- +REPORTING BUGS +-------------- +Report bugs to + +LICENSE +------- +libtracefs is Free Software licensed under the GNU LGPL 2.1 + +RESOURCES +--------- +https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ + +COPYING +------- +Copyright \(C) 2020 VMware, Inc. Free use of this software is granted under +the terms of the GNU Public License (GPL).