From patchwork Sat Jun 5 11:10:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37AFBC4743E for ; Sat, 5 Jun 2021 11:10:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 21B8D613B4 for ; Sat, 5 Jun 2021 11:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230220AbhFELMq convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:12:46 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:44718 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230118AbhFELMo (ORCPT ); Sat, 5 Jun 2021 07:12:44 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-476-XAGNN3y7OmmpP6l2y-Mz6Q-1; Sat, 05 Jun 2021 07:10:47 -0400 X-MC-Unique: XAGNN3y7OmmpP6l2y-Mz6Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9DFBF1927800; Sat, 5 Jun 2021 11:10:45 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC6DA614FD; Sat, 5 Jun 2021 11:10:42 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 02/19] x86/ftrace: Remove fault protection code in prepare_ftrace_return Date: Sat, 5 Jun 2021 13:10:17 +0200 Message-Id: <20210605111034.1810858-3-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Steven Rostedt (VMware)" Removing the fault protection code when writing return_hooker to stack. As Steven noted: > That protection was there from the beginning due to being "paranoid", > considering ftrace was bricking network cards. But that protection > would not have even protected against that. Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Jiri Olsa --- arch/x86/kernel/ftrace.c | 38 +++----------------------------------- 1 file changed, 3 insertions(+), 35 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 1b3ce3b4a2a2..c555624da989 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -625,12 +625,10 @@ int ftrace_disable_ftrace_graph_caller(void) * Hook the return address and push it in the stack of return addrs * in current thread info. */ -void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, +void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer) { unsigned long return_hooker = (unsigned long)&return_to_handler; - unsigned long old; - int faulted; /* * When resuming from suspend-to-ram, this function can be indirectly @@ -650,37 +648,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, if (unlikely(atomic_read(¤t->tracing_graph_pause))) return; - /* - * Protect against fault, even if it shouldn't - * happen. This tool is too much intrusive to - * ignore such a protection. - */ - asm volatile( - "1: " _ASM_MOV " (%[parent]), %[old]\n" - "2: " _ASM_MOV " %[return_hooker], (%[parent])\n" - " movl $0, %[faulted]\n" - "3:\n" - - ".section .fixup, \"ax\"\n" - "4: movl $1, %[faulted]\n" - " jmp 3b\n" - ".previous\n" - - _ASM_EXTABLE(1b, 4b) - _ASM_EXTABLE(2b, 4b) - - : [old] "=&r" (old), [faulted] "=r" (faulted) - : [parent] "r" (parent), [return_hooker] "r" (return_hooker) - : "memory" - ); - - if (unlikely(faulted)) { - ftrace_graph_stop(); - WARN_ON(1); - return; - } - - if (function_graph_enter(old, self_addr, frame_pointer, parent)) - *parent = old; + if (!function_graph_enter(*parent, ip, frame_pointer, parent)) + *parent = return_hooker; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ From patchwork Sat Jun 5 11:10:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB07AC4743D for ; Sat, 5 Jun 2021 11:10:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEFF1613B4 for ; Sat, 5 Jun 2021 11:10:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230196AbhFELMn convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:12:43 -0400 Received: from us-smtp-delivery-44.mimecast.com ([207.211.30.44]:32857 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230118AbhFELMm (ORCPT ); Sat, 5 Jun 2021 07:12:42 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-473-tpueDtf4NaWC1xCEwPAJQQ-1; Sat, 05 Jun 2021 07:10:50 -0400 X-MC-Unique: tpueDtf4NaWC1xCEwPAJQQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B35A5803620; Sat, 5 Jun 2021 11:10:48 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id F0D0A614FD; Sat, 5 Jun 2021 11:10:45 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 03/19] x86/ftrace: Make function graph use ftrace directly Date: Sat, 5 Jun 2021 13:10:18 +0200 Message-Id: <20210605111034.1810858-4-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Steven Rostedt (VMware)" We don't need special hook for graph tracer entry point, but instead we can use graph_ops::func function to install the return_hooker. This moves the graph tracing setup _before_ the direct trampoline prepares the stack, so the return_hooker will be called when the direct trampoline is finished. This simplifies the code, because we don't need to take into account the direct trampoline setup when preparing the graph tracer hooker and we can allow function graph tracer on entries registered with direct trampoline. Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Jiri Olsa --- arch/x86/include/asm/ftrace.h | 9 +++++++-- arch/x86/kernel/ftrace.c | 37 ++++++++++++++++++++++++++++++++--- arch/x86/kernel/ftrace_64.S | 29 +-------------------------- include/linux/ftrace.h | 6 ++++++ kernel/trace/fgraph.c | 8 +++++--- 5 files changed, 53 insertions(+), 36 deletions(-) diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h index 9f3130f40807..024d9797646e 100644 --- a/arch/x86/include/asm/ftrace.h +++ b/arch/x86/include/asm/ftrace.h @@ -57,6 +57,13 @@ arch_ftrace_get_regs(struct ftrace_regs *fregs) #define ftrace_instruction_pointer_set(fregs, _ip) \ do { (fregs)->regs.ip = (_ip); } while (0) + +struct ftrace_ops; +#define ftrace_graph_func ftrace_graph_func +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs); +#else +#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR #endif #ifdef CONFIG_DYNAMIC_FTRACE @@ -65,8 +72,6 @@ struct dyn_arch_ftrace { /* No extra data needed for x86 */ }; -#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR - #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* __ASSEMBLY__ */ #endif /* CONFIG_FUNCTION_TRACER */ diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index c555624da989..804fcc6ef2c7 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -527,7 +527,7 @@ static void *addr_from_call(void *ptr) return ptr + CALL_INSN_SIZE + call.disp; } -void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, +void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer); /* @@ -541,7 +541,8 @@ static void *static_tramp_func(struct ftrace_ops *ops, struct dyn_ftrace *rec) void *ptr; if (ops && ops->trampoline) { -#ifdef CONFIG_FUNCTION_GRAPH_TRACER +#if !defined(CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS) && \ + defined(CONFIG_FUNCTION_GRAPH_TRACER) /* * We only know about function graph tracer setting as static * trampoline. @@ -589,8 +590,9 @@ void arch_ftrace_trampoline_free(struct ftrace_ops *ops) #ifdef CONFIG_FUNCTION_GRAPH_TRACER #ifdef CONFIG_DYNAMIC_FTRACE -extern void ftrace_graph_call(void); +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +extern void ftrace_graph_call(void); static const char *ftrace_jmp_replace(unsigned long ip, unsigned long addr) { return text_gen_insn(JMP32_INSN_OPCODE, (void *)ip, (void *)addr); @@ -618,7 +620,17 @@ int ftrace_disable_ftrace_graph_caller(void) return ftrace_mod_jmp(ip, &ftrace_stub); } +#else /* !CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ +int ftrace_enable_ftrace_graph_caller(void) +{ + return 0; +} +int ftrace_disable_ftrace_graph_caller(void) +{ + return 0; +} +#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* !CONFIG_DYNAMIC_FTRACE */ /* @@ -629,6 +641,7 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer) { unsigned long return_hooker = (unsigned long)&return_to_handler; + int bit; /* * When resuming from suspend-to-ram, this function can be indirectly @@ -648,7 +661,25 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, if (unlikely(atomic_read(¤t->tracing_graph_pause))) return; + bit = ftrace_test_recursion_trylock(ip, *parent); + if (bit < 0) + return; + if (!function_graph_enter(*parent, ip, frame_pointer, parent)) *parent = return_hooker; + + ftrace_test_recursion_unlock(bit); +} + +#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs) +{ + struct pt_regs *regs = &fregs->regs; + unsigned long *stack = (unsigned long *)kernel_stack_pointer(regs); + + prepare_ftrace_return(ip, (unsigned long *)stack, 0); } +#endif + #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index a8eb084a7a9a..7a879901f103 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -174,11 +174,6 @@ SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL) SYM_FUNC_END(ftrace_caller); SYM_FUNC_START(ftrace_epilogue) -#ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - jmp ftrace_stub -#endif - /* * This is weak to keep gas from relaxing the jumps. * It is also used to copy the retq for trampolines. @@ -288,15 +283,6 @@ SYM_FUNC_START(__fentry__) cmpq $ftrace_stub, ftrace_trace_function jnz trace -fgraph_trace: -#ifdef CONFIG_FUNCTION_GRAPH_TRACER - cmpq $ftrace_stub, ftrace_graph_return - jnz ftrace_graph_caller - - cmpq $ftrace_graph_entry_stub, ftrace_graph_entry - jnz ftrace_graph_caller -#endif - SYM_INNER_LABEL(ftrace_stub, SYM_L_GLOBAL) retq @@ -314,25 +300,12 @@ trace: CALL_NOSPEC r8 restore_mcount_regs - jmp fgraph_trace + jmp ftrace_stub SYM_FUNC_END(__fentry__) EXPORT_SYMBOL(__fentry__) #endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_FUNC_START(ftrace_graph_caller) - /* Saves rbp into %rdx and fills first parameter */ - save_mcount_regs - - leaq MCOUNT_REG_SIZE+8(%rsp), %rsi - movq $0, %rdx /* No framepointers needed */ - call prepare_ftrace_return - - restore_mcount_regs - - retq -SYM_FUNC_END(ftrace_graph_caller) - SYM_FUNC_START(return_to_handler) subq $24, %rsp diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a69f363b61bf..40b493908f09 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -614,6 +614,12 @@ void ftrace_modify_all_code(int command); extern void ftrace_graph_caller(void); extern int ftrace_enable_ftrace_graph_caller(void); extern int ftrace_disable_ftrace_graph_caller(void); +#ifndef ftrace_graph_func +#define ftrace_graph_func ftrace_stub +#define FTRACE_OPS_GRAPH_STUB | FTRACE_OPS_FL_STUB +#else +#define FTRACE_OPS_GRAPH_STUB +#endif #else static inline int ftrace_enable_ftrace_graph_caller(void) { return 0; } static inline int ftrace_disable_ftrace_graph_caller(void) { return 0; } diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index b8a0d1d564fb..58e96b45e9da 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -115,6 +115,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, { struct ftrace_graph_ent trace; +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS /* * Skip graph tracing if the return location is served by direct trampoline, * since call sequence and return addresses are unpredictable anyway. @@ -124,6 +125,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, if (ftrace_direct_func_count && ftrace_find_rec_direct(ret - MCOUNT_INSN_SIZE)) return -EBUSY; +#endif trace.func = func; trace.depth = ++current->curr_ret_depth; @@ -333,10 +335,10 @@ unsigned long ftrace_graph_ret_addr(struct task_struct *task, int *idx, #endif /* HAVE_FUNCTION_GRAPH_RET_ADDR_PTR */ static struct ftrace_ops graph_ops = { - .func = ftrace_stub, + .func = ftrace_graph_func, .flags = FTRACE_OPS_FL_INITIALIZED | - FTRACE_OPS_FL_PID | - FTRACE_OPS_FL_STUB, + FTRACE_OPS_FL_PID + FTRACE_OPS_GRAPH_STUB, #ifdef FTRACE_GRAPH_TRAMP_ADDR .trampoline = FTRACE_GRAPH_TRAMP_ADDR, /* trampoline_size is only needed for dynamically allocated tramps */ From patchwork Sat Jun 5 11:10:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454962 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76A8BC4743D for ; Sat, 5 Jun 2021 11:11:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 611BA61354 for ; Sat, 5 Jun 2021 11:11:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbhFELMw convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:12:52 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:30655 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230209AbhFELMu (ORCPT ); Sat, 5 Jun 2021 07:12:50 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-594-l4h8A3QKMq-RhYGJ235PZQ-1; Sat, 05 Jun 2021 07:10:57 -0400 X-MC-Unique: l4h8A3QKMq-RhYGJ235PZQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D450C801817; Sat, 5 Jun 2021 11:10:55 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 289BE2B0B1; Sat, 5 Jun 2021 11:10:51 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 05/19] ftrace: Add ftrace_add_rec_direct function Date: Sat, 5 Jun 2021 13:10:20 +0200 Message-Id: <20210605111034.1810858-6-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Factor out the code that adds (ip, addr) tuple to direct_functions hash in new ftrace_add_rec_direct function. It will be used in following patches. Signed-off-by: Jiri Olsa --- kernel/trace/ftrace.c | 60 ++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 24 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 2e8a3fde7104..9e584710f542 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2388,6 +2388,39 @@ unsigned long ftrace_find_rec_direct(unsigned long ip) return entry->direct; } +static struct ftrace_func_entry* +ftrace_add_rec_direct(unsigned long ip, unsigned long addr, + struct ftrace_hash **free_hash) +{ + struct ftrace_func_entry *entry; + + if (ftrace_hash_empty(direct_functions) || + direct_functions->count > 2 * (1 << direct_functions->size_bits)) { + struct ftrace_hash *new_hash; + int size = ftrace_hash_empty(direct_functions) ? 0 : + direct_functions->count + 1; + + if (size < 32) + size = 32; + + new_hash = dup_hash(direct_functions, size); + if (!new_hash) + return NULL; + + *free_hash = direct_functions; + direct_functions = new_hash; + } + + entry = kmalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return NULL; + + entry->ip = ip; + entry->direct = addr; + __add_hash_entry(direct_functions, entry); + return entry; +} + static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { @@ -5105,27 +5138,6 @@ int register_ftrace_direct(unsigned long ip, unsigned long addr) } ret = -ENOMEM; - if (ftrace_hash_empty(direct_functions) || - direct_functions->count > 2 * (1 << direct_functions->size_bits)) { - struct ftrace_hash *new_hash; - int size = ftrace_hash_empty(direct_functions) ? 0 : - direct_functions->count + 1; - - if (size < 32) - size = 32; - - new_hash = dup_hash(direct_functions, size); - if (!new_hash) - goto out_unlock; - - free_hash = direct_functions; - direct_functions = new_hash; - } - - entry = kmalloc(sizeof(*entry), GFP_KERNEL); - if (!entry) - goto out_unlock; - direct = ftrace_find_direct_func(addr); if (!direct) { direct = ftrace_alloc_direct_func(addr); @@ -5135,9 +5147,9 @@ int register_ftrace_direct(unsigned long ip, unsigned long addr) } } - entry->ip = ip; - entry->direct = addr; - __add_hash_entry(direct_functions, entry); + entry = ftrace_add_rec_direct(ip, addr, &free_hash); + if (!entry) + goto out_unlock; ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); if (ret) From patchwork Sat Jun 5 11:10:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E941FC47082 for ; Sat, 5 Jun 2021 11:11:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1B8B613B4 for ; Sat, 5 Jun 2021 11:11:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbhFELNA convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:00 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:35639 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230242AbhFELM7 (ORCPT ); Sat, 5 Jun 2021 07:12:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-146-oascXX71PQWJyz-sFqAXYg-1; Sat, 05 Jun 2021 07:11:05 -0400 X-MC-Unique: oascXX71PQWJyz-sFqAXYg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 01133801817; Sat, 5 Jun 2021 11:11:04 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D52B614FD; Sat, 5 Jun 2021 11:11:01 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 08/19] ftrace/samples: Add multi direct interface test module Date: Sat, 5 Jun 2021 13:10:23 +0200 Message-Id: <20210605111034.1810858-9-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding simple module that uses multi direct interface: register_ftrace_direct_multi unregister_ftrace_direct_multi The init function registers trampoline for 2 functions, and exit function unregisters them. Signed-off-by: Jiri Olsa --- samples/ftrace/Makefile | 1 + samples/ftrace/ftrace-direct-multi.c | 52 ++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 samples/ftrace/ftrace-direct-multi.c diff --git a/samples/ftrace/Makefile b/samples/ftrace/Makefile index 4ce896e10b2e..ab1d1c05c288 100644 --- a/samples/ftrace/Makefile +++ b/samples/ftrace/Makefile @@ -3,6 +3,7 @@ obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-too.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-modify.o +obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-multi.o CFLAGS_sample-trace-array.o := -I$(src) obj-$(CONFIG_SAMPLE_TRACE_ARRAY) += sample-trace-array.o diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c new file mode 100644 index 000000000000..76b34d46d11c --- /dev/null +++ b/samples/ftrace/ftrace-direct-multi.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include + +#include /* for handle_mm_fault() */ +#include +#include + +void my_direct_func(unsigned long ip) +{ + trace_printk("ip %lx\n", ip); +} + +extern void my_tramp(void *); + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:" +" pushq %rbp\n" +" movq %rsp, %rbp\n" +" pushq %rdi\n" +" movq 8(%rbp), %rdi\n" +" call my_direct_func\n" +" popq %rdi\n" +" leave\n" +" ret\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +static struct ftrace_ops direct; + +static int __init ftrace_direct_multi_init(void) +{ + ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); + ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); +} + +static void __exit ftrace_direct_multi_exit(void) +{ + unregister_ftrace_direct_multi(&direct); +} + +module_init(ftrace_direct_multi_init); +module_exit(ftrace_direct_multi_exit); + +MODULE_AUTHOR("Jiri Olsa"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); +MODULE_LICENSE("GPL"); From patchwork Sat Jun 5 11:10:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98EF9C47082 for ; Sat, 5 Jun 2021 11:11:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8301B61354 for ; Sat, 5 Jun 2021 11:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230270AbhFELNI convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:08 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:35951 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229998AbhFELNH (ORCPT ); Sat, 5 Jun 2021 07:13:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-326-ScToTuwDNR-CHNWvJeJnTg-1; Sat, 05 Jun 2021 07:11:13 -0400 X-MC-Unique: ScToTuwDNR-CHNWvJeJnTg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F3281802B4F; Sat, 5 Jun 2021 11:11:11 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6503C614FD; Sat, 5 Jun 2021 11:11:07 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 10/19] bpf: Allow to store caller's ip as argument Date: Sat, 5 Jun 2021 13:10:25 +0200 Message-Id: <20210605111034.1810858-11-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When we will have multiple functions attached to trampoline we need to propagate the function's address to the bpf program. Adding new BPF_TRAMP_F_IP_ARG flag to arch_prepare_bpf_trampoline function that will store origin caller's address before function's arguments. Signed-off-by: Jiri Olsa --- arch/x86/net/bpf_jit_comp.c | 18 ++++++++++++++---- include/linux/bpf.h | 5 +++++ 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index b77e6bd78354..d2425c18272a 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1951,7 +1951,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i void *orig_call) { int ret, i, cnt = 0, nr_args = m->nr_args; - int stack_size = nr_args * 8; + int stack_size = nr_args * 8, ip_arg = 0; struct bpf_tramp_progs *fentry = &tprogs[BPF_TRAMP_FENTRY]; struct bpf_tramp_progs *fexit = &tprogs[BPF_TRAMP_FEXIT]; struct bpf_tramp_progs *fmod_ret = &tprogs[BPF_TRAMP_MODIFY_RETURN]; @@ -1975,6 +1975,9 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i */ orig_call += X86_PATCH_SIZE; + if (flags & BPF_TRAMP_F_IP_ARG) + stack_size += 8; + prog = image; EMIT1(0x55); /* push rbp */ @@ -1982,7 +1985,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i EMIT4(0x48, 0x83, 0xEC, stack_size); /* sub rsp, stack_size */ EMIT1(0x53); /* push rbx */ - save_regs(m, &prog, nr_args, stack_size); + if (flags & BPF_TRAMP_F_IP_ARG) { + emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); + EMIT4(0x48, 0x83, 0xe8, X86_PATCH_SIZE); /* sub $X86_PATCH_SIZE,%rax*/ + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -stack_size); + ip_arg = 8; + } + + save_regs(m, &prog, nr_args, stack_size - ip_arg); if (flags & BPF_TRAMP_F_CALL_ORIG) { /* arg1: mov rdi, im */ @@ -2011,7 +2021,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (flags & BPF_TRAMP_F_CALL_ORIG) { - restore_regs(m, &prog, nr_args, stack_size); + restore_regs(m, &prog, nr_args, stack_size - ip_arg); if (flags & BPF_TRAMP_F_ORIG_STACK) { emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); @@ -2052,7 +2062,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i } if (flags & BPF_TRAMP_F_RESTORE_REGS) - restore_regs(m, &prog, nr_args, stack_size); + restore_regs(m, &prog, nr_args, stack_size - ip_arg); /* This needs to be done regardless. If there were fmod_ret programs, * the return value is only updated on the stack and still needs to be diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 16fc600503fb..6cbf3c81c650 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -559,6 +559,11 @@ struct btf_func_model { */ #define BPF_TRAMP_F_ORIG_STACK BIT(3) +/* First argument is IP address of the caller. Makes sense for fentry/fexit + * programs only. + */ +#define BPF_TRAMP_F_IP_ARG BIT(4) + /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. Pick a number to fit into BPF_IMAGE_SIZE / 2 */ From patchwork Sat Jun 5 11:10:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54BBFC4743E for ; Sat, 5 Jun 2021 11:11:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3D5EC61354 for ; Sat, 5 Jun 2021 11:11:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230299AbhFELNN convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:13 -0400 Received: from us-smtp-delivery-44.mimecast.com ([207.211.30.44]:47130 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230281AbhFELNL (ORCPT ); Sat, 5 Jun 2021 07:13:11 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-261-szM1dAJAMbmo1x1MJsOcQg-1; Sat, 05 Jun 2021 07:11:17 -0400 X-MC-Unique: szM1dAJAMbmo1x1MJsOcQg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 07E21801817; Sat, 5 Jun 2021 11:11:15 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 51248614FD; Sat, 5 Jun 2021 11:11:12 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 11/19] bpf: Add support to load multi func tracing program Date: Sat, 5 Jun 2021 13:10:26 +0200 Message-Id: <20210605111034.1810858-12-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding support to load tracing program with new BPF_F_MULTI_FUNC flag, that allows the program to be loaded without specific function to be attached to. The verifier assumes the program is using all (6) available arguments as unsigned long values. We can't add extra ip argument at this time, because JIT on x86 would fail to process this function. Instead we allow to access extra first 'ip' argument in btf_ctx_access. Such program will be allowed to be attached to multiple functions in following patches. Signed-off-by: Jiri Olsa --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 7 +++++++ kernel/bpf/btf.c | 5 +++++ kernel/bpf/syscall.c | 35 +++++++++++++++++++++++++++++----- kernel/bpf/verifier.c | 3 ++- tools/include/uapi/linux/bpf.h | 7 +++++++ 6 files changed, 52 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 6cbf3c81c650..23221e0e8d3c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -845,6 +845,7 @@ struct bpf_prog_aux { bool sleepable; bool tail_call_reachable; struct hlist_node tramp_hlist; + bool multi_func; /* BTF_KIND_FUNC_PROTO for valid attach_btf_id */ const struct btf_type *attach_func_proto; /* function name for valid attach_btf_id */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 2c1ba70abbf1..ad9340fb14d4 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1109,6 +1109,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index a6e39c5ea0bf..c233aaa6a709 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -4679,6 +4679,11 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, args++; nr_args--; } + if (prog->aux->multi_func) { + if (arg == 0) + return true; + arg--; + } if (arg > nr_args) { bpf_log(log, "func '%s' doesn't have %d-th argument\n", diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 50457019da27..8f59090280b5 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -31,6 +31,7 @@ #include #include #include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -1979,7 +1980,8 @@ static int bpf_prog_load_check_attach(enum bpf_prog_type prog_type, enum bpf_attach_type expected_attach_type, struct btf *attach_btf, u32 btf_id, - struct bpf_prog *dst_prog) + struct bpf_prog *dst_prog, + bool multi_func) { if (btf_id) { if (btf_id > BTF_MAX_TYPE) @@ -1999,6 +2001,14 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type, } } + if (multi_func) { + if (prog_type != BPF_PROG_TYPE_TRACING) + return -EINVAL; + if (!attach_btf || btf_id) + return -EINVAL; + return 0; + } + if (attach_btf && (!btf_id || dst_prog)) return -EINVAL; @@ -2114,6 +2124,16 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type) } } +#define DEFINE_BPF_MULTI_FUNC(args...) \ + extern int bpf_multi_func(args); \ + int __init bpf_multi_func(args) { return 0; } + +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, + unsigned long a5, unsigned long a6) + +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) + /* last field in 'union bpf_attr' used by this command */ #define BPF_PROG_LOAD_LAST_FIELD fd_array @@ -2124,6 +2144,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) struct btf *attach_btf = NULL; int err; char license[128]; + bool multi_func; bool is_gpl; if (CHECK_ATTR(BPF_PROG_LOAD)) @@ -2133,7 +2154,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) BPF_F_ANY_ALIGNMENT | BPF_F_TEST_STATE_FREQ | BPF_F_SLEEPABLE | - BPF_F_TEST_RND_HI32)) + BPF_F_TEST_RND_HI32 | + BPF_F_MULTI_FUNC)) return -EINVAL; if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && @@ -2164,6 +2186,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) if (is_perfmon_prog_type(type) && !perfmon_capable()) return -EPERM; + multi_func = attr->prog_flags & BPF_F_MULTI_FUNC; + /* attach_prog_fd/attach_btf_obj_fd can specify fd of either bpf_prog * or btf, we need to check which one it is */ @@ -2182,7 +2206,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) return -ENOTSUPP; } } - } else if (attr->attach_btf_id) { + } else if (attr->attach_btf_id || multi_func) { /* fall back to vmlinux BTF, if BTF type ID is specified */ attach_btf = bpf_get_btf_vmlinux(); if (IS_ERR(attach_btf)) @@ -2195,7 +2219,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) bpf_prog_load_fixup_attach_type(attr); if (bpf_prog_load_check_attach(type, attr->expected_attach_type, attach_btf, attr->attach_btf_id, - dst_prog)) { + dst_prog, multi_func)) { if (dst_prog) bpf_prog_put(dst_prog); if (attach_btf) @@ -2215,10 +2239,11 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) prog->expected_attach_type = attr->expected_attach_type; prog->aux->attach_btf = attach_btf; - prog->aux->attach_btf_id = attr->attach_btf_id; + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; prog->aux->dst_prog = dst_prog; prog->aux->offload_requested = !!attr->prog_ifindex; prog->aux->sleepable = attr->prog_flags & BPF_F_SLEEPABLE; + prog->aux->multi_func = multi_func; err = security_bpf_prog_alloc(prog->aux); if (err) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1de4b8c6ee42..194adddee2ec 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13276,7 +13276,8 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) if (!bpf_iter_prog_supported(prog)) return -EINVAL; return 0; - } + } else if (prog->aux->multi_func) + return prog->type == BPF_PROG_TYPE_TRACING ? 0 : -EINVAL; if (prog->type == BPF_PROG_TYPE_LSM) { ret = bpf_lsm_verify_prog(&env->log, prog); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 2c1ba70abbf1..ad9340fb14d4 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1109,6 +1109,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * From patchwork Sat Jun 5 11:10:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CFDFC4743F for ; Sat, 5 Jun 2021 11:11:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85ED661242 for ; Sat, 5 Jun 2021 11:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230322AbhFELNT convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:19 -0400 Received: from us-smtp-delivery-44.mimecast.com ([207.211.30.44]:46598 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230309AbhFELNR (ORCPT ); Sat, 5 Jun 2021 07:13:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-203-dgpVB63sMNugNYU4tMrT8g-1; Sat, 05 Jun 2021 07:11:26 -0400 X-MC-Unique: dgpVB63sMNugNYU4tMrT8g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B09B803620; Sat, 5 Jun 2021 11:11:24 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 753FE614FD; Sat, 5 Jun 2021 11:11:21 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 14/19] libbpf: Add btf__find_by_pattern_kind function Date: Sat, 5 Jun 2021 13:10:29 +0200 Message-Id: <20210605111034.1810858-15-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding btf__find_by_pattern_kind function that returns array of BTF ids for given function name pattern. Using libc's regex.h support for that. Signed-off-by: Jiri Olsa --- tools/lib/bpf/btf.c | 68 +++++++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/btf.h | 3 ++ 2 files changed, 71 insertions(+) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index b46760b93bb4..421dd6c1e44a 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) /* Copyright (c) 2018 Facebook */ +#define _GNU_SOURCE #include #include #include @@ -16,6 +17,7 @@ #include #include #include +#include #include "btf.h" #include "bpf.h" #include "libbpf.h" @@ -711,6 +713,72 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name, return libbpf_err(-ENOENT); } +static bool is_wildcard(char c) +{ + static const char *wildchars = "*?[|"; + + return strchr(wildchars, c); +} + +int btf__find_by_pattern_kind(const struct btf *btf, + const char *type_pattern, __u32 kind, + __s32 **__ids) +{ + __u32 i, nr_types = btf__get_nr_types(btf); + __s32 *ids = NULL; + int cnt = 0, alloc = 0, ret; + regex_t regex; + char *pattern; + + if (kind == BTF_KIND_UNKN || !strcmp(type_pattern, "void")) + return 0; + + /* When the pattern does not start with wildcard, treat it as + * if we'd want to match it from the beginning of the string. + */ + asprintf(&pattern, "%s%s", + is_wildcard(type_pattern[0]) ? "^" : "", + type_pattern); + + ret = regcomp(®ex, pattern, REG_EXTENDED); + if (ret) { + pr_warn("failed to compile regex\n"); + free(pattern); + return -EINVAL; + } + + free(pattern); + + for (i = 1; i <= nr_types; i++) { + const struct btf_type *t = btf__type_by_id(btf, i); + const char *name; + __s32 *p; + + if (btf_kind(t) != kind) + continue; + name = btf__name_by_offset(btf, t->name_off); + if (name && regexec(®ex, name, 0, NULL, 0)) + continue; + if (cnt == alloc) { + alloc = max(100, alloc * 3 / 2); + p = realloc(ids, alloc * sizeof(__u32)); + if (!p) { + free(ids); + regfree(®ex); + return -ENOMEM; + } + ids = p; + } + + ids[cnt] = i; + cnt++; + } + + regfree(®ex); + *__ids = ids; + return cnt ?: -ENOENT; +} + static bool btf_is_modifiable(const struct btf *btf) { return (void *)btf->hdr != btf->raw_data; diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index b54f1c3ebd57..036857aded94 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -371,6 +371,9 @@ btf_var_secinfos(const struct btf_type *t) return (struct btf_var_secinfo *)(t + 1); } +int btf__find_by_pattern_kind(const struct btf *btf, + const char *type_pattern, __u32 kind, + __s32 **__ids); #ifdef __cplusplus } /* extern "C" */ #endif From patchwork Sat Jun 5 11:10:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07D1EC4743D for ; Sat, 5 Jun 2021 11:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E848A610A8 for ; Sat, 5 Jun 2021 11:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbhFELNY convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:24 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:43631 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230330AbhFELNX (ORCPT ); Sat, 5 Jun 2021 07:13:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-558-wYdvKePDPZKEVCDa-jvReg-1; Sat, 05 Jun 2021 07:11:31 -0400 X-MC-Unique: wYdvKePDPZKEVCDa-jvReg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4CA50801817; Sat, 5 Jun 2021 11:11:30 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8DC16614FD; Sat, 5 Jun 2021 11:11:27 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 16/19] selftests/bpf: Add fentry multi func test Date: Sat, 5 Jun 2021 13:10:31 +0200 Message-Id: <20210605111034.1810858-17-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adding selftest for fentry multi func test that attaches to bpf_fentry_test* functions and checks argument values based on the processed function. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/multi_check.h | 52 +++++++++++++++++++ .../bpf/prog_tests/fentry_multi_test.c | 43 +++++++++++++++ .../selftests/bpf/progs/fentry_multi_test.c | 18 +++++++ 3 files changed, 113 insertions(+) create mode 100644 tools/testing/selftests/bpf/multi_check.h create mode 100644 tools/testing/selftests/bpf/prog_tests/fentry_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/fentry_multi_test.c diff --git a/tools/testing/selftests/bpf/multi_check.h b/tools/testing/selftests/bpf/multi_check.h new file mode 100644 index 000000000000..36c2a93f9be3 --- /dev/null +++ b/tools/testing/selftests/bpf/multi_check.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __MULTI_CHECK_H +#define __MULTI_CHECK_H + +extern unsigned long long bpf_fentry_test[8]; + +static __attribute__((unused)) inline +void multi_arg_check(unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 *test_result) +{ + if (ip == bpf_fentry_test[0]) { + *test_result += (int) a == 1; + } else if (ip == bpf_fentry_test[1]) { + *test_result += (int) a == 2 && (__u64) b == 3; + } else if (ip == bpf_fentry_test[2]) { + *test_result += (char) a == 4 && (int) b == 5 && (__u64) c == 6; + } else if (ip == bpf_fentry_test[3]) { + *test_result += (void *) a == (void *) 7 && (char) b == 8 && (int) c == 9 && (__u64) d == 10; + } else if (ip == bpf_fentry_test[4]) { + *test_result += (__u64) a == 11 && (void *) b == (void *) 12 && (short) c == 13 && (int) d == 14 && (__u64) e == 15; + } else if (ip == bpf_fentry_test[5]) { + *test_result += (__u64) a == 16 && (void *) b == (void *) 17 && (short) c == 18 && (int) d == 19 && (void *) e == (void *) 20 && (__u64) f == 21; + } else if (ip == bpf_fentry_test[6]) { + *test_result += 1; + } else if (ip == bpf_fentry_test[7]) { + *test_result += 1; + } +} + +static __attribute__((unused)) inline +void multi_ret_check(unsigned long ip, int ret, __u64 *test_result) +{ + if (ip == bpf_fentry_test[0]) { + *test_result += ret == 2; + } else if (ip == bpf_fentry_test[1]) { + *test_result += ret == 5; + } else if (ip == bpf_fentry_test[2]) { + *test_result += ret == 15; + } else if (ip == bpf_fentry_test[3]) { + *test_result += ret == 34; + } else if (ip == bpf_fentry_test[4]) { + *test_result += ret == 65; + } else if (ip == bpf_fentry_test[5]) { + *test_result += ret == 111; + } else if (ip == bpf_fentry_test[6]) { + *test_result += ret == 0; + } else if (ip == bpf_fentry_test[7]) { + *test_result += ret == 0; + } +} + +#endif /* __MULTI_CHECK_H */ diff --git a/tools/testing/selftests/bpf/prog_tests/fentry_multi_test.c b/tools/testing/selftests/bpf/prog_tests/fentry_multi_test.c new file mode 100644 index 000000000000..e4a8089533d6 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/fentry_multi_test.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "fentry_multi_test.skel.h" +#include "trace_helpers.h" + +void test_fentry_multi_test(void) +{ + struct fentry_multi_test *skel = NULL; + unsigned long long *bpf_fentry_test; + __u32 duration = 0, retval; + int err, prog_fd; + + skel = fentry_multi_test__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fentry_multi_skel_load")) + goto cleanup; + + bpf_fentry_test = &skel->bss->bpf_fentry_test[0]; + ASSERT_OK(kallsyms_find("bpf_fentry_test1", &bpf_fentry_test[0]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test2", &bpf_fentry_test[1]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test3", &bpf_fentry_test[2]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test4", &bpf_fentry_test[3]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test5", &bpf_fentry_test[4]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test6", &bpf_fentry_test[5]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test7", &bpf_fentry_test[6]), "kallsyms_find"); + ASSERT_OK(kallsyms_find("bpf_fentry_test8", &bpf_fentry_test[7]), "kallsyms_find"); + + err = fentry_multi_test__attach(skel); + if (!ASSERT_OK(err, "fentry_attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test_result, 8, "test_result"); + + fentry_multi_test__detach(skel); + +cleanup: + fentry_multi_test__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/fentry_multi_test.c b/tools/testing/selftests/bpf/progs/fentry_multi_test.c new file mode 100644 index 000000000000..a443fc958e5a --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fentry_multi_test.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include "multi_check.h" + +char _license[] SEC("license") = "GPL"; + +unsigned long long bpf_fentry_test[8]; + +__u64 test_result = 0; + +SEC("fentry.multi/bpf_fentry_test*") +int BPF_PROG(test, unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) +{ + multi_arg_check(ip, a, b, c, d, e, f, &test_result); + return 0; +} From patchwork Sat Jun 5 11:10:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 454956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A213BC4743E for ; Sat, 5 Jun 2021 11:11:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 884B261244 for ; Sat, 5 Jun 2021 11:11:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230366AbhFELNh convert rfc822-to-8bit (ORCPT ); Sat, 5 Jun 2021 07:13:37 -0400 Received: from us-smtp-delivery-44.mimecast.com ([205.139.111.44]:48054 "EHLO us-smtp-delivery-44.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230403AbhFELNb (ORCPT ); Sat, 5 Jun 2021 07:13:31 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-200-iW_eKBkEOEmHtfj0-I_cIg-1; Sat, 05 Jun 2021 07:11:41 -0400 X-MC-Unique: iW_eKBkEOEmHtfj0-I_cIg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74AA4803622; Sat, 5 Jun 2021 11:11:39 +0000 (UTC) Received: from krava.cust.in.nbox.cz (unknown [10.40.192.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA2D6614FD; Sat, 5 Jun 2021 11:11:36 +0000 (UTC) From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Steven Rostedt (VMware)" Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Daniel Xu , Viktor Malik Subject: [PATCH 19/19] selftests/bpf: Temporary fix for fentry_fexit_multi_test Date: Sat, 5 Jun 2021 13:10:34 +0200 Message-Id: <20210605111034.1810858-20-jolsa@kernel.org> In-Reply-To: <20210605111034.1810858-1-jolsa@kernel.org> References: <20210605111034.1810858-1-jolsa@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jolsa@kernel.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When multi_arg_check is used from 2 different places I'm getting compilation fail, which I did not deciphered yet: $ CLANG=/opt/clang/bin/clang LLC=/opt/clang/bin/llc make CLNG-BPF [test_maps] fentry_fexit_multi_test.o progs/fentry_fexit_multi_test.c:18:2: error: too many args to t24: i64 = \ GlobalAddress 0, \ progs/fentry_fexit_multi_test.c:18:2 @[ progs/fentry_fexit_multi_test.c:16:5 ] multi_arg_check(ip, a, b, c, d, e, f, &test1_arg_result); ^ progs/fentry_fexit_multi_test.c:25:2: error: too many args to t32: i64 = \ GlobalAddress 0, \ progs/fentry_fexit_multi_test.c:25:2 @[ progs/fentry_fexit_multi_test.c:23:5 ] multi_arg_check(ip, a, b, c, d, e, f, &test2_arg_result); ^ In file included from progs/fentry_fexit_multi_test.c:5: /home/jolsa/linux-qemu/tools/testing/selftests/bpf/multi_check.h:9:6: error: defined with too many args void multi_arg_check(unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 *test_result) ^ /home/jolsa/linux-qemu/tools/testing/selftests/bpf/multi_check.h:9:6: error: defined with too many args /home/jolsa/linux-qemu/tools/testing/selftests/bpf/multi_check.h:9:6: error: defined with too many args 5 errors generated. make: *** [Makefile:470: /home/jolsa/linux-qemu/tools/testing/selftests/bpf/fentry_fexit_multi_test.o] Error 1 As a temporary fix adding 2 instaces of multi_arg_check function, one for each caller. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/multi_check.h | 41 ++++++++++--------- .../bpf/progs/fentry_fexit_multi_test.c | 7 +++- .../selftests/bpf/progs/fentry_multi_test.c | 4 +- .../selftests/bpf/progs/fexit_multi_test.c | 4 +- 4 files changed, 32 insertions(+), 24 deletions(-) diff --git a/tools/testing/selftests/bpf/multi_check.h b/tools/testing/selftests/bpf/multi_check.h index 36c2a93f9be3..f720a6f9c6e4 100644 --- a/tools/testing/selftests/bpf/multi_check.h +++ b/tools/testing/selftests/bpf/multi_check.h @@ -5,26 +5,27 @@ extern unsigned long long bpf_fentry_test[8]; -static __attribute__((unused)) inline -void multi_arg_check(unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 *test_result) -{ - if (ip == bpf_fentry_test[0]) { - *test_result += (int) a == 1; - } else if (ip == bpf_fentry_test[1]) { - *test_result += (int) a == 2 && (__u64) b == 3; - } else if (ip == bpf_fentry_test[2]) { - *test_result += (char) a == 4 && (int) b == 5 && (__u64) c == 6; - } else if (ip == bpf_fentry_test[3]) { - *test_result += (void *) a == (void *) 7 && (char) b == 8 && (int) c == 9 && (__u64) d == 10; - } else if (ip == bpf_fentry_test[4]) { - *test_result += (__u64) a == 11 && (void *) b == (void *) 12 && (short) c == 13 && (int) d == 14 && (__u64) e == 15; - } else if (ip == bpf_fentry_test[5]) { - *test_result += (__u64) a == 16 && (void *) b == (void *) 17 && (short) c == 18 && (int) d == 19 && (void *) e == (void *) 20 && (__u64) f == 21; - } else if (ip == bpf_fentry_test[6]) { - *test_result += 1; - } else if (ip == bpf_fentry_test[7]) { - *test_result += 1; - } +#define MULTI_ARG_CHECK(_name) \ +static __attribute__((unused)) inline \ +void _name ## _multi_arg_check(unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 *test_result) \ +{ \ + if (ip == bpf_fentry_test[0]) { \ + *test_result += (int) a == 1; \ + } else if (ip == bpf_fentry_test[1]) { \ + *test_result += (int) a == 2 && (__u64) b == 3; \ + } else if (ip == bpf_fentry_test[2]) { \ + *test_result += (char) a == 4 && (int) b == 5 && (__u64) c == 6; \ + } else if (ip == bpf_fentry_test[3]) { \ + *test_result += (void *) a == (void *) 7 && (char) b == 8 && (int) c == 9 && (__u64) d == 10; \ + } else if (ip == bpf_fentry_test[4]) { \ + *test_result += (__u64) a == 11 && (void *) b == (void *) 12 && (short) c == 13 && (int) d == 14 && (__u64) e == 15; \ + } else if (ip == bpf_fentry_test[5]) { \ + *test_result += (__u64) a == 16 && (void *) b == (void *) 17 && (short) c == 18 && (int) d == 19 && (void *) e == (void *) 20 && (__u64) f == 21; \ + } else if (ip == bpf_fentry_test[6]) { \ + *test_result += 1; \ + } else if (ip == bpf_fentry_test[7]) { \ + *test_result += 1; \ + } \ } static __attribute__((unused)) inline diff --git a/tools/testing/selftests/bpf/progs/fentry_fexit_multi_test.c b/tools/testing/selftests/bpf/progs/fentry_fexit_multi_test.c index e25ab0085399..dc5b51f20b84 100644 --- a/tools/testing/selftests/bpf/progs/fentry_fexit_multi_test.c +++ b/tools/testing/selftests/bpf/progs/fentry_fexit_multi_test.c @@ -6,6 +6,9 @@ char _license[] SEC("license") = "GPL"; +MULTI_ARG_CHECK(fentry) +MULTI_ARG_CHECK(fexit) + unsigned long long bpf_fentry_test[8]; __u64 test1_arg_result = 0; @@ -15,14 +18,14 @@ __u64 test2_ret_result = 0; SEC("fentry.multi/bpf_fentry_test*") int BPF_PROG(test1, unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) { - multi_arg_check(ip, a, b, c, d, e, f, &test1_arg_result); + fentry_multi_arg_check(ip, a, b, c, d, e, f, &test1_arg_result); return 0; } SEC("fexit.multi/") int BPF_PROG(test2, unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) { - multi_arg_check(ip, a, b, c, d, e, f, &test2_arg_result); + fexit_multi_arg_check(ip, a, b, c, d, e, f, &test2_arg_result); multi_ret_check(ip, ret, &test2_ret_result); return 0; } diff --git a/tools/testing/selftests/bpf/progs/fentry_multi_test.c b/tools/testing/selftests/bpf/progs/fentry_multi_test.c index a443fc958e5a..b3a025632e77 100644 --- a/tools/testing/selftests/bpf/progs/fentry_multi_test.c +++ b/tools/testing/selftests/bpf/progs/fentry_multi_test.c @@ -6,6 +6,8 @@ char _license[] SEC("license") = "GPL"; +MULTI_ARG_CHECK(fentry) + unsigned long long bpf_fentry_test[8]; __u64 test_result = 0; @@ -13,6 +15,6 @@ __u64 test_result = 0; SEC("fentry.multi/bpf_fentry_test*") int BPF_PROG(test, unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f) { - multi_arg_check(ip, a, b, c, d, e, f, &test_result); + fentry_multi_arg_check(ip, a, b, c, d, e, f, &test_result); return 0; } diff --git a/tools/testing/selftests/bpf/progs/fexit_multi_test.c b/tools/testing/selftests/bpf/progs/fexit_multi_test.c index 365575cf05a0..8af0d65128d6 100644 --- a/tools/testing/selftests/bpf/progs/fexit_multi_test.c +++ b/tools/testing/selftests/bpf/progs/fexit_multi_test.c @@ -6,6 +6,8 @@ char _license[] SEC("license") = "GPL"; +MULTI_ARG_CHECK(fexit) + unsigned long long bpf_fentry_test[8]; __u64 test_arg_result = 0; @@ -14,7 +16,7 @@ __u64 test_ret_result = 0; SEC("fexit.multi/bpf_fentry_test*") int BPF_PROG(test, unsigned long ip, __u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, int ret) { - multi_arg_check(ip, a, b, c, d, e, f, &test_arg_result); + fexit_multi_arg_check(ip, a, b, c, d, e, f, &test_arg_result); multi_ret_check(ip, ret, &test_ret_result); return 0; }