From patchwork Fri Apr 24 02:44:08 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AKASHI Takahiro X-Patchwork-Id: 47483 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f71.google.com (mail-la0-f71.google.com [209.85.215.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 4065F2121F for ; Fri, 24 Apr 2015 02:45:36 +0000 (UTC) Received: by layy10 with SMTP id y10sf8856194lay.0 for ; Thu, 23 Apr 2015 19:45:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=/RYnL2+mwtv3cmbE3gJqLcJTpuuKwxJX0E7zjE8HOts=; b=A/sikOTM2C9i+Ou3wgc6oEo+MMQltEZrc3UwuTuL758QBHrclJ2A4fzBAV5gv2+7Ic c4dNgdat2VGpRoOHAWL2AuZ21T3TGvu2YuY9bMuz4al+DGHLtSts4Ly7spdFimc1Twyt CB25gYyG0Zqvb/+vr+TNdII9JDCXY9hBdsa3pj4KsTYsITzMjyFUEh9oDTNLLHlOFH7u 9TdJJNIkarmQyEVSP1PTQFQz+ZOUCbT/lzSTHi5Dd5TBrpWvSes2eWi3cJYnvKVt+iIb t5hQsGltSf675lfRpmrObVLtah3nd6jkitOPuN5Jmnjjgi2slOgX8AYlgsm5PgNlMtB3 y8YA== X-Gm-Message-State: ALoCoQkz0m0GwEG6ftilRgSoatDN2OqLnvQYmyFGGgLmGoLNbm/YOijw+EVgxVbbmdGLCn5Ohb/C X-Received: by 10.180.211.168 with SMTP id nd8mr689670wic.4.1429843535247; Thu, 23 Apr 2015 19:45:35 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.88.3 with SMTP id bc3ls433362lab.22.gmail; Thu, 23 Apr 2015 19:45:34 -0700 (PDT) X-Received: by 10.112.16.42 with SMTP id c10mr5011199lbd.103.1429843534786; Thu, 23 Apr 2015 19:45:34 -0700 (PDT) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com. [209.85.217.182]) by mx.google.com with ESMTPS id zs2si7239586lbb.124.2015.04.23.19.45.34 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2015 19:45:34 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.182 as permitted sender) client-ip=209.85.217.182; Received: by lbcga7 with SMTP id ga7so26780059lbc.1 for ; Thu, 23 Apr 2015 19:45:34 -0700 (PDT) X-Received: by 10.152.4.137 with SMTP id k9mr4940748lak.29.1429843534677; Thu, 23 Apr 2015 19:45:34 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.67.65 with SMTP id l1csp922867lbt; Thu, 23 Apr 2015 19:45:33 -0700 (PDT) X-Received: by 10.70.63.39 with SMTP id d7mr1969972pds.106.1429843531575; Thu, 23 Apr 2015 19:45:31 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c7si15269350pdn.193.2015.04.23.19.45.30; Thu, 23 Apr 2015 19:45:31 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754077AbbDXCpU (ORCPT + 27 others); Thu, 23 Apr 2015 22:45:20 -0400 Received: from mail-pa0-f52.google.com ([209.85.220.52]:34590 "EHLO mail-pa0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754028AbbDXCpO (ORCPT ); Thu, 23 Apr 2015 22:45:14 -0400 Received: by pacyx8 with SMTP id yx8so35096679pac.1 for ; Thu, 23 Apr 2015 19:45:13 -0700 (PDT) X-Received: by 10.68.89.100 with SMTP id bn4mr10813820pbb.48.1429843513761; Thu, 23 Apr 2015 19:45:13 -0700 (PDT) Received: from localhost.localdomain (61-205-1-167m5.grp1.mineo.jp. [61.205.1.167]) by mx.google.com with ESMTPSA id do16sm9488624pac.15.2015.04.23.19.45.06 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 23 Apr 2015 19:45:12 -0700 (PDT) From: AKASHI Takahiro To: rostedt@goodmis.org, mingo@kernel.org, jpoimboe@redhat.com, sjenning@redhat.com, jkosina@suse.cz, vojtech@suse.cz, catalin.marinas@arm.com, will.deacon@arm.com Cc: broonie@kernel.org, masami.hiramatsu.pt@hitachi.com, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linaro-kernel@lists.linaro.org, linux-kernel@vger.kernel.org, AKASHI Takahiro Subject: [RFC 3/4] arm64: ftrace: add DYNAMIC_TRACE_WITH_REGS version Date: Fri, 24 Apr 2015 11:44:08 +0900 Message-Id: <1429843449-7388-4-git-send-email-takahiro.akashi@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1429843449-7388-1-git-send-email-takahiro.akashi@linaro.org> References: <1429843449-7388-1-git-send-email-takahiro.akashi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: takahiro.akashi@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.182 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , CONFIG_DYNAMIC_TRACE_WITH_REGS is a prerequisite for ftrace-based kprobes as well as livepatch. This patch adds ftrace_regs_caller(), which will pass pt_regs info to ftrace handlers, to enable this config. Signed-off-by: AKASHI Takahiro --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/ftrace.h | 4 ++ arch/arm64/kernel/entry-ftrace.S | 124 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/ftrace.c | 24 +++++++- 4 files changed, 152 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1b8e973..c3678ed 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -54,6 +54,7 @@ config ARM64 select HAVE_DMA_ATTRS select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE + select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_FTRACE_MCOUNT_RECORD select HAVE_FUNCTION_TRACER diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h index c5534fa..5a14269 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -16,6 +16,10 @@ #define MCOUNT_ADDR ((unsigned long)_mcount) #define MCOUNT_INSN_SIZE AARCH64_INSN_SIZE +#ifdef CONFIG_DYNAMIC_FTRACE +#define ARCH_SUPPORTS_FTRACE_OPS 1 +#endif + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 08cafc5..fed28eb 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -10,6 +10,7 @@ */ #include +#include #include #include @@ -86,6 +87,95 @@ add \reg, \reg, #8 .endm +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +/* + * stack layout after mcount_save_regs in _mcount(): + * + * current sp/fp => 0:+---------+ + * in _mcount() |pt_regs | + * | | + * | +-----+ + * | | x29 | -> instrumented function's fp + * | +-----+ + * | | x30 | -> _mcount()'s lr + * | +-----+ (= instrumented function's pc) + * +S_FRAME_SIZE | | + * old sp => :+-----+--- + * when instrumented | | + * function calls | ... | + * _mcount() | | + * | | + * instrumented => +xx:+-----+ + * function's fp | x29 | -> parent's fp + * +-----+ + * | x30 | -> instrumented function's lr (= parent's pc) + * +-----+ + * | ... | + */ + .macro mcount_save_regs + sub sp, sp, #S_FRAME_SIZE + + stp x0, x1, [sp, #8 * 0] + stp x2, x3, [sp, #8 * 2] + stp x4, x5, [sp, #8 * 4] + stp x6, x7, [sp, #8 * 6] + stp x8, x9, [sp, #8 * 8] + stp x10, x11, [sp, #8 * 10] + stp x12, x13, [sp, #8 * 12] + stp x14, x15, [sp, #8 * 14] + stp x16, x17, [sp, #8 * 16] + stp x18, x19, [sp, #8 * 18] + stp x20, x21, [sp, #8 * 20] + stp x22, x23, [sp, #8 * 22] + stp x24, x25, [sp, #8 * 24] + stp x26, x27, [sp, #8 * 26] + stp x28, x29, [sp, #8 * 28] + + ldr x0, [x29, #8] + mcount_adjust_addr x0, x0 + str x0, [sp, #S_LR] + + add x0, sp, #S_FRAME_SIZE + str x0, [sp, #S_SP] + + mcount_adjust_addr x0, x30 + str x0, [sp, #S_PC] + + ldr x0, [sp, #8 * 0] + + mov x29, sp + .endm + + /* for instrumented function */ + .macro mcount_get_lr1 reg + ldr \reg, [x29, #8 * 29] + ldr \reg, [\reg, #8] + mcount_adjust_addr \reg, \reg + .endm + + .macro mcount_restore_rest_regs + ldp x0, x1, [sp, #8 * 0] + ldp x2, x3, [sp, #8 * 2] + ldp x4, x5, [sp, #8 * 4] + ldp x6, x7, [sp, #8 * 6] + ldp x8, x9, [sp, #8 * 8] + ldp x10, x11, [sp, #8 * 10] + ldp x12, x13, [sp, #8 * 12] + ldp x14, x15, [sp, #8 * 14] + ldp x16, x17, [sp, #8 * 16] + ldp x18, x19, [sp, #8 * 18] + ldp x20, x21, [sp, #8 * 20] + ldp x22, x23, [sp, #8 * 22] + ldp x24, x25, [sp, #8 * 24] + ldp x26, x27, [sp, #8 * 26] + ldp x28, x29, [sp, #8 * 28] + ldr x30, [sp, #S_LR] + add sp, sp, #S_FRAME_SIZE + + mcount_enter + .endm +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + #ifndef CONFIG_DYNAMIC_FTRACE /* * void _mcount(unsigned long return_address) @@ -156,6 +246,10 @@ ENTRY(ftrace_caller) mcount_get_pc0 x0 // function's pc mcount_get_lr x1 // function's lr + adrp x2, function_trace_op + add x2, x2, #:lo12:function_trace_op + ldr x2, [x2] // current ftrace_ops + mov x3, xzr // pt_regs (NULL) .global ftrace_call ftrace_call: // tracer(pc, lr); @@ -171,6 +265,36 @@ ftrace_graph_call: // ftrace_graph_caller(); mcount_exit ENDPROC(ftrace_caller) + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +/* + * void ftrace_regs_caller() + */ +ENTRY(ftrace_regs_caller) + mcount_save_regs + + mcount_get_pc0 x0 // function's pc + mcount_get_lr1 x1 // function's lr + adrp x2, function_trace_op + add x2, x2, #:lo12:function_trace_op + ldr x2, [x2] // current ftrace_ops + mov x3, sp // pt_regs + + .global ftrace_regs_call +ftrace_regs_call: // tracer(pc, lr, ops, regs); + nop // This will be replaced with "bl xxx" + // where xxx can be any kind of tracer. + + mcount_restore_rest_regs + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + b ftrace_graph_call +#endif + + mcount_exit +ENDPROC(ftrace_regs_caller) +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_REGS */ + #endif /* CONFIG_DYNAMIC_FTRACE */ ENTRY(ftrace_stub) diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index c851be7..5440673 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -56,12 +56,21 @@ int ftrace_update_ftrace_func(ftrace_func_t func) { unsigned long pc; u32 new; + int ret; pc = (unsigned long)&ftrace_call; new = aarch64_insn_gen_branch_imm(pc, (unsigned long)func, AARCH64_INSN_BRANCH_LINK); + ret = ftrace_modify_code(pc, 0, new, false); - return ftrace_modify_code(pc, 0, new, false); + if (!ret) { + pc = (unsigned long)&ftrace_regs_call; + new = aarch64_insn_gen_branch_imm(pc, (unsigned long)func, + AARCH64_INSN_BRANCH_LINK); + ret = ftrace_modify_code(pc, 0, new, false); + } + + return ret; } /* @@ -97,6 +106,19 @@ int __init ftrace_dyn_arch_init(void) { return 0; } + +int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, + unsigned long addr) +{ + unsigned long pc = rec->ip; + u32 old, new; + + old = aarch64_insn_gen_branch_imm(pc, old_addr, + AARCH64_INSN_BRANCH_LINK); + new = aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); + + return ftrace_modify_code(pc, old, new, true); +} #endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER