From patchwork Tue Apr 21 01:13:13 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: AKASHI Takahiro X-Patchwork-Id: 47384 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f199.google.com (mail-wi0-f199.google.com [209.85.212.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 3147320553 for ; Tue, 21 Apr 2015 01:14:00 +0000 (UTC) Received: by wixv7 with SMTP id v7sf800872wix.0 for ; Mon, 20 Apr 2015 18:13:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=K3MJVzPp2yWGFPzytr9wZujUpHovzi8DkfURhncPT4E=; b=IpvIwlnTk2SKi8IdZPFB63J1PPD3ALCjQ7IGzsaq0MYHYW2/LVc04gyeZUabqxhn5r SBVf+p2zseIfxj4AAYaEmDs2KwI9VQYgKywbd+ISET0j4vCowDVsVSpR8UAkMdO+HKET D3r0JkG20axmYNqTQHQb765yvwTr2Td+Xc6Q4Tuk363E56h1h/FqgcN1baQuSceShmzt c3JnzTY6Kf/HfGPEEDvPN5PDih7CKkOukxMzIBmcATx40YF08EHhPCVMEsqsosxXhkAp e8yRp9v5nkW1Vz2UwmIGHNIqLhiXrhPA38MIwLblFCj9nUSqI1H9aw5LJ05H8VHjPmk8 si8g== X-Gm-Message-State: ALoCoQmAkQ4IowmODip1wZ/cduPpaDVlsATFZh0MrZD9Jnc7cZ4heraWr78EmsApweOElECtcoti X-Received: by 10.180.106.136 with SMTP id gu8mr7093334wib.6.1429578839417; Mon, 20 Apr 2015 18:13:59 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.29.36 with SMTP id g4ls916402lah.33.gmail; Mon, 20 Apr 2015 18:13:59 -0700 (PDT) X-Received: by 10.152.178.197 with SMTP id da5mr18340423lac.56.1429578839216; Mon, 20 Apr 2015 18:13:59 -0700 (PDT) Received: from mail-la0-f44.google.com (mail-la0-f44.google.com. [209.85.215.44]) by mx.google.com with ESMTPS id q8si206320lal.71.2015.04.20.18.13.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Apr 2015 18:13:58 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.44 as permitted sender) client-ip=209.85.215.44; Received: by lagv1 with SMTP id v1so139991016lag.3 for ; Mon, 20 Apr 2015 18:13:58 -0700 (PDT) X-Received: by 10.112.204.6 with SMTP id ku6mr17706230lbc.73.1429578838843; Mon, 20 Apr 2015 18:13:58 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.67.65 with SMTP id l1csp336856lbt; Mon, 20 Apr 2015 18:13:57 -0700 (PDT) X-Received: by 10.66.119.71 with SMTP id ks7mr32055333pab.147.1429578836873; Mon, 20 Apr 2015 18:13:56 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id nt1si462289pdb.23.2015.04.20.18.13.55; Mon, 20 Apr 2015 18:13:56 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752534AbbDUBNv (ORCPT + 26 others); Mon, 20 Apr 2015 21:13:51 -0400 Received: from mail-pa0-f47.google.com ([209.85.220.47]:33522 "EHLO mail-pa0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751544AbbDUBNu (ORCPT ); Mon, 20 Apr 2015 21:13:50 -0400 Received: by paboj16 with SMTP id oj16so224236570pab.0 for ; Mon, 20 Apr 2015 18:13:49 -0700 (PDT) X-Received: by 10.66.124.227 with SMTP id ml3mr32993302pab.28.1429578829435; Mon, 20 Apr 2015 18:13:49 -0700 (PDT) Received: from localhost.localdomain (61-205-3-184m5.grp1.mineo.jp. [61.205.3.184]) by mx.google.com with ESMTPSA id pv2sm199181pbb.12.2015.04.20.18.13.44 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 20 Apr 2015 18:13:47 -0700 (PDT) From: AKASHI Takahiro To: jason.wessel@windriver.com, catalin.marinas@arm.com, will.deacon@arm.com Cc: broonie@kernel.org, yong.zhao@amd.com, Vijaya.Kumar@caviumnetworks.com, kgdb-bugreport@lists.sourceforge.net, linux-arm-kernel@lists.infradead.org, linaro-kernel@lists.linaro.org, linux-kernel@vger.kernel.org, AKASHI Takahiro Subject: [RESEND PATCH] arm64: kgdb: fix single stepping Date: Tue, 21 Apr 2015 10:13:13 +0900 Message-Id: <1429578793-3971-1-git-send-email-takahiro.akashi@linaro.org> X-Mailer: git-send-email 1.9.1 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: takahiro.akashi@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.44 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Jason, Could you please review my patch below? See also arm64 maintainer's comment: http://lists.infradead.org/pipermail/linux-arm-kernel/2015-January/313712.html Thanks, -Takahiro AKASHI I tried to verify kgdb in vanilla kernel on fast model, but it seems that the single stepping with kgdb doesn't work correctly since its first appearance at v3.15. On v3.15, 'stepi' command after breaking the kernel at some breakpoint steps forward to the next instruction, but the succeeding 'stepi' never goes beyond that. On v3.16, 'stepi' moves forward and stops at the next instruction just after enable_dbg in el1_dbg, and never goes beyond that. This variance of behavior seems to come in with the following patch in v3.16: commit 2a2830703a23 ("arm64: debug: avoid accessing mdscr_el1 on fault paths where possible") This patch (1) moves kgdb_disable_single_step() from 'c' command handling to single step handler. This makes sure that single stepping gets effective at every 's' command. Please note that, under the current implementation, single step bit in spsr, which is cleared by the first single stepping, will not be set again for the consecutive 's' commands because single step bit in mdscr is still kept on (that is, kernel_active_single_step() in kgdb_arch_handle_exception() is true). (2) re-implements kgdb_roundup_cpus() because the current implementation enabled interrupts naively. See below. (3) removes 'enable_dbg' in el1_dbg. Single step bit in mdscr is turned on in do_handle_exception()-> kgdb_handle_expection() before returning to debugged context, and if debug exception is enabled in el1_dbg, we will see unexpected single- stepping in el1_dbg. Since v3.18, the following patch does the same: commit 1059c6bf8534 ("arm64: debug: don't re-enable debug exceptions on return from el1_dbg) (4) masks interrupts while single-stepping one instruction. If an interrupt is caught during processing a single-stepping, debug exception is unintentionally enabled by el1_irq's 'enable_dbg' before returning to debugged context. Thus, like in (2), we will see unexpected single-stepping in el1_irq. Basically (1) and (2) are for v3.15, (3) and (4) for v3.1[67]. * issue fixed by (2): Without (2), we would see another problem if a breakpoint is set at interrupt-sensible places, like gic_handle_irq(): KGDB: re-enter error: breakpoint removed ffffffc000081258 ------------[ cut here ]------------ WARNING: CPU: 0 PID: 650 at kernel/debug/debug_core.c:435 kgdb_handle_exception+0x1dc/0x1f4() Modules linked in: CPU: 0 PID: 650 Comm: sh Not tainted 3.17.0-rc2+ #177 Call trace: [] dump_backtrace+0x0/0x130 [] show_stack+0x10/0x1c [] dump_stack+0x74/0xb8 [] warn_slowpath_common+0x8c/0xb4 [] warn_slowpath_null+0x14/0x20 [] kgdb_handle_exception+0x1d8/0x1f4 [] kgdb_brk_fn+0x18/0x28 [] brk_handler+0x9c/0xe8 [] do_debug_exception+0x3c/0xac Exception stack(0xffffffc07e027650 to 0xffffffc07e027770) ... [] el1_dbg+0x14/0x68 [] kgdb_cpu_enter+0x464/0x5c0 [] kgdb_handle_exception+0x190/0x1f4 [] kgdb_brk_fn+0x18/0x28 [] brk_handler+0x9c/0xe8 [] do_debug_exception+0x3c/0xac Exception stack(0xffffffc07e027ac0 to 0xffffffc07e027be0) ... [] el1_dbg+0x14/0x68 [] __handle_sysrq+0x11c/0x190 [] write_sysrq_trigger+0x4c/0x60 [] proc_reg_write+0x54/0x84 [] vfs_write+0x98/0x1c8 [] SyS_write+0x40/0xa0 Once some interrupt occurs, a breakpoint at gic_handle_irq() triggers kgdb. Kgdb then calls kgdb_roundup_cpus() to sync with other cpus. Current kgdb_roundup_cpus() unmasks interrupts temporarily to use smp_call_function(). This eventually allows another interrupt to occur and likely results in hitting a breakpoint at gic_handle_irq() again since debug exception is always enabled in el1_irq. We can avoid this issue by specifying "nokgdbroundup" in kernel parameter, but this will also leave other cpus be in unknown state in terms of kgdb, and may result in interfering with kgdb activity. Signed-off-by: AKASHI Takahiro --- arch/arm64/kernel/kgdb.c | 60 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 46 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c index a0d10c5..81b5910 100644 --- a/arch/arm64/kernel/kgdb.c +++ b/arch/arm64/kernel/kgdb.c @@ -19,9 +19,13 @@ * along with this program. If not, see . */ +#include #include +#include #include #include +#include +#include #include struct dbg_reg_def_t dbg_reg_def[DBG_MAX_REG_NUM] = { @@ -95,6 +99,9 @@ struct dbg_reg_def_t dbg_reg_def[DBG_MAX_REG_NUM] = { { "fpcr", 4, -1 }, }; +static DEFINE_PER_CPU(unsigned int, kgdb_pstate); +static DEFINE_PER_CPU(struct irq_work, kgdb_irq_work); + char *dbg_get_reg(int regno, void *mem, struct pt_regs *regs) { if (regno >= DBG_MAX_REG_NUM || regno < 0) @@ -176,18 +183,14 @@ int kgdb_arch_handle_exception(int exception_vector, int signo, * over and over again. */ kgdb_arch_update_addr(linux_regs, remcom_in_buffer); - atomic_set(&kgdb_cpu_doing_single_step, -1); - kgdb_single_step = 0; - - /* - * Received continue command, disable single step - */ - if (kernel_active_single_step()) - kernel_disable_single_step(); err = 0; break; case 's': + /* mask interrupts while single stepping */ + __this_cpu_write(kgdb_pstate, linux_regs->pstate); + linux_regs->pstate |= PSR_I_BIT; + /* * Update step address value with address passed * with step packet. @@ -198,8 +201,6 @@ int kgdb_arch_handle_exception(int exception_vector, int signo, */ kgdb_arch_update_addr(linux_regs, remcom_in_buffer); atomic_set(&kgdb_cpu_doing_single_step, raw_smp_processor_id()); - kgdb_single_step = 1; - /* * Enable single step handling */ @@ -229,6 +230,18 @@ static int kgdb_compiled_brk_fn(struct pt_regs *regs, unsigned int esr) static int kgdb_step_brk_fn(struct pt_regs *regs, unsigned int esr) { + unsigned int pstate; + + kernel_disable_single_step(); + atomic_set(&kgdb_cpu_doing_single_step, -1); + + /* restore interrupt mask status */ + pstate = __this_cpu_read(kgdb_pstate); + if (pstate & PSR_I_BIT) + regs->pstate |= PSR_I_BIT; + else + regs->pstate &= ~PSR_I_BIT; + kgdb_handle_exception(1, SIGTRAP, 0, regs); return 0; } @@ -249,16 +262,27 @@ static struct step_hook kgdb_step_hook = { .fn = kgdb_step_brk_fn }; -static void kgdb_call_nmi_hook(void *ignored) +static void kgdb_roundup_hook(struct irq_work *work) { kgdb_nmicallback(raw_smp_processor_id(), get_irq_regs()); } void kgdb_roundup_cpus(unsigned long flags) { - local_irq_enable(); - smp_call_function(kgdb_call_nmi_hook, NULL, 0); - local_irq_disable(); + int cpu; + struct cpumask mask; + struct irq_work *work; + + mask = *cpu_online_mask; + cpumask_clear_cpu(smp_processor_id(), &mask); + cpu = cpumask_first(&mask); + if (cpu >= nr_cpu_ids) + return; + + for_each_cpu(cpu, &mask) { + work = per_cpu_ptr(&kgdb_irq_work, cpu); + irq_work_queue_on(work, cpu); + } } static int __kgdb_notify(struct die_args *args, unsigned long cmd) @@ -299,6 +323,8 @@ static struct notifier_block kgdb_notifier = { int kgdb_arch_init(void) { int ret = register_die_notifier(&kgdb_notifier); + int cpu; + struct irq_work *work; if (ret != 0) return ret; @@ -306,6 +332,12 @@ int kgdb_arch_init(void) register_break_hook(&kgdb_brkpt_hook); register_break_hook(&kgdb_compiled_brkpt_hook); register_step_hook(&kgdb_step_hook); + + for_each_possible_cpu(cpu) { + work = per_cpu_ptr(&kgdb_irq_work, cpu); + init_irq_work(work, kgdb_roundup_hook); + } + return 0; }