From patchwork Tue Apr 11 09:10:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Yan X-Patchwork-Id: 97224 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp1716758qgd; Tue, 11 Apr 2017 02:10:50 -0700 (PDT) X-Received: by 10.99.97.6 with SMTP id v6mr60878691pgb.186.1491901850200; Tue, 11 Apr 2017 02:10:50 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t61si16238310plb.258.2017.04.11.02.10.49; Tue, 11 Apr 2017 02:10:50 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754257AbdDKJKp (ORCPT + 23 others); Tue, 11 Apr 2017 05:10:45 -0400 Received: from mail-pg0-f47.google.com ([74.125.83.47]:34522 "EHLO mail-pg0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754177AbdDKJKm (ORCPT ); Tue, 11 Apr 2017 05:10:42 -0400 Received: by mail-pg0-f47.google.com with SMTP id 21so119625162pgg.1 for ; Tue, 11 Apr 2017 02:10:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=HaVbObFMYCXddorJUJ5Izss9h3tUZCa0eWZYQ4o5nGw=; b=Qtz273lm6/RvkG4C0sFFLLFCf1uRaxAik4LhwmxZeBIIunyLsgZpVXlou15CMYgEI1 3/HSMbY/suKAqtxIHxLJYozBQdg3/6dCG4+4o81VJjUIl50S35L83KCWgxiSkWYSWyG+ E5KBAkvBWnaDoJJrEXxW2zLCHE8aVw3vo3HEM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=HaVbObFMYCXddorJUJ5Izss9h3tUZCa0eWZYQ4o5nGw=; b=SIqq5CupV2//7z8IxWxAipSSDU9UcZ7qTaEll3aFq0c1Y9n+Yx+0iRxjbdVezleWmf kCB1xtdbAzZIZorUnQauiPV8RrwdKrHaytJbWFtalhuZQ7+YO/QHtYTo+NZAB7AuG5By +hdcWQWm0LchIP+m5N9cs44deoBqLUc9ooKXMo8cRL4sThE3gQm2bxWqP7veLJqTCzWk ra37NP2OQzJzkbiBmhh6YGzzoHxesVsovhEmryHtVNmXn+eC4LiVHS0ldjhKIqUSlRBp KohWw5GJEkSHfNw/xp3ZJYE0QHLskOkcQVZALeY540p7kNrO5fkYonFRWR8rqGkO0Wd/ hUTw== X-Gm-Message-State: AFeK/H2LOpoy4T8+nWl+jVettlTuJxFE6/8b0DyPkMJfaZc4HsY/5gz6YC4jroOLrXaXClSU X-Received: by 10.98.2.195 with SMTP id 186mr47191122pfc.9.1491901841711; Tue, 11 Apr 2017 02:10:41 -0700 (PDT) Received: from localhost.localdomain (li1563-109.members.linode.com. [139.162.83.109]) by smtp.gmail.com with ESMTPSA id r77sm8196394pfe.105.2017.04.11.02.10.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 11 Apr 2017 02:10:40 -0700 (PDT) From: Leo Yan To: Mathieu Poirier , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Leo Yan Subject: [PATCH RFC 0/4] coresight: support dump ETB RAM Date: Tue, 11 Apr 2017 17:10:25 +0800 Message-Id: <1491901829-18477-1-git-send-email-leo.yan@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ### Introduction ### Embedded Trace Buffer (ETB) provides on-chip storage of trace data, usually has buffer size from 2KB to 8KB. These data has been used for profiling and this has been well implemented in coresight driver. This patch is to explore ETB RAM data for postmortem debugging. Due ETB RAM buffer has small size, so the real trace data caused error is easily to be overwritten by other PEs; but we could consider ETB RAM data is quite useful for postmortem debugging with below scenarios: Case 1: if system is bus lockup and CPU pipeline stalls for bus accessing, CPUs have no more chance to fill enough data into ETB RAM so after analyze ETB RAM we can quickly get to know the culprit if bus lock is caused by improper programs, one often example is wrongly to access the module without enable the module's clock. For this case, we can rely on watchdog to trigger SoC reset and if lucky the ETB RAM can survive after reset. So for this case, after system reboot we can save ETB RAM before any new data input into it. Case 2: There also has another hardware design with local ETB buffer (ARM DDI 0461B) chapter 1.2.7. Local ETF, with this kind design every CPU may has one dedicated ETB RAM. So it's quite handy that we can use alive CPU to help dump the hang CPU ETB RAM. Then we can quickly get to know what's the last point the CPU has executed before its hang. ### Implementation ### Based on current Coresight ETB driver, we only needs some minor enhancement so can support dump ETB RAM with two methods. Patches 0001/0002 are minor fixes so can support more scenarios for ETB RAM dumping. Patch 0003 is to dump ETB RAM after system reboot, this is for the platforms which use watchdog reset and ETB RAM can survive. Patch 0004 is to dump ETB RAM when panic happens, so we can save ETB RAM into memory. If we connect this with Kdump, then we can easily extract the ETB RAM from vmcore. ### Usage ### To dump ETB RAM after reboot, simply use below command: # dd if=/dev/f6402000.etf of=cstrace.bin To dump ETB RAM for kernel panic, we need add "crash_kexec_post_notifiers" into kernel command line so let kernel call panic notifiers before launch dump kernel. After dump kernel has booted up, we need use below methods to ETB RAM offline analysis: On the target: # cp /proc/vmcore ./vmcore # scp ./vmcore your@hostpc On the host PC: # ./crash vmcore vmlinux crash> log [...] [ 112.600051] coresight-tmc f6402000.etf: Flush ETB buffer 0x2000@0xffff800038300080 [ 112.614743] Starting crashdump kernel... [ 112.618681] Bye! crash> rd 0xffff800038300080 0x2000 -r /tmp/cstrace.bin 8192 bytes copied from 0xffff800038300080 to /tmp/cstrace.bin After we get cstrace.bin data, we can use OpenCSD snapshot method to parse ETB trace data. These two methods have been verified on Hikey, For Hikey snapshot config files you can refer [1]. For total kernel patches for integration Kdump and Coresight, you can refer [2]. [1] http://people.linaro.org/~leo.yan/opencsd_hikey/hikey_snapshot.tgz [2] https://git.linaro.org/people/leo.yan/linux-debug-workshop.git/log/?h=coresight_etb_dump ### TODO ### Need work for ETB1.0 driver, this is based on review and comments for this patch set. Leo Yan (4): coresight: tmc: check dump buffer is overflow coresight: tmc: set read pointer before dump RAM coresight: tmc: dump RAM when device is disabled coresight: tmc: dump RAM for panic drivers/hwtracing/coresight/coresight-tmc-etf.c | 86 ++++++++++++++++++++++++- drivers/hwtracing/coresight/coresight-tmc.h | 2 + 2 files changed, 85 insertions(+), 3 deletions(-) -- 2.7.4