From patchwork Fri May 22 14:55:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Thompson X-Patchwork-Id: 187155 Delivered-To: patches@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp2579595ilb; Fri, 22 May 2020 07:55:16 -0700 (PDT) X-Received: by 2002:a1c:2843:: with SMTP id o64mr13481908wmo.158.1590159316218; Fri, 22 May 2020 07:55:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590159316; cv=none; d=google.com; s=arc-20160816; b=Ey/RnooWyLHrr2pWm1ulWEXccHQFl2NUafJZaBCFrDu4ERah0Y4roKFS2l+svfX3E/ QwfwY82z4KIluGQun0VgKFrdd/bOPIYpMmjoj3EZnk+k6LbJ4lFVmJ0Kf+MBvz+/IXkH tYWgkd/y/Sb7sunynxXm5POn7YuvltYSsbfN2t6WkAcld5YAcHij6LWRmqbbLw+zMmFZ xDKrIQDpwe36DNOp46K5ZMzV2LkYcFza0No/G44Rx926TuXUA1ne5Kg5PSk8c5q8HqNL kvvTgf23R7+mP4GA5SAT1imxSBHupq3Bskzf/DD/AOUH/0gfWcSMzFXYNFzmUqGqNQnt V2Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Z2JTvHuqFiAH+e0uHvAv7QUdItZgp1++hBsH29TGAGE=; b=t4xPWGQnwu57CKAYdG4rOt+Wc6Ry6OSNrjaoyROuMbmxU5bMv93xKuLLaYXztv9yw0 6cO7iEkkgX/hOJM1rgVvoONFRg8nb27eBviJIn8pfy/fj6cyBZAf2w/oVqjVNfK50n4f KQzfhb2AO/xw0IQyEHPNq9ISAkjJiMOoxs7KTG7+PcmhRDATYW7MKniWHlFAeStDIXpk jbG1CIJiQDo7IuyYMP0g3syMd3d6CXqAuYjQnSmHGLb7pmx7M85cq+xNgAXchZPeWllq 7eKsdtUab5+niW4jw+zvfGJlXoR+NJ0Mxye/K1q4gmORsr3GhIbsXPq9fa3uplCwrHCA 2Sug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RH3hJubU; spf=pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=daniel.thompson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h13sor404884wre.23.2020.05.22.07.55.16 for (Google Transport Security); Fri, 22 May 2020 07:55:16 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RH3hJubU; spf=pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=daniel.thompson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z2JTvHuqFiAH+e0uHvAv7QUdItZgp1++hBsH29TGAGE=; b=RH3hJubUWGtSNW24GhTx6ufaJ6TRraOrbbB6edaZ1SxjVy3F3fER9ye7M9DV2/koq2 WolLZoTDKPlKaRDouJvsUsCVShp5GvsPjxVf/vBlD/kFDBl2QP/5obzIUP+C/DliXKFe vWBbx3oaBCHHqGyG00lPrlsfgZua0ucfU7gneN0OBBm88TXXdm3WRzT32ng9qk+9ZzFh UvzEk05phTl5Vurl2f8gX/swSGoyVuEc8AtTZnZIAfWIo53XvjMUP16u91mtv7nQ5Dsd C/dtaKrFdXg8UuWpnXiR/uEA3LADGd0mTIdfzp1TkIHIRHLFSVWLPTxxG9wU5PP1z82f I9EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z2JTvHuqFiAH+e0uHvAv7QUdItZgp1++hBsH29TGAGE=; b=iPDtHh0tIznFNY7TmtcWb2kEQqTTua9dLucTw9eHwVepI0y9EFwP3qwjastBLGaAkP BizPjjxVMzwDA+jDKXPPYAGuS1JsoLlpjc7N6m2L3lMX9JR4T4DQ9Fi9RRXpn13tivL2 jcY1s6Zx6/keXdwtLKRzaMmsRw4mfflq2j4I3wFZDZDH3Y1WcY42/Yslmu3TXTKTtw/x T2Fw1Gl1Pz+DlkSdajXoHCM/lmApeCAi8ZMPAluwhREAWL639N0T6wer3YMIRkqCSl1/ uAF9XpyaA1LtriN9iXDGgkGPt2A01CtA1zTUMhwv234Rt5LvfK6kNW33OJJ0W7IDbNQ9 FsFA== X-Gm-Message-State: AOAM531KrKDXZO1nFNDDaHknBeUH1hok98ggMLREOwTTPxD+DkwKi1BM eltjMJfFej12HEKmi8yf22EEfPG0 X-Google-Smtp-Source: ABdhPJzVBNmBWWvQ9nOtr1g+uNGIt9/otWbCh5kkWpEi6Kn9nuax/sD2PRXbA3VTqgPQqIPvlzBrKA== X-Received: by 2002:adf:eb08:: with SMTP id s8mr3721175wrn.361.1590159315732; Fri, 22 May 2020 07:55:15 -0700 (PDT) Return-Path: Received: from wychelm.lan (cpc141214-aztw34-2-0-cust773.18-1.cable.virginm.net. [86.9.19.6]) by smtp.gmail.com with ESMTPSA id i21sm9746911wml.5.2020.05.22.07.55.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 May 2020 07:55:15 -0700 (PDT) From: Daniel Thompson To: sumit.garg@linaro.org, jason.wessel@windriver.com, dianders@chromium.org Cc: Daniel Thompson , kgdb-bugreport@lists.sourceforge.net, linux-kernel@vger.kernel.org, patches@linaro.org, pmladek@suse.com, sergey.senozhatsky@gmail.com, Peter Zijlstra , Ingo Molnar , Will Deacon Subject: [RFC PATCH 1/2] debug: Convert dbg_slave_lock to an atomic Date: Fri, 22 May 2020 15:55:09 +0100 Message-Id: <20200522145510.2109799-2-daniel.thompson@linaro.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200522145510.2109799-1-daniel.thompson@linaro.org> References: <20200522145510.2109799-1-daniel.thompson@linaro.org> MIME-Version: 1.0 Currently the debug core takes and releases dbg_slave_lock, knowing it to be uncontended, as a means to set and clear a flag that it uses to herd the other cores in to or out of a holding pen. Let's convert this to a regular atomic instead. This change is worthwhile simply for the subtle increase in clarity in a very tangled bit of code. It is also useful because it removes a raw_spin_lock() from within the debug trap, which will make it easier to introduce spin lock debugging code for the debug trap handler. Signed-off-by: Daniel Thompson --- kernel/debug/debug_core.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) -- 2.25.4 diff --git a/kernel/debug/debug_core.c b/kernel/debug/debug_core.c index 2b7c9b67931d..8f43171ddeac 100644 --- a/kernel/debug/debug_core.c +++ b/kernel/debug/debug_core.c @@ -121,12 +121,12 @@ static struct kgdb_bkpt kgdb_break[KGDB_MAX_BREAKPOINTS] = { atomic_t kgdb_active = ATOMIC_INIT(-1); EXPORT_SYMBOL_GPL(kgdb_active); static DEFINE_RAW_SPINLOCK(dbg_master_lock); -static DEFINE_RAW_SPINLOCK(dbg_slave_lock); /* * We use NR_CPUs not PERCPU, in case kgdb is used to debug early * bootup code (which might not have percpu set up yet): */ +static atomic_t slaves_must_spin; static atomic_t masters_in_kgdb; static atomic_t slaves_in_kgdb; static atomic_t kgdb_break_tasklet_var; @@ -615,7 +615,7 @@ static int kgdb_cpu_enter(struct kgdb_state *ks, struct pt_regs *regs, dump_stack(); kgdb_info[cpu].exception_state &= ~DCPU_WANT_BT; } else if (kgdb_info[cpu].exception_state & DCPU_IS_SLAVE) { - if (!raw_spin_is_locked(&dbg_slave_lock)) + if (!atomic_read(&slaves_must_spin)) goto return_normal; } else { return_normal: @@ -677,7 +677,7 @@ static int kgdb_cpu_enter(struct kgdb_state *ks, struct pt_regs *regs, * CPU in a spin state while the debugger is active */ if (!kgdb_single_step) - raw_spin_lock(&dbg_slave_lock); + atomic_set(&slaves_must_spin, 1); #ifdef CONFIG_SMP /* If send_ready set, slaves are already waiting */ @@ -741,7 +741,7 @@ static int kgdb_cpu_enter(struct kgdb_state *ks, struct pt_regs *regs, dbg_io_ops->post_exception(); if (!kgdb_single_step) { - raw_spin_unlock(&dbg_slave_lock); + atomic_set(&slaves_must_spin, 0); /* Wait till all the CPUs have quit from the debugger. */ while (kgdb_do_roundup && atomic_read(&slaves_in_kgdb)) cpu_relax(); From patchwork Fri May 22 14:55:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Thompson X-Patchwork-Id: 187156 Delivered-To: patches@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp2579611ilb; Fri, 22 May 2020 07:55:17 -0700 (PDT) X-Received: by 2002:a05:6000:10d2:: with SMTP id b18mr3667971wrx.366.1590159317339; Fri, 22 May 2020 07:55:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590159317; cv=none; d=google.com; s=arc-20160816; b=LeQ18Yux0rLQMjO44m69cFJFu3DnDHAYqBvXYuEcWkLkcig6x3qJ+UptjkAD/u2e24 B7QPjlZGANCq+kyVo080p7TLLTT7WNU4zK+jgQMOlm/PwMM8UzHE5OZ2kiOUjENI5d+o Uobgnc0I/jyfTbPDIedgInFvLLOltN0BcTtrKpZMV5wZaflaiG12/+j2uE+t5HGtnB7I nfPwVlDWmYM68y6+NOsPbDv+ts8xjmaZahXNUoSm6TPw8+wSbmNhtnqNyqa570+WysG/ QOendOyjzYSGHC/pP9aM2QJF2OirAO18MEtZDwnDgXfsCc1Hr5gSpgzd/TfRIwlNNGB3 O0Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=kkTz4sTk7s012DbHMp3X0LwsnQSwOjg9lEMirTJ/PWk=; b=fESwp+Qk2oiW83WF92AJDmOB23k6bglHfLEE0RemkEiVoDlO+MRqmic8YA9MKtZmDl 9gtVRJMa5okf3ntqfEOkC/v9dCMTQf362Sbytf36M2uImqKKNa3UWf+c+8KsrafIO0nq 8O3++Kn+E59xPvqzfKn0lSDolKdDqLTtJgTWo57bdwzvVG6hjPRJvex+5c9DnqR1swTi BnX1FjLLmbhTYUNUud5Xd7fTmdBfAHE19UmNM4Vpd16gtykCPJTMq9Q8EEiJrhCQWKb/ Yngyr79J21SDxnGNJeLZAtSnLIeffL0o0FZYBJFaqRWYZyCoEDgDGoNIpEZ3GgS/xRBg bfQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CmyoV4Da; spf=pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=daniel.thompson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c83sor5859577wmf.11.2020.05.22.07.55.17 for (Google Transport Security); Fri, 22 May 2020 07:55:17 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CmyoV4Da; spf=pass (google.com: domain of daniel.thompson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=daniel.thompson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkTz4sTk7s012DbHMp3X0LwsnQSwOjg9lEMirTJ/PWk=; b=CmyoV4DakSUdEOfu7cjPJzdTIPnJ6TKlRfsXiilcn49CqPDcUOzFGX9DBib2tD3/pm QApEtLdvp6Fa4KNTp/oT0fEOwbQWtU5CZ3x2SCCeUD/WspsDejdt5xibpZ3dKW+8N9vd wpMgNvOZId9/GA5XVtZF4Lq5us4bbMV8sDPJ0/XlIQLKxCHSnelKR8n1Ie62Om5LxWUl EQYUU91KChW8wkfIxjdDK6WH+98E07MH6kLCa43YRIb/FcPg4qpXBzBKU46RZRSX/jmR 1g6SynQVCCRNNQZhm8gmkXSfMar3NbaV1LuIrwazVbgceNf5GLdXyLQ2URJ4NKjQlMdB t4LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkTz4sTk7s012DbHMp3X0LwsnQSwOjg9lEMirTJ/PWk=; b=KW8YPRnTwbAxgvxYYmXQ/YT3h4dZqlcZSUlOsXwL++iTwKpRyD9gHoNLLO6oTzGdxx GosnpEG3YeSD+AGKntH9jyiAMDxYufeDEz4cVSBTwmlFRY3F4NqpinQ6XRlcBgJsGz5L RJUgbwttic86tcWrcAPfPGxL6X3nexiHCcJOoqbBHKXvFhmKa9NMFR7ER3szqXw3IFNw bOd5sS71FAnXYgWutWUO5f/bmVLPqyVdICutNd3FSqUb8s2sJMHyj8ox5gMider2rO1r OFMQedLpHCEXOnntdK+E3Z9Yuuw8Lgg5SZ9BvwmV3tzuk2FKAUSMEubAte9plPYq3EW4 i+4w== X-Gm-Message-State: AOAM532zXwgJrGhojd0wp58ZrcRtYvQOyyKtUG5oifS/WhDZGT14EVCY LKhHmA83UZ+S8+9weuTS2W0Y9zao X-Google-Smtp-Source: ABdhPJxQXqSFq49qzrvg5dJHfbQcmZrcW5ufJyiYOXZjrM7OJFBvSL22Ako9KERokDxGdIZxZR7/nA== X-Received: by 2002:a1c:25c3:: with SMTP id l186mr13757890wml.103.1590159316931; Fri, 22 May 2020 07:55:16 -0700 (PDT) Return-Path: Received: from wychelm.lan (cpc141214-aztw34-2-0-cust773.18-1.cable.virginm.net. [86.9.19.6]) by smtp.gmail.com with ESMTPSA id i21sm9746911wml.5.2020.05.22.07.55.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 May 2020 07:55:16 -0700 (PDT) From: Daniel Thompson To: sumit.garg@linaro.org, jason.wessel@windriver.com, dianders@chromium.org Cc: Daniel Thompson , kgdb-bugreport@lists.sourceforge.net, linux-kernel@vger.kernel.org, patches@linaro.org, pmladek@suse.com, sergey.senozhatsky@gmail.com, Peter Zijlstra , Ingo Molnar , Will Deacon Subject: [RFC PATCH 2/2] locking/spinlock/debug: Add checks for kgdb trap safety Date: Fri, 22 May 2020 15:55:10 +0100 Message-Id: <20200522145510.2109799-3-daniel.thompson@linaro.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200522145510.2109799-1-daniel.thompson@linaro.org> References: <20200522145510.2109799-1-daniel.thompson@linaro.org> MIME-Version: 1.0 In general it is not safe to call spin_lock() whilst executing in the kgdb trap handler. The trap can be entered from all sorts of execution context (NMI, IRQ, irqs disabled, etc) and the kgdb/kdb needs to be as resillient as possible. Currently it is difficult to spot mistakes in the kgdb/kdb logic (especially so for kdb because it uses more kernel features than pure-kgdb). Let's provide a means to bring attention to deadlock risks in the debug code. Signed-off-by: Daniel Thompson --- include/linux/kgdb.h | 16 ++++++++++++++++ kernel/locking/spinlock_debug.c | 4 ++++ lib/Kconfig.kgdb | 11 +++++++++++ 3 files changed, 31 insertions(+) -- 2.25.4 diff --git a/include/linux/kgdb.h b/include/linux/kgdb.h index b072aeb1fd78..de30ce8078cf 100644 --- a/include/linux/kgdb.h +++ b/include/linux/kgdb.h @@ -332,4 +332,20 @@ extern void kgdb_panic(const char *msg); #define dbg_late_init() static inline void kgdb_panic(const char *msg) {} #endif /* ! CONFIG_KGDB */ + +#ifdef CONFIG_KGDB_DEBUG_SPINLOCK +/** + * check_kgdb_context_before() - Check whether to issue a spinlock warning + * + * Currently this only reports when the master processor violates the + * locking rules (because we are using the in_dbg_master() macro since + * we are confident that will avoid false positives). + * + * Return: True if we are executing in the debug trap + */ +static inline int check_kgdb_context_before(void) { return in_dbg_master(); } +#else +static inline int check_kgdb_context_before(void) { return 0; } +#endif + #endif /* _KGDB_H_ */ diff --git a/kernel/locking/spinlock_debug.c b/kernel/locking/spinlock_debug.c index b9d93087ee66..b49789e0fed8 100644 --- a/kernel/locking/spinlock_debug.c +++ b/kernel/locking/spinlock_debug.c @@ -12,6 +12,7 @@ #include #include #include +#include void __raw_spin_lock_init(raw_spinlock_t *lock, const char *name, struct lock_class_key *key, short inner) @@ -84,6 +85,7 @@ debug_spin_lock_before(raw_spinlock_t *lock) SPIN_BUG_ON(READ_ONCE(lock->owner) == current, lock, "recursion"); SPIN_BUG_ON(READ_ONCE(lock->owner_cpu) == raw_smp_processor_id(), lock, "cpu recursion"); + SPIN_BUG_ON(check_kgdb_context_before(), lock, "in debug trap"); } static inline void debug_spin_lock_after(raw_spinlock_t *lock) @@ -174,6 +176,7 @@ int do_raw_read_trylock(rwlock_t *lock) void do_raw_read_unlock(rwlock_t *lock) { RWLOCK_BUG_ON(lock->magic != RWLOCK_MAGIC, lock, "bad magic"); + RWLOCK_BUG_ON(check_kgdb_context_before(), lock, "in debug trap"); arch_read_unlock(&lock->raw_lock); } @@ -183,6 +186,7 @@ static inline void debug_write_lock_before(rwlock_t *lock) RWLOCK_BUG_ON(lock->owner == current, lock, "recursion"); RWLOCK_BUG_ON(lock->owner_cpu == raw_smp_processor_id(), lock, "cpu recursion"); + RWLOCK_BUG_ON(check_kgdb_context_before(), lock, "in debug trap"); } static inline void debug_write_lock_after(rwlock_t *lock) diff --git a/lib/Kconfig.kgdb b/lib/Kconfig.kgdb index 933680b59e2d..4d57900d6c53 100644 --- a/lib/Kconfig.kgdb +++ b/lib/Kconfig.kgdb @@ -29,6 +29,17 @@ config KGDB_SERIAL_CONSOLE Share a serial console with kgdb. Sysrq-g must be used to break in initially. +config KGDB_DEBUG_SPINLOCK + bool "KGDB: Check for spin lock usage when system is halted" + select DEBUG_SPINLOCK + default n + help + Say Y here to catch spin lock waiting when we are running + in the kgdb trap handler and report it. When the trap handler + is executing all other system activity is halted and spin lock + contention will lead to deadlock. This makes any spin lock wait + from this execution context risky. + config KGDB_TESTS bool "KGDB: internal test suite" default n