From patchwork Tue May 13 20:37:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 889802 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E32C1A3A8D; Tue, 13 May 2025 20:37:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168679; cv=none; b=tBWIaImVTpr1wlrCktyrUtfaKXpGZxjRekjR24gEy6dsEJh9AiV2AeQ/Qjb7MU3G9Yy8fFdud2s81cl4B6CEHK6E2/79eRTzTZjd9WO0tVvrHfadrZ81mFC1fP87V91umSzd+XvBu604Rlm7/094YY2DCtmP0AdqaVzyQPYgXmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168679; c=relaxed/simple; bh=hyHqRWQkgZHimpVOiM5Vr4ymbwLjjHj3vyxzS3WaAW8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mexyEP6OzDEov+HgZ3wWIUEQoWm/Kp/GefiR7GGMKvyT2FWVU7KNai7s1nS6l2OHqJtmiBdKKgafpaE2awtzMBPirqaQoUlbg0TnN48GCWjPvvhKCih6xVe8zzYcQ9csPCUL3Z88dwrdBDr+6XbWIqeLtzwzh2OFQrgg7HclzyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IMzBGrsm; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IMzBGrsm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1747168678; x=1778704678; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hyHqRWQkgZHimpVOiM5Vr4ymbwLjjHj3vyxzS3WaAW8=; b=IMzBGrsmUbyh3LZjHw5+6kasJf94KGAF0odU2T35e+oa/iEyQj8uEdst fjL/StFctUBF/HiTnH+YEslKoYQDyBhMbL2jUHjNfaOKOk2C4jqIUfLsE tgTrRSZenKwEVfG9aEfRyIyWXiH0qxjh3VaUQmQqaNxeOTBZryMiyJ0jC WDwYyujh2Vfv6DHu3z36fH2VJ2XMzsa+8hAdHLs5crId00JOxYl2SKlv2 tyac11XutO49DmxZI7XdMcJeAyU/NDbtqkOfAkvfeJIz8wN8AUJXIxUcB XIeK/yKZb5LzkgElA9nvEskZ6TDxquRGNQX7gExZs1IbpxZytD7uBOqfy w==; X-CSE-ConnectionGUID: Wa1Coh/cTGiPqoayBXJSMg== X-CSE-MsgGUID: mDH0WrUqRiKRDEvIl3zFsQ== X-IronPort-AV: E=McAfee;i="6700,10204,11432"; a="49160397" X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="49160397" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2025 13:37:56 -0700 X-CSE-ConnectionGUID: Vhzvh1imRuGZrVTfH1G2PQ== X-CSE-MsgGUID: OduXNNR1QUS8yfqCK9BMdQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="138241714" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by fmviesa008.fm.intel.com with ESMTP; 13 May 2025 13:37:55 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v6 1/9] x86/fred, KVM: VMX: Pass event data to the FRED entry point from KVM Date: Tue, 13 May 2025 13:37:55 -0700 Message-ID: <20250513203803.2636561-2-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513203803.2636561-1-sohil.mehta@intel.com> References: <20250513203803.2636561-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Zeng Guang Extend the FRED entry point from KVM to take an extra argument to allow KVM to invoke the FRED event dispatch framework with event data. The first use of this extended API is to pass the NMI-source bitmap for NMI-induced VM exits. Read the VMCS exit qualification field to get the NMI-source information and store it as event data precisely in the format expected by the FRED event framework. Read the VMCS exit qualification unconditionally since almost all upcoming CPUs are expected to enable FRED and NMI-source together. In the rare case that NMI-source isn't enabled, the extra VMREAD would be harmless since the exit qualification is expected to be zero. Suggested-by: Sean Christopherson Signed-off-by: Zeng Guang Signed-off-by: Sohil Mehta --- v6: No change v5: Read the VMCS exit qualification unconditionally. (Sean) Combine related patches into one. --- arch/x86/entry/entry_64_fred.S | 2 +- arch/x86/include/asm/fred.h | 9 +++++---- arch/x86/kvm/vmx/vmx.c | 5 +++-- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index 29c5c32c16c3..a61256be9703 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -93,7 +93,7 @@ SYM_FUNC_START(asm_fred_entry_from_kvm) * +--------+-----------------+ */ push $0 /* Reserved, must be 0 */ - push $0 /* Event data, 0 for IRQ/NMI */ + push %rsi /* Event data for IRQ/NMI */ push %rdi /* fred_ss handed in by the caller */ push %rbp pushf diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 2a29e5216881..a4de57e578c4 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -64,14 +64,15 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) void asm_fred_entrypoint_user(void); void asm_fred_entrypoint_kernel(void); -void asm_fred_entry_from_kvm(struct fred_ss); +void asm_fred_entry_from_kvm(struct fred_ss ss, unsigned long edata); __visible void fred_entry_from_user(struct pt_regs *regs); __visible void fred_entry_from_kernel(struct pt_regs *regs); __visible void __fred_entry_from_kvm(struct pt_regs *regs); /* Can be called from noinstr code, thus __always_inline */ -static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) +static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector, + unsigned long edata) { struct fred_ss ss = { .ss =__KERNEL_DS, @@ -81,7 +82,7 @@ static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int .lm = 1, }; - asm_fred_entry_from_kvm(ss); + asm_fred_entry_from_kvm(ss, edata); } void cpu_init_fred_exceptions(void); @@ -109,7 +110,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { ret static inline void cpu_init_fred_exceptions(void) { } static inline void cpu_init_fred_rsps(void) { } static inline void fred_complete_exception_setup(void) { } -static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { } +static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector, unsigned long edata) { } static inline void fred_sync_rsp0(unsigned long rsp0) { } static inline void fred_update_rsp0(void) { } #endif /* CONFIG_X86_FRED */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5c5766467a61..1d43d4a2f6b6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7079,7 +7079,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); if (cpu_feature_enabled(X86_FEATURE_FRED)) - fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector); + fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector, 0); else vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector)); kvm_after_interrupt(vcpu); @@ -7393,7 +7393,8 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, is_nmi(vmx_get_intr_info(vcpu))) { kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); if (cpu_feature_enabled(X86_FEATURE_FRED)) - fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); + fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR, + vmx_get_exit_qual(vcpu)); else vmx_do_nmi_irqoff(); kvm_after_interrupt(vcpu); From patchwork Tue May 13 20:37:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 889801 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A89C1F3FD0; Tue, 13 May 2025 20:37:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168681; cv=none; b=m9fsoiOxyKX1Ze3aorC/tiNjnO67PlUdBBmJLglz8M3OCG06EeIKqQWQzlhfZvbogSNlHpcqnPnL/OgBiSyAYqrZ/7ZDCf6hWQNy51s/JuwjRy12vM7KN6rMx9K4n5SgrNFzO3tQL+d8iqZPGSeTHaqNVl+yHe0ftTVV/w1X7II= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168681; c=relaxed/simple; bh=bf4f0FtErss0et6pnp0FJf7srNyY2MRbqPiZKdJRnTI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CxjFI57nxk3U/WS1LksxPjx0IlSeNYytd41L9eTOx0Zmi7mnM+8tU/q2DbABSS1mP78QMeevdsOptYdSnT5M3hrHdAuCK1Rq9N34islWhf3geGblibOtbZogLh2b7n0ih6wKve3gKQSKUA5NI6UG+1wpxhgnUshuhzgfgcGgQRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JE01mFch; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JE01mFch" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1747168679; x=1778704679; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bf4f0FtErss0et6pnp0FJf7srNyY2MRbqPiZKdJRnTI=; b=JE01mFchDDsEb7Jgd72qw/mV2l08ZVhR2Qk78St/57AMXx+1AoyLbbSq EGLT/7jFY2Yx8CUViwOUCmiTR5lRNrr+qQXDYz58UtZNN2C7x/vjxNhLE ZzalUeRDP+miZmbFDmm4v8sg8uC/w0GRKYtmcccnL9K+h1Ttg43s+Tvvf /4smuXYW8Tuivm2jg8jG4zwbJFi5YxWckEtGPDwnWDiCgZuWqgt5+yeMH iflt7c8vZHfM9Jl23Y9UW1Pp357q35G2KAzwYXl9gIEB/TUaLGQUuYt7D zvZe0iStyhEJ3pjCBFhLCf6xBhua4GsooByLk6NHCrADZUPoGVjtE49e+ w==; X-CSE-ConnectionGUID: 7uRMoZjGRL6bBA9ZgS3bVQ== X-CSE-MsgGUID: Os+mriFyQbqJAnPAfqclug== X-IronPort-AV: E=McAfee;i="6700,10204,11432"; a="49160423" X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="49160423" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2025 13:37:57 -0700 X-CSE-ConnectionGUID: Ttt9udV+ROufOEwJY9P0VA== X-CSE-MsgGUID: gc+tCI4lRJuPJsW+uHIbkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="138241721" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by fmviesa008.fm.intel.com with ESMTP; 13 May 2025 13:37:57 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v6 3/9] x86/nmi: Extend the registration interface to include the NMI-source vector Date: Tue, 13 May 2025 13:37:57 -0700 Message-ID: <20250513203803.2636561-4-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513203803.2636561-1-sohil.mehta@intel.com> References: <20250513203803.2636561-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To prepare for NMI-source reporting, add a source vector argument to the NMI handler registration interface. Later, this will be used to register NMI handlers with a unique source vector that can be used to identify the originator of the NMI. For now, just extend the interface and pass zero as the source vector for all handlers. No functional change intended. Originally-by: Jacob Pan Signed-off-by: Sohil Mehta --- v6: No change. v5: Split the patch into two parts. This one only extends the interface. --- arch/x86/events/amd/ibs.c | 2 +- arch/x86/events/core.c | 2 +- arch/x86/include/asm/nmi.h | 5 ++++- arch/x86/kernel/apic/hw_nmi.c | 3 +-- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/mshyperv.c | 3 +-- arch/x86/kernel/kgdb.c | 6 ++---- arch/x86/kernel/nmi_selftest.c | 7 +++---- arch/x86/kernel/smp.c | 4 ++-- arch/x86/platform/uv/uv_nmi.c | 4 ++-- drivers/acpi/apei/ghes.c | 2 +- drivers/char/ipmi/ipmi_watchdog.c | 3 +-- drivers/edac/igen6_edac.c | 3 +-- drivers/watchdog/hpwdt.c | 6 +++--- 14 files changed, 24 insertions(+), 28 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 0252b7ea8bca..45dece8bee84 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1486,7 +1486,7 @@ static __init int perf_event_ibs_init(void) if (ret) goto err_op; - ret = register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ibs"); + ret = register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ibs", 0); if (ret) goto err_nmi; diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 6866cc5acb0b..b84b8be1f075 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2115,7 +2115,7 @@ static int __init init_hw_perf_events(void) x86_pmu.config_mask = X86_RAW_EVENT_MASK; perf_events_lapic_init(); - register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI"); + register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI", 0); unconstrained = (struct event_constraint) __EVENT_CONSTRAINT(0, x86_pmu.cntr_mask64, diff --git a/arch/x86/include/asm/nmi.h b/arch/x86/include/asm/nmi.h index 79d88d12c8fb..f0a577bf7bba 100644 --- a/arch/x86/include/asm/nmi.h +++ b/arch/x86/include/asm/nmi.h @@ -54,6 +54,7 @@ struct nmiaction { u64 max_duration; unsigned long flags; const char *name; + u8 source_vector; }; /** @@ -62,6 +63,7 @@ struct nmiaction { * @fn: The NMI handler * @fg: Flags associated with the NMI handler * @n: Name of the NMI handler + * @src: NMI-source based vector for the NMI handler * @init: Optional __init* attributes for struct nmiaction * * Adds the provided handler to the list of handlers for the specified @@ -75,13 +77,14 @@ struct nmiaction { * * Return: 0 on success, or an error code on failure. */ -#define register_nmi_handler(t, fn, fg, n, init...) \ +#define register_nmi_handler(t, fn, fg, n, src, init...) \ ({ \ static struct nmiaction init fn##_na = { \ .list = LIST_HEAD_INIT(fn##_na.list), \ .handler = (fn), \ .name = (n), \ .flags = (fg), \ + .source_vector = (src), \ }; \ __register_nmi_handler((t), &fn##_na); \ }) diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 45af535c44a0..612b77660d05 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -53,8 +53,7 @@ NOKPROBE_SYMBOL(nmi_cpu_backtrace_handler); static int __init register_nmi_cpu_backtrace_handler(void) { - register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, - 0, "arch_bt"); + register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, 0, "arch_bt", 0); return 0; } early_initcall(register_nmi_cpu_backtrace_handler); diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c index 06e3cf7229ce..17804ba0b02f 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -774,7 +774,7 @@ static int __init inject_init(void) debugfs_init(); - register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify"); + register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify", 0); mce_register_injector_chain(&inject_nb); setup_inj_struct(&i_mce); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 3e2533954675..d643d6fb3cfa 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -548,8 +548,7 @@ static void __init ms_hyperv_init_platform(void) lapic_timer_period); } - register_nmi_handler(NMI_UNKNOWN, hv_nmi_unknown, NMI_FLAG_FIRST, - "hv_nmi_unknown"); + register_nmi_handler(NMI_UNKNOWN, hv_nmi_unknown, NMI_FLAG_FIRST, "hv_nmi_unknown", 0); #endif #ifdef CONFIG_X86_IO_APIC diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 9c9faa1634fb..ab2d1b79b79e 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -602,13 +602,11 @@ int kgdb_arch_init(void) if (retval) goto out; - retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, - 0, "kgdb"); + retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, 0, "kgdb", 0); if (retval) goto out1; - retval = register_nmi_handler(NMI_UNKNOWN, kgdb_nmi_handler, - 0, "kgdb"); + retval = register_nmi_handler(NMI_UNKNOWN, kgdb_nmi_handler, 0, "kgdb", 0); if (retval) goto out2; diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index a010e9d062bf..b203e4371816 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -40,8 +40,7 @@ static int __init nmi_unk_cb(unsigned int val, struct pt_regs *regs) static void __init init_nmi_testsuite(void) { /* trap all the unknown NMIs we may generate */ - register_nmi_handler(NMI_UNKNOWN, nmi_unk_cb, 0, "nmi_selftest_unk", - __initdata); + register_nmi_handler(NMI_UNKNOWN, nmi_unk_cb, 0, "nmi_selftest_unk", 0, __initdata); } static void __init cleanup_nmi_testsuite(void) @@ -63,8 +62,8 @@ static void __init test_nmi_ipi(struct cpumask *mask) { unsigned long timeout; - if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, - NMI_FLAG_FIRST, "nmi_selftest", __initdata)) { + if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, NMI_FLAG_FIRST, + "nmi_selftest", 0, __initdata)) { nmi_fail = FAILURE; return; } diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 18266cc3d98c..b80812aa06c3 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -142,8 +142,8 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_reboot) static int register_stop_handler(void) { - return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, - NMI_FLAG_FIRST, "smp_stop"); + return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, NMI_FLAG_FIRST, "smp_stop", + 0); } static void native_stop_other_cpus(int wait) diff --git a/arch/x86/platform/uv/uv_nmi.c b/arch/x86/platform/uv/uv_nmi.c index 5c50e550ab63..473c34eb264c 100644 --- a/arch/x86/platform/uv/uv_nmi.c +++ b/arch/x86/platform/uv/uv_nmi.c @@ -1029,10 +1029,10 @@ static int uv_handle_nmi_ping(unsigned int reason, struct pt_regs *regs) static void uv_register_nmi_notifier(void) { - if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv")) + if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv", 0)) pr_warn("UV: NMI handler failed to register\n"); - if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping")) + if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping", 0)) pr_warn("UV: PING NMI handler failed to register\n"); } diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index b72772494655..95bd3a64608f 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -1318,7 +1318,7 @@ static void ghes_nmi_add(struct ghes *ghes) { mutex_lock(&ghes_list_mutex); if (list_empty(&ghes_nmi)) - register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes"); + register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes", 0); list_add_rcu(&ghes->list, &ghes_nmi); mutex_unlock(&ghes_list_mutex); } diff --git a/drivers/char/ipmi/ipmi_watchdog.c b/drivers/char/ipmi/ipmi_watchdog.c index f1875b2bebbc..5db402c4b9e7 100644 --- a/drivers/char/ipmi/ipmi_watchdog.c +++ b/drivers/char/ipmi/ipmi_watchdog.c @@ -1267,8 +1267,7 @@ static void check_parms(void) } } if (do_nmi && !nmi_handler_registered) { - rv = register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0, - "ipmi"); + rv = register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0, "ipmi", 0); if (rv) { pr_warn("Can't register nmi handler\n"); return; diff --git a/drivers/edac/igen6_edac.c b/drivers/edac/igen6_edac.c index 5807517ee32d..3a6e7334e94c 100644 --- a/drivers/edac/igen6_edac.c +++ b/drivers/edac/igen6_edac.c @@ -1363,8 +1363,7 @@ static int register_err_handler(void) return 0; } - rc = register_nmi_handler(NMI_SERR, ecclog_nmi_handler, - 0, IGEN6_NMI_NAME); + rc = register_nmi_handler(NMI_SERR, ecclog_nmi_handler, 0, IGEN6_NMI_NAME, 0); if (rc) { igen6_printk(KERN_ERR, "Failed to register NMI handler\n"); return rc; diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c index ae30e394d176..5246706afcf6 100644 --- a/drivers/watchdog/hpwdt.c +++ b/drivers/watchdog/hpwdt.c @@ -242,13 +242,13 @@ static int hpwdt_init_nmi_decoding(struct pci_dev *dev) /* * Only one function can register for NMI_UNKNOWN */ - retval = register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt", 0); if (retval) goto error; - retval = register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt", 0); if (retval) goto error1; - retval = register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt", 0); if (retval) goto error2; From patchwork Tue May 13 20:37:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 889800 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1C512C0333; Tue, 13 May 2025 20:38:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168683; cv=none; b=qcZ2PRm+xPgDlotxiTKntSd8TG6SvoIRaN1TNtWM1sHO3q6FsP5OyKiBIOaZr5towh40XsiLRP8klEJSxVZTAr5+Z/lfux5cTo/BrNCyHmGuodP6WQ7uX4QYsoJHbpmdpuwCCrhMouAkgmp6BNuHKvzFZAwAMycMAlW05av8+pA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168683; c=relaxed/simple; bh=UsS8b44BK8xWm+Kpa+GFLO2iwDJW/KAOeH2MgmgPXow=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fh243Oxe7CwvVAKN4um8rNpYuha0b2spmFXXoH3/bIRtyQ5iWjbHBsed4b5dG8Yr6X9Z7ycd1omEqxfscIz2cPdP4CPYO2SUwdOr9q90tMhhppdId9xOdcE1z0KUZJaOs7Eg++VWoGpMk8zOS6cUrrPrz7NvbnzM31yrt07mGlw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U8dmZ8UG; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U8dmZ8UG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1747168681; x=1778704681; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UsS8b44BK8xWm+Kpa+GFLO2iwDJW/KAOeH2MgmgPXow=; b=U8dmZ8UGNKAZz6ZafgFlmSbIgOA34jseBR1YS+gL2rU2SLKbyGB1m+RX kMO8wD1yDowPMjlxbJ6bqhRUliPQbxhjLisTtPojoRq4sM5QcZX1Hk1Cx c7kzvgVmCu0t4pRoNpo4sQYl+LHPxZGsilZDf7NrSWxQBS10XHv/d7ivu JokqfqL7o492oNj9UIB9Sw+eb88OGRYg8ySBexnYwS8gKgY9f7Xe2mGP5 Twz6Ihmw9TuFpbDjmq3P1GEQiEw3BWyR2XVnXucPC5KAabiMzsPLjwTVT T40jw+gegcgA6Yf4oA5kDSrF6QJ0b+GkKno4nHX1FDh1LgXHEZPHOREYL w==; X-CSE-ConnectionGUID: TzEh+w4OSOCXF2wXS+fE4w== X-CSE-MsgGUID: tJ5lom61R8mNeBGXJJxLFw== X-IronPort-AV: E=McAfee;i="6700,10204,11432"; a="49160450" X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="49160450" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2025 13:37:59 -0700 X-CSE-ConnectionGUID: j5ZqVJErRvSOkvaqAtfg7g== X-CSE-MsgGUID: dYb9E6QLRbqVzxTBiol/cw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="138241727" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by fmviesa008.fm.intel.com with ESMTP; 13 May 2025 13:37:58 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v6 5/9] x86/nmi: Add support to handle NMIs with source information Date: Tue, 13 May 2025 13:37:59 -0700 Message-ID: <20250513203803.2636561-6-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513203803.2636561-1-sohil.mehta@intel.com> References: <20250513203803.2636561-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The NMI-source bitmap is delivered as FRED event data to the kernel. When available, use NMI-source based filtering to determine the exact handlers to run. Activate NMI-source based filtering only for Local NMIs. While handling platform NMI types (such as SERR and IOCHK), do not use the source bitmap. They have only one handler registered per type, so there is no need to disambiguate between multiple handlers. Some third-party chipsets may send NMI messages with a hardcoded vector of 2, which would result in bit 2 being set in the NMI-source bitmap. Skip the local NMI handlers in this situation. Bit 0 of the source bitmap is set by the hardware whenever a source vector was not used while generating an NMI, or the originator could not be reliably identified. Poll all the registered handlers in that case. When multiple handlers need to be executed, adhere to the existing priority scheme and execute the handlers registered with NMI_FLAG_FIRST before others. The logic for handling legacy NMIs is unaffected since the source bitmap would always have all bits set. Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Sohil Mehta --- v6: Get rid of a separate NMI source matching function Set source_bitmap to ULONG_MAX to match all sources by default v5: Significantly simplify NMI-source handling logic. Get rid of a separate lookup table for NMI-source vectors. Adhere to existing priority scheme for handling NMIs. --- arch/x86/kernel/nmi.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 1a24e8df1bdf..55ecbe2ab5e4 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -130,6 +130,7 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration) static int nmi_handle(unsigned int type, struct pt_regs *regs) { struct nmi_desc *desc = nmi_to_desc(type); + unsigned long source_bitmap = ULONG_MAX; nmi_handler_t ehandler; struct nmiaction *a; int handled=0; @@ -148,16 +149,45 @@ static int nmi_handle(unsigned int type, struct pt_regs *regs) rcu_read_lock(); + /* + * Activate NMI source-based filtering only for Local NMIs. + * + * Platform NMI types (such as SERR and IOCHK) have only one + * handler registered per type, so there is no need to + * disambiguate between multiple handlers. + * + * Also, if a platform source ends up setting bit 2 in the + * source bitmap, the local NMI handlers would be skipped since + * none of them use this reserved vector. + * + * For Unknown NMIs, avoid using the source bitmap to ensure all + * potential handlers have a chance to claim responsibility. + */ + if (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE) && type == NMI_LOCAL) { + source_bitmap = fred_event_data(regs); + + /* Reset the bitmap if a valid source could not be identified */ + if (WARN_ON_ONCE(!source_bitmap) || (source_bitmap & BIT(NMIS_VECTOR_NONE))) + source_bitmap = ULONG_MAX; + } + /* * NMIs are edge-triggered, which means if you have enough * of them concurrently, you can lose some because only one * can be latched at any given time. Walk the whole list * to handle those situations. + * + * However, NMI-source reporting does not have this limitation. + * When NMI sources have been identified, only run the handlers + * that match the reported vectors. */ list_for_each_entry_rcu(a, &desc->head, list) { int thishandled; u64 delta; + if (!(source_bitmap & BIT(a->source_vector))) + continue; + delta = sched_clock(); thishandled = a->handler(type, regs); handled += thishandled; From patchwork Tue May 13 20:38:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 889799 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CDBB2C1098; Tue, 13 May 2025 20:38:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168684; cv=none; b=XtEpsoj9okjJcSdkvXcgPG8maUHu/ViXqKjK8xTRvl/oYV1NI2mYYeA1BJagJR4xTfryO5Vgqj0b3TrIFs3PYMncJkQHNn0mpMRHtUnxCU+lCF3XkytWV/AQo9po/tvjVUxGBKmaqUPd9TbcGQx1zwgLa3YeVgJFWaKXYgQAZEs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168684; c=relaxed/simple; bh=tuP0z5QJYB3K4sZ/QOSUtwEDpmxwT+ADMb45rtxF958=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hekL7lqBPhv7wWDY6YBq//31r7dl2gRTJUsd6VzJRbRXS3lu6mL5hde+hT2MjTIW4WK5SG7jmFG+ybqJb9nDF4IYQhoBkh0AvdZox/IrPLyiQ5EOqmvFNW37uGgQZORMtjO9u0UX/lscJRh6cwosa5IjP6RROInS0zJUFtYOG6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QlAJxtkQ; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QlAJxtkQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1747168683; x=1778704683; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tuP0z5QJYB3K4sZ/QOSUtwEDpmxwT+ADMb45rtxF958=; b=QlAJxtkQ2rZeu64K69LsFOV4ZSV6waPTZMecXokLN/+PaQT7G9vx1/MI fTpqWM66Hk+vVISZTvggnFTGCdvZX+cLl24Wa4Vz7LoTYEn2mUfkW6X6V mwMfqlYn77+D2Zvcl3Q8tDckIijoDJJTqIWj/sf4TQgKXptpxiOTpbXIZ pA4E0/YGoYpBuYcWhKIxTEOlgjNkxMFPwizRpN/jReHXEoDq+oOiJBYlZ jgtxaVNbiDb33d0I32Q9OtXB6sb/f4A0u37Z8WnU6zrGllBetg5cGC5oy 178EbnfURpzpq4I2HhSIgE/9SarXkO55ne4yqIa1KVZf4orP48yf/QhlB w==; X-CSE-ConnectionGUID: YZLLzq3mTjOjqhTkHQ8J5w== X-CSE-MsgGUID: HVx6UgARTS64GM6nkYlYgg== X-IronPort-AV: E=McAfee;i="6700,10204,11432"; a="49160474" X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="49160474" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2025 13:38:00 -0700 X-CSE-ConnectionGUID: LnsfcjOvSLCJQFVQ6UcCKg== X-CSE-MsgGUID: c3Uf+7qEQGarDivPmKTZrg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="138241733" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by fmviesa008.fm.intel.com with ESMTP; 13 May 2025 13:37:59 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v6 7/9] x86/nmi: Enable NMI-source for IPIs delivered as NMIs Date: Tue, 13 May 2025 13:38:01 -0700 Message-ID: <20250513203803.2636561-8-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513203803.2636561-1-sohil.mehta@intel.com> References: <20250513203803.2636561-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With the IPI handling APIs ready to support the new NMI encoding, encode the NMI delivery mode directly with the NMI-source vectors to trigger NMIs. Move most of the existing NMI-based IPIs to use the new NMI-source vectors, except for the microcode rendezvous NMI and the crash reboot NMI. NMI handling for them is special-cased in exc_nmi() and does not need NMI-source reporting. However, in the future, it might be useful to assign a source vector to all NMI sources to improve isolation and debuggability. Originally-by: Jacob Pan Suggested-by: Sean Christopherson Co-developed-by: Xin Li (Intel) Signed-off-by: Xin Li (Intel) Signed-off-by: Sohil Mehta --- v6: Include asm/nmi.h to avoid compile errors. (LKP) v5: Encode APIC_DM_NMI directly with the NMI-source vector. --- arch/x86/include/asm/apic.h | 8 ++++++++ arch/x86/kernel/apic/hw_nmi.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/kgdb.c | 2 +- arch/x86/kernel/nmi_selftest.c | 2 +- arch/x86/kernel/smp.c | 2 +- 6 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 9c3d5932d591..99033bfb26ea 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #define ARCH_APICTIMER_STOPS_ON_C3 1 @@ -23,6 +24,13 @@ #define APIC_EXTNMI_ALL 1 #define APIC_EXTNMI_NONE 2 +/* Trigger NMIs with source information */ +#define TEST_NMI (APIC_DM_NMI | NMIS_VECTOR_TEST) +#define SMP_STOP_NMI (APIC_DM_NMI | NMIS_VECTOR_SMP_STOP) +#define BT_NMI (APIC_DM_NMI | NMIS_VECTOR_BT) +#define KGDB_NMI (APIC_DM_NMI | NMIS_VECTOR_KGDB) +#define MCE_NMI (APIC_DM_NMI | NMIS_VECTOR_MCE) + /* * Debugging macros */ diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 4e04f13d2de9..586f4b25feae 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -33,7 +33,7 @@ u64 hw_nmi_get_sample_period(int watchdog_thresh) #ifdef arch_trigger_cpumask_backtrace static void nmi_raise_cpu_backtrace(cpumask_t *mask) { - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, BT_NMI); } void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c index a3c753dfce91..6328a607ffc4 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -269,7 +269,7 @@ static void __maybe_unused raise_mce(struct mce *m) mce_irq_ipi, NULL, 0); preempt_enable(); } else if (m->inject_flags & MCJ_NMI_BROADCAST) - __apic_send_IPI_mask(mce_inject_cpumask, NMI_VECTOR); + __apic_send_IPI_mask(mce_inject_cpumask, MCE_NMI); } start = jiffies; while (!cpumask_empty(mce_inject_cpumask)) { diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 9ca4b141da0c..3dedc5f57541 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -416,7 +416,7 @@ static void kgdb_disable_hw_debug(struct pt_regs *regs) */ void kgdb_roundup_cpus(void) { - apic_send_IPI_allbutself(NMI_VECTOR); + apic_send_IPI_allbutself(KGDB_NMI); } #endif diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index 5196023b31dc..c5c91f520c69 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -71,7 +71,7 @@ static void __init test_nmi_ipi(struct cpumask *mask) /* sync above data before sending NMI */ wmb(); - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, TEST_NMI); /* Don't wait longer than a second */ timeout = USEC_PER_SEC; diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 5be1c0bdf901..614acec5655f 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -217,7 +217,7 @@ static void native_stop_other_cpus(int wait) pr_emerg("Shutting down cpus with NMI\n"); for_each_cpu(cpu, &cpus_stop_mask) - __apic_send_IPI(cpu, NMI_VECTOR); + __apic_send_IPI(cpu, SMP_STOP_NMI); } /* * Don't wait longer than 10 ms if the caller didn't From patchwork Tue May 13 20:38:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 889798 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 522B62C1799; Tue, 13 May 2025 20:38:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168685; cv=none; b=iEHKxiJs3nV1ZyGWOocopTdBCrDlbft1bQf8TGzZD+YVdnDdRch1mMMvzsCcW5/QuQAX22e90VR08oe5+jHtnJTFZCg6QmKN9nCn4QDXnhfrsR8Xu8Dg8RasseimFuNeLRCtuWvg3sbygsN0Hg9E18Ts5IowvZx3bx36Cp6DuEg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747168685; c=relaxed/simple; bh=agLNp8oAxzQaWYhjfHFlOBrApju2Hbb0chhD8+zYEOA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=E9duIGLx0Fj6biFNhDDuBBV4lNeLNdy0IW9Q5v9w6gszlPp9jzFgYSd/l6rEjG8bbMnaqIMDeb1DUBt/iRZx5KQWZUZu76iVeuPfWKjKUC5L5mkGLEXIPu7ZOPhoM5AeJlH2tjLNIErYPDyL1Lm9ixeYtlQRt5Kj8/Xwb/fCEt0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SlMflK2O; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SlMflK2O" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1747168683; x=1778704683; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=agLNp8oAxzQaWYhjfHFlOBrApju2Hbb0chhD8+zYEOA=; b=SlMflK2Ozr16aF5H5ehYlRXNlpcUmvALZCqGgA13XyBw7BpNk+2COTp8 pSaOxDQ1JcXidMDw4sQGHKQdTSmkoGrPqfHalPqcKy/G2XhircdVErQQH ZW7inRo2Fs6wZfwCVi/dSklMdbW3TUq3YzRAUQblr4cNnOYtuQgUZanr/ wfpC4e9MsBrj4acQnGzL7aXvDH4nv8e3gogyuzCH6Y6cKoupR98dcb+qk l3TEwI7sK0TT8sFMXZ9sDstSKugozphKbE/ShCxwm8gFm1UquWnousGHN RbQStn5lYbOr5lkKsFHiGdmg27SA65Z7CC3sirJL9am3YaP1N+tiYpIQK A==; X-CSE-ConnectionGUID: V2wVW8IIT1yC6ehFsK5WXg== X-CSE-MsgGUID: +Dc6+vhnR8yIvD6SqIYo6g== X-IronPort-AV: E=McAfee;i="6700,10204,11432"; a="49160486" X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="49160486" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2025 13:38:00 -0700 X-CSE-ConnectionGUID: UCq+j34SQnyxy4tBG8PDfQ== X-CSE-MsgGUID: VT1VdeJ9T1Sfv0wsLdaadw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,286,1739865600"; d="scan'208";a="138241737" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by fmviesa008.fm.intel.com with ESMTP; 13 May 2025 13:37:59 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v6 8/9] perf/x86: Enable NMI-source reporting for perfmon Date: Tue, 13 May 2025 13:38:02 -0700 Message-ID: <20250513203803.2636561-9-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250513203803.2636561-1-sohil.mehta@intel.com> References: <20250513203803.2636561-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jacob Pan Program the designated PMI NMI-source vector into the local vector table for the PMU. An NMI for the PMU would directly invoke the PMI handler without polling other NMI handlers, resulting in reduced PMI delivery latency. Co-developed-by: Zeng Guang Signed-off-by: Zeng Guang Signed-off-by: Jacob Pan Signed-off-by: Sohil Mehta Reviewed-by: Kan Liang Tested-by: Sandipan Das # AMD overlapping bits --- v6: Picked up a tested-by tag. v5: No significant change. --- arch/x86/events/core.c | 4 ++-- arch/x86/events/intel/core.c | 6 +++--- arch/x86/include/asm/apic.h | 1 + 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 031e908f0d61..42b270526631 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -1695,7 +1695,7 @@ int x86_pmu_handle_irq(struct pt_regs *regs) * This generic handler doesn't seem to have any issues where the * unmasking occurs so it was left at the top. */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); for_each_set_bit(idx, x86_pmu.cntr_mask, X86_PMC_IDX_MAX) { if (!test_bit(idx, cpuc->active_mask)) @@ -1737,7 +1737,7 @@ void perf_events_lapic_init(void) /* * Always use NMI for PMU */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); } static int diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 09d2d66c9f21..87c624686c58 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3202,7 +3202,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * NMI handler. */ if (!late_ack && !mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); intel_bts_disable_local(); cpuc->enabled = 0; __intel_pmu_disable_all(true); @@ -3239,7 +3239,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) done: if (mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); /* Only restore PMU state when it's active. See x86_pmu_disable(). */ cpuc->enabled = pmu_enabled; if (pmu_enabled) @@ -3252,7 +3252,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * Haswell CPUs. */ if (late_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); return handled; } diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 99033bfb26ea..d637717d42bd 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -30,6 +30,7 @@ #define BT_NMI (APIC_DM_NMI | NMIS_VECTOR_BT) #define KGDB_NMI (APIC_DM_NMI | NMIS_VECTOR_KGDB) #define MCE_NMI (APIC_DM_NMI | NMIS_VECTOR_MCE) +#define PERF_NMI (APIC_DM_NMI | NMIS_VECTOR_PMI) /* * Debugging macros