From patchwork Thu Jun 12 21:48:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 896146 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73C452D5419; Thu, 12 Jun 2025 21:50:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765030; cv=none; b=KuSSevxOEOblXzJUnljtUxWC9uZholPSJZD4EaaMB6veVu3nFaPPKT1eJNtOVLJzGeAtSgoAfvdcZwi0wTD+AsnN+uPwHUIB5tG4GA5i8ke05D/Yf80IulSPE7NefQ+d0enPZbgbKRkjohmUq+tAmCKmumbLVFWObaIpzvsAGg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765030; c=relaxed/simple; bh=TPNvueekJgJ+6J+ut2o8hb/2cBDnuiMm8XuCz7DRZyE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P7dWdktilO+jznMQ5BDAux3Gb0w9rPh/HK0DzDyEZr9tj/phLddtjoSs0fONfpfJxYWniUjZ7h/a4EbeLyFidk4R+7/AQydut4Czmskd5lrKs9WsYvILiuxFE0F8Jk83hpyqdtEkvcK87iWFGBpJQaW3RCMcFH5AZzjGyJ6wUMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QS9zHrir; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QS9zHrir" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765028; x=1781301028; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TPNvueekJgJ+6J+ut2o8hb/2cBDnuiMm8XuCz7DRZyE=; b=QS9zHrirgs6MjAutDVQcoFrl+URdnAkZEKOSswk0hzWVA2vOu+crgbKc QQAdjLYbt3zhzYMFkFLjy8nYrvegb2XWBinjoS8ojogUwVsF51VLx29+1 E+ZxbZ+rX+3vJOiW42oGql+TD8Yoj5u6MB9TOMy4hycD8AP5VNiP3wdM/ EqHvnIFTPrKIHPtS0xWZZAFi0tTncEfEbAtbaAueYBws2A6nvB8/7Hu2X AiUhnLCW3Y0lZHs35cvf/6Ua2XAHHJRgU8uulX/YrUSFqgQkkw58J1LLV KB/kfL6BDobKfSkzfZe54fJUv/hI3RepXm8HHaQ8Lulqnugrn7wDyaZ5F A==; X-CSE-ConnectionGUID: Al2D1JlJR8OgLvT9+fFImw== X-CSE-MsgGUID: ZvsP5f4rQFGRJhAH/MBtng== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065295" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065295" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:26 -0700 X-CSE-ConnectionGUID: +4JjIdO4R+qu1KI0pUsD3w== X-CSE-MsgGUID: BkRwew3wSPWPBHilWxT/HQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881935" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:25 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 01/10] x86/fred: Provide separate IRQ vs. NMI wrappers for entry from KVM Date: Thu, 12 Jun 2025 14:48:40 -0700 Message-ID: <20250612214849.3950094-2-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Provide separate wrappers for forwarding IRQs vs NMIs from KVM in anticipation of adding support for NMI source reporting, which will add an NMI-only parameter, i.e. will further pollute the current API with a param that is a hardcoded for one of the two call sites. Opportunistically tag the non-FRED NMI wrapper __always_inline, as the compiler could theoretically generate a function call and trigger and a (completely benign) "leaving noinstr" warning. Signed-off-by: Sean Christopherson Signed-off-by: Sohil Mehta --- v7: New patch --- arch/x86/include/asm/fred.h | 30 ++++++++++++++++++++++-------- arch/x86/kvm/vmx/vmx.c | 4 ++-- 2 files changed, 24 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 12b34d5b2953..552332ca060c 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -71,15 +72,27 @@ __visible void fred_entry_from_user(struct pt_regs *regs); __visible void fred_entry_from_kernel(struct pt_regs *regs); __visible void __fred_entry_from_kvm(struct pt_regs *regs); -/* Can be called from noinstr code, thus __always_inline */ -static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) +/* Must be called from noinstr code, thus __always_inline */ +static __always_inline void fred_nmi_from_kvm(void) { struct fred_ss ss = { - .ss =__KERNEL_DS, - .type = type, - .vector = vector, - .nmi = type == EVENT_TYPE_NMI, - .lm = 1, + .ss = __KERNEL_DS, + .type = EVENT_TYPE_NMI, + .vector = NMI_VECTOR, + .nmi = true, + .lm = 1, + }; + + asm_fred_entry_from_kvm(ss); +} + +static inline void fred_irq_from_kvm(unsigned int vector) +{ + struct fred_ss ss = { + .ss = __KERNEL_DS, + .type = EVENT_TYPE_EXTINT, + .vector = vector, + .lm = 1, }; asm_fred_entry_from_kvm(ss); @@ -110,7 +123,8 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { ret static inline void cpu_init_fred_exceptions(void) { } static inline void cpu_init_fred_rsps(void) { } static inline void fred_complete_exception_setup(void) { } -static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { } +static __always_inline void fred_nmi_from_kvm(void) { } +static inline void fred_irq_from_kvm(unsigned int vector) { } static inline void fred_sync_rsp0(unsigned long rsp0) { } static inline void fred_update_rsp0(void) { } #endif /* CONFIG_X86_FRED */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 4953846cb30d..8183886eda3d 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6991,7 +6991,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); if (cpu_feature_enabled(X86_FEATURE_FRED)) - fred_entry_from_kvm(EVENT_TYPE_EXTINT, vector); + fred_irq_from_kvm(vector); else vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector)); kvm_after_interrupt(vcpu); @@ -7264,7 +7264,7 @@ noinstr void vmx_handle_nmi(struct kvm_vcpu *vcpu) kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); if (cpu_feature_enabled(X86_FEATURE_FRED)) - fred_entry_from_kvm(EVENT_TYPE_NMI, NMI_VECTOR); + fred_nmi_from_kvm(); else vmx_do_nmi_irqoff(); kvm_after_interrupt(vcpu); From patchwork Thu Jun 12 21:48:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 895844 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B93302D8DB1; Thu, 12 Jun 2025 21:50:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765030; cv=none; b=tAkTkXVN7I7FNDmJD6WC7Vsg5luViBAmtnkOBT1UOKlH3HfX8Ec1IO0AOO1GWSak1Zb49LvAXGiosyiip9yIbWjDzVv0iB24Mhbvp11nqUV22sbOwiWo8ogGvUWVrWQjp+0cZ7lwiEYNfqIkIGMexA4aofOXbr1COLPYvEN8euI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765030; c=relaxed/simple; bh=137GFuU0KoBhNJyOTqbSzEaWwq8IPMkvvJDiWbT7sJ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kAOGX28+BOPRM1nEfh+l53M3dwjS83G6oYQM6uNeZI3gKC5PoBdPW+ZoZIM99N/wyNLsf2lS1bJVDzwEbXjNyyE4DXcZKfix/6hJcV3l25e02ALHZlLgwrA95SrNzxf5P6lSbnUOFSev7EZHHYXVpRF38R+G9HRq1bd3DCGXwvg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QR5iz42j; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QR5iz42j" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765029; x=1781301029; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=137GFuU0KoBhNJyOTqbSzEaWwq8IPMkvvJDiWbT7sJ8=; b=QR5iz42jRmqDZbXYnu6A5UCqvZrWsTp6l2IlGsmXJLv8ytZUK/WB9dvk fnmDX4uL4P0W/lDJEnwcVmBTY5gcQLFq2c4ctxsRupTRwMomptn4+FqZm fWNjT+bBb08H7d6heZQg9Qk3A8re0o+pn4YTVFBqpsHQZFcTS+9z0caMj IvnK3UOBLaRm2ZDdfELJ0IbSzhyUgKuQjGj37I0i319tAAK0eAQNOfk8o KA+c4rPPvAzHdpVt87GkOwhtaeOuarF27NQJWN4dsBEtErN0h81qleGNH KuKFJFC3nyX69T1RiXGL0qkV0zGynm2oiL/Ya0jSML3+0X8ZD47DDJfI/ Q==; X-CSE-ConnectionGUID: 8h2nae7mTO+gD/qPP4FLZg== X-CSE-MsgGUID: sX3HBmmyRpye2YaSmRnzeg== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065307" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065307" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:27 -0700 X-CSE-ConnectionGUID: MIiUCDU6TLeeYuJIpa97LA== X-CSE-MsgGUID: rkm4chbuTtGgFW6Ln+MC+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881943" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:26 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 02/10] x86/fred: Pass event data to the NMI entry point from KVM Date: Thu, 12 Jun 2025 14:48:41 -0700 Message-ID: <20250612214849.3950094-3-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Extend the FRED NMI entry point from KVM to take an extra argument to allow KVM to invoke the FRED event dispatch framework with event data. This API is used to pass the NMI-source bitmap for NMI-induced VM exits. Read the VMCS exit qualification field to get the NMI-source information and store it as event data precisely in the format expected by the FRED event framework. Read the VMCS exit qualification unconditionally since almost all upcoming CPUs are expected to enable FRED and NMI-source together. In the rare case that NMI-source isn't enabled, the extra VMREAD would be harmless since the exit qualification is expected to be zero. Suggested-by: Sean Christopherson Originally-by: Zeng Guang Signed-off-by: Sohil Mehta Acked-by: Sean Christopherson --- v7: Pass the event data from KVM only for NMI. (Sean) v6: No change v5: Read the VMCS exit qualification unconditionally. (Sean) Combine related patches into one. --- arch/x86/entry/entry_64_fred.S | 2 +- arch/x86/include/asm/fred.h | 11 ++++++----- arch/x86/kvm/vmx/vmx.c | 2 +- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index 29c5c32c16c3..1c9c6e036233 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -93,7 +93,7 @@ SYM_FUNC_START(asm_fred_entry_from_kvm) * +--------+-----------------+ */ push $0 /* Reserved, must be 0 */ - push $0 /* Event data, 0 for IRQ/NMI */ + push %rsi /* Event data for NMI */ push %rdi /* fred_ss handed in by the caller */ push %rbp pushf diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 552332ca060c..bccf4a3c06b8 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -66,14 +66,14 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) void asm_fred_entrypoint_user(void); void asm_fred_entrypoint_kernel(void); -void asm_fred_entry_from_kvm(struct fred_ss); +void asm_fred_entry_from_kvm(struct fred_ss ss, unsigned long edata); __visible void fred_entry_from_user(struct pt_regs *regs); __visible void fred_entry_from_kernel(struct pt_regs *regs); __visible void __fred_entry_from_kvm(struct pt_regs *regs); /* Must be called from noinstr code, thus __always_inline */ -static __always_inline void fred_nmi_from_kvm(void) +static __always_inline void fred_nmi_from_kvm(unsigned long edata) { struct fred_ss ss = { .ss = __KERNEL_DS, @@ -83,7 +83,7 @@ static __always_inline void fred_nmi_from_kvm(void) .lm = 1, }; - asm_fred_entry_from_kvm(ss); + asm_fred_entry_from_kvm(ss, edata); } static inline void fred_irq_from_kvm(unsigned int vector) @@ -95,7 +95,8 @@ static inline void fred_irq_from_kvm(unsigned int vector) .lm = 1, }; - asm_fred_entry_from_kvm(ss); + /* Event data is always zero for IRQ */ + asm_fred_entry_from_kvm(ss, 0); } void cpu_init_fred_exceptions(void); @@ -123,7 +124,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { ret static inline void cpu_init_fred_exceptions(void) { } static inline void cpu_init_fred_rsps(void) { } static inline void fred_complete_exception_setup(void) { } -static __always_inline void fred_nmi_from_kvm(void) { } +static __always_inline void fred_nmi_from_kvm(unsigned long edata) { } static inline void fred_irq_from_kvm(unsigned int vector) { } static inline void fred_sync_rsp0(unsigned long rsp0) { } static inline void fred_update_rsp0(void) { } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 8183886eda3d..69832ca31e5b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7264,7 +7264,7 @@ noinstr void vmx_handle_nmi(struct kvm_vcpu *vcpu) kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); if (cpu_feature_enabled(X86_FEATURE_FRED)) - fred_nmi_from_kvm(); + fred_nmi_from_kvm(vmx_get_exit_qual(vcpu)); else vmx_do_nmi_irqoff(); kvm_after_interrupt(vcpu); From patchwork Thu Jun 12 21:48:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 896145 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BA312DECAA; Thu, 12 Jun 2025 21:50:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765031; cv=none; b=M6hE/JfcNdF75bbaH56lKhXK94s9EBa9mucusP4/7Fip4BgGMl9MG8Sp8tdnDFGwoJDkYo0RLe8CM4+lUgjiZOBaRVi2pW6yhk2FUjjYdg+FjFxfz8DGjuy/vw6IHq2EBj0XkLKvamBPU4ePsiq+eJ+ii6wvUKAjMapj5unaHKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765031; c=relaxed/simple; bh=nEU4c5folNgSZMdRvC95laRn9b0KITzv+SiB81fV5c4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ESSUtxPZE9SoiilPJ/SCUUVufY8suN8HMdzr9ggzPEPaE50mONC7aHFahaiZWif3o9rgeSO2BzpIcRgfJRnDbkqJr1JvDCaDVhp+wMmb9Oq05lcgAKo+fGwtZeGLqt9ld2dZ07DOpQ1Hy0nBGp30WUC+s8h5KQywGt07BWcJrZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mT/gdVrc; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mT/gdVrc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765029; x=1781301029; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nEU4c5folNgSZMdRvC95laRn9b0KITzv+SiB81fV5c4=; b=mT/gdVrcdCTKmun64yol/JKwbcsnawj+1682mpqDhLzXLnJGXOHWpS8C fTJ7YVU/MMrqcDfmxZb2Uf6KAoUGasCTUAwkrLuHUzuFctez6j4egA5bw c9HcLgYz/UZ0pC8ZVsqgoX0igBV+F6Q/syopa4b4uo4tSVuumEzSDGCwm efLwHAnhKumh5wVQxGEtnYe3aIkjspVBRJeunmrgj1sPRI8L5e6prO1FC ZbBLRIEJEPR7U7w0gl2ulaOIgLY5vHkcUu5QzttmVqn+qQxMg6iFL5J2v qGyhhY8s2ICupPJGmEbTBTv87A2LmcQ+KNBKCYSOTaO/u9ibTO6LVuWZ8 g==; X-CSE-ConnectionGUID: lZEZjgHYRbuT+suHZwziMw== X-CSE-MsgGUID: omrkx81yRJG/hPgZToa6fw== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065320" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065320" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:28 -0700 X-CSE-ConnectionGUID: YrD6BlnuTMuaRgWFuloeZQ== X-CSE-MsgGUID: FMyAUOwDRLKTB3RCrR0Swg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881950" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:27 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 03/10] x86/cpufeatures: Add the CPUID feature bit for NMI-source reporting Date: Thu, 12 Jun 2025 14:48:42 -0700 Message-ID: <20250612214849.3950094-4-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 NMI-source reporting is introduced to report the sources of NMIs with FRED event delivery based on vectors in NMI interrupt messages or the local APIC. This enables the kernel to avoid the latency incurred by going over the entire NMI handler list and reduces ambiguity about the source of an NMI. Enumerate NMI-source reporting in cpufeatures.h. Also, since NMI-source reporting uses the FRED event dispatch framework, make it dependent on FRED in the CPUID dependency table. This ensures that NMI-source reporting gets disabled when FRED is disabled. NMI-source reporting is intended as a kernel feature and does not need userspace enumeration or configuration. There is no need to expose it to userspace through /proc/cpuinfo. Originally-by: Jacob Pan Signed-off-by: Sohil Mehta --- v7: No change. v6: No change. v5: Add NMI-source to the CPUID dependency table. Do not expose NMI-source feature through /proc/cpuinfo. --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/cpuid-deps.c | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index ee176236c2be..b6c907666d5f 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -321,6 +321,7 @@ #define X86_FEATURE_FRED (12*32+17) /* "fred" Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18) /* Load "kernel" (userspace) GS */ #define X86_FEATURE_WRMSRNS (12*32+19) /* Non-serializing WRMSR */ +#define X86_FEATURE_NMI_SOURCE (12*32+20) /* NMI-Source reporting with FRED */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* Support for VPMADD52[H,L]UQ */ #define X86_FEATURE_LAM (12*32+26) /* "lam" Linear Address Masking */ diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c index 46efcbd6afa4..87a334f639d5 100644 --- a/arch/x86/kernel/cpu/cpuid-deps.c +++ b/arch/x86/kernel/cpu/cpuid-deps.c @@ -88,6 +88,7 @@ static const struct cpuid_dep cpuid_deps[] = { { X86_FEATURE_AMX_INT8, X86_FEATURE_AMX_TILE }, { X86_FEATURE_SHSTK, X86_FEATURE_XSAVES }, { X86_FEATURE_FRED, X86_FEATURE_LKGS }, + { X86_FEATURE_NMI_SOURCE, X86_FEATURE_FRED }, { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, {} }; From patchwork Thu Jun 12 21:48:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 895843 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69CEF2DECCC; Thu, 12 Jun 2025 21:50:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765032; cv=none; b=g6EW2EsfpH4vJacVLsHEp4ih39r5AUUJ1xru6G5R2pTsAue/eSZiYgpoM8+PsDGgK4/0Vwe6Hsl0Rx0m8H6RIMs+RlOGKgOjMaYpO6Zb1D2btoRAomWNVLQKx3ok9MOMwfxhZ7Pz/Q6jPKmirclT69lkU9g6gvdiAtESbSz3ug4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765032; c=relaxed/simple; bh=2/xhG0RxEEkF6sBv0GVk6FTgsRwJBueY4hWmcwCEohw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DBUTx7XeIL8+worowta0kdKptu677WHVLCFgedcedX5FGeNs/Ya3BDRuoyVmqwbe543ROx/UUzeqrRQh4QW4bx6XurWAZwmUqJK0wPVyAKc4MUaWmCxWSio3RVNlCL8NDTu6n0MLVA6JrBuExgfhE5MRhztzaLO6+cULm8qcrx0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=i7iAanpO; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="i7iAanpO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765030; x=1781301030; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2/xhG0RxEEkF6sBv0GVk6FTgsRwJBueY4hWmcwCEohw=; b=i7iAanpOs4lEXXlOLQSODx41EBZZhF480ssfvxIf0CTBxnRdMdlZQfhC 2w8GOHtk66oREQRw/7fvutO+oKo4WnZyHdJLEC/Y9j/96UnxNSp+3hK7i rOQDsAf42BXTT/EhSEzjNKP1rmPmTFTEm8UUYMz6yQ9IUHlzBZWdGLExG 77tfGO1tye7ghXFoodLyYsyBB7FBIv4s1hd6kAqWHvwipobMCY4jBwwGL O5s3YEU98jBpHQoEK7R9rcKF72Isi1ThOQYqXKwE6TwHKCdXPLpQlNoiH 3ZfJEf6tawOkG0K3g+Aau69lGRVDbNRx2IejKIUNhIr4M6/hufHNndfiu Q==; X-CSE-ConnectionGUID: JbntafMlRu2Pb1iZ3ofNcg== X-CSE-MsgGUID: SndOTVf0Rke80tYCAdHxDw== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065332" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065332" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:29 -0700 X-CSE-ConnectionGUID: WFypfaUVSA2ZX/aLhGqxDA== X-CSE-MsgGUID: YgV9Uhs1SoGU4Z0OHMaH4g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881957" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:29 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 04/10] x86/nmi: Extend the registration interface to include the NMI-source vector Date: Thu, 12 Jun 2025 14:48:43 -0700 Message-ID: <20250612214849.3950094-5-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To prepare for NMI-source reporting, add a source vector argument to the NMI handler registration interface. Later, this will be used to register NMI handlers with a unique source vector that can be used to identify the originator of the NMI. Vector 0 is reserved by the hardware for situations when a source vector is not used while generating an NMI or the originator could not be reliably identified. Registering an NMI handler with vector 0 is equivalent to not using NMI-source reporting. For now, just extend the interface with no source information (vector 0) for all the handlers. No functional change intended. Originally-by: Jacob Pan Signed-off-by: Sohil Mehta Reviewed-by: Xin Li (Intel) --- v7: Use an enum for defining source vectors (DaveH). Use NMIS_NO_SOURCE instead of zero (DaveH). Add review tag (Xin). v6: No change. v5: Split the patch into two parts. This one only extends the interface. --- arch/x86/events/amd/ibs.c | 2 +- arch/x86/events/core.c | 2 +- arch/x86/include/asm/nmi.h | 16 +++++++++++++++- arch/x86/kernel/apic/hw_nmi.c | 3 +-- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/mshyperv.c | 2 +- arch/x86/kernel/kgdb.c | 6 ++---- arch/x86/kernel/nmi_selftest.c | 6 +++--- arch/x86/kernel/smp.c | 4 ++-- arch/x86/platform/uv/uv_nmi.c | 4 ++-- drivers/acpi/apei/ghes.c | 2 +- drivers/char/ipmi/ipmi_watchdog.c | 3 +-- drivers/edac/igen6_edac.c | 3 +-- drivers/watchdog/hpwdt.c | 6 +++--- 14 files changed, 35 insertions(+), 26 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 112f43b23ebf..6f8f0d663f2f 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1485,7 +1485,7 @@ static __init int perf_event_ibs_init(void) if (ret) goto err_op; - ret = register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ibs"); + ret = register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ibs", NMIS_NO_SOURCE); if (ret) goto err_nmi; diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 7610f26dfbd9..ec59837d10d1 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2129,7 +2129,7 @@ static int __init init_hw_perf_events(void) x86_pmu.config_mask = X86_RAW_EVENT_MASK; perf_events_lapic_init(); - register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI"); + register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI", NMIS_NO_SOURCE); unconstrained = (struct event_constraint) __EVENT_CONSTRAINT(0, x86_pmu.cntr_mask64, diff --git a/arch/x86/include/asm/nmi.h b/arch/x86/include/asm/nmi.h index 79d88d12c8fb..42820c4f59b9 100644 --- a/arch/x86/include/asm/nmi.h +++ b/arch/x86/include/asm/nmi.h @@ -48,12 +48,24 @@ enum { typedef int (*nmi_handler_t)(unsigned int, struct pt_regs *); +/** + * enum nmi_source_vectors - NMI-source vectors are used to identify the + * origin of an NMI and to route the NMI directly to the appropriate + * handler. + * + * @NMIS_NO_SOURCE: Reserved for undefined or unidentified sources. + */ +enum nmi_source_vectors { + NMIS_NO_SOURCE = 0, +}; + struct nmiaction { struct list_head list; nmi_handler_t handler; u64 max_duration; unsigned long flags; const char *name; + enum nmi_source_vectors source_vector; }; /** @@ -62,6 +74,7 @@ struct nmiaction { * @fn: The NMI handler * @fg: Flags associated with the NMI handler * @n: Name of the NMI handler + * @src: NMI-source vector for the NMI handler * @init: Optional __init* attributes for struct nmiaction * * Adds the provided handler to the list of handlers for the specified @@ -75,13 +88,14 @@ struct nmiaction { * * Return: 0 on success, or an error code on failure. */ -#define register_nmi_handler(t, fn, fg, n, init...) \ +#define register_nmi_handler(t, fn, fg, n, src, init...)\ ({ \ static struct nmiaction init fn##_na = { \ .list = LIST_HEAD_INIT(fn##_na.list), \ .handler = (fn), \ .name = (n), \ .flags = (fg), \ + .source_vector = (src), \ }; \ __register_nmi_handler((t), &fn##_na); \ }) diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 45af535c44a0..d09e771723ed 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -53,8 +53,7 @@ NOKPROBE_SYMBOL(nmi_cpu_backtrace_handler); static int __init register_nmi_cpu_backtrace_handler(void) { - register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, - 0, "arch_bt"); + register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, 0, "arch_bt", NMIS_NO_SOURCE); return 0; } early_initcall(register_nmi_cpu_backtrace_handler); diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c index d02c4f556cd0..ba70ef8a1964 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -775,7 +775,7 @@ static int __init inject_init(void) debugfs_init(); - register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify"); + register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify", NMIS_NO_SOURCE); mce_register_injector_chain(&inject_nb); setup_inj_struct(&i_mce); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index c78f860419d6..c093f7baab6a 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -550,7 +550,7 @@ static void __init ms_hyperv_init_platform(void) } register_nmi_handler(NMI_UNKNOWN, hv_nmi_unknown, NMI_FLAG_FIRST, - "hv_nmi_unknown"); + "hv_nmi_unknown", NMIS_NO_SOURCE); #endif #ifdef CONFIG_X86_IO_APIC diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 102641fd2172..ae28fdeefc7f 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -602,13 +602,11 @@ int kgdb_arch_init(void) if (retval) goto out; - retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, - 0, "kgdb"); + retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, 0, "kgdb", NMIS_NO_SOURCE); if (retval) goto out1; - retval = register_nmi_handler(NMI_UNKNOWN, kgdb_nmi_handler, - 0, "kgdb"); + retval = register_nmi_handler(NMI_UNKNOWN, kgdb_nmi_handler, 0, "kgdb", NMIS_NO_SOURCE); if (retval) goto out2; diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index a010e9d062bf..c4fffa868160 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -41,7 +41,7 @@ static void __init init_nmi_testsuite(void) { /* trap all the unknown NMIs we may generate */ register_nmi_handler(NMI_UNKNOWN, nmi_unk_cb, 0, "nmi_selftest_unk", - __initdata); + NMIS_NO_SOURCE, __initdata); } static void __init cleanup_nmi_testsuite(void) @@ -63,8 +63,8 @@ static void __init test_nmi_ipi(struct cpumask *mask) { unsigned long timeout; - if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, - NMI_FLAG_FIRST, "nmi_selftest", __initdata)) { + if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, NMI_FLAG_FIRST, + "nmi_selftest", NMIS_NO_SOURCE, __initdata)) { nmi_fail = FAILURE; return; } diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 18266cc3d98c..c9435730dfb0 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -142,8 +142,8 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_reboot) static int register_stop_handler(void) { - return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, - NMI_FLAG_FIRST, "smp_stop"); + return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, NMI_FLAG_FIRST, + "smp_stop", NMIS_NO_SOURCE); } static void native_stop_other_cpus(int wait) diff --git a/arch/x86/platform/uv/uv_nmi.c b/arch/x86/platform/uv/uv_nmi.c index 5c50e550ab63..5af368710f69 100644 --- a/arch/x86/platform/uv/uv_nmi.c +++ b/arch/x86/platform/uv/uv_nmi.c @@ -1029,10 +1029,10 @@ static int uv_handle_nmi_ping(unsigned int reason, struct pt_regs *regs) static void uv_register_nmi_notifier(void) { - if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv")) + if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv", NMIS_NO_SOURCE)) pr_warn("UV: NMI handler failed to register\n"); - if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping")) + if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping", NMIS_NO_SOURCE)) pr_warn("UV: PING NMI handler failed to register\n"); } diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index f0584ccad451..5e8632f34769 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -1421,7 +1421,7 @@ static void ghes_nmi_add(struct ghes *ghes) { mutex_lock(&ghes_list_mutex); if (list_empty(&ghes_nmi)) - register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes"); + register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes", NMIS_NO_SOURCE); list_add_rcu(&ghes->list, &ghes_nmi); mutex_unlock(&ghes_list_mutex); } diff --git a/drivers/char/ipmi/ipmi_watchdog.c b/drivers/char/ipmi/ipmi_watchdog.c index ab759b492fdd..944caef3dd31 100644 --- a/drivers/char/ipmi/ipmi_watchdog.c +++ b/drivers/char/ipmi/ipmi_watchdog.c @@ -1227,8 +1227,7 @@ static void check_parms(void) } } if (do_nmi && !nmi_handler_registered) { - rv = register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0, - "ipmi"); + rv = register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0, "ipmi", NMIS_NO_SOURCE); if (rv) { pr_warn("Can't register nmi handler\n"); return; diff --git a/drivers/edac/igen6_edac.c b/drivers/edac/igen6_edac.c index 1930dc00c791..9c7913e5bd3e 100644 --- a/drivers/edac/igen6_edac.c +++ b/drivers/edac/igen6_edac.c @@ -1419,8 +1419,7 @@ static int register_err_handler(void) return 0; } - rc = register_nmi_handler(NMI_SERR, ecclog_nmi_handler, - 0, IGEN6_NMI_NAME); + rc = register_nmi_handler(NMI_SERR, ecclog_nmi_handler, 0, IGEN6_NMI_NAME, NMIS_NO_SOURCE); if (rc) { igen6_printk(KERN_ERR, "Failed to register NMI handler\n"); return rc; diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c index ae30e394d176..90ae59a09270 100644 --- a/drivers/watchdog/hpwdt.c +++ b/drivers/watchdog/hpwdt.c @@ -242,13 +242,13 @@ static int hpwdt_init_nmi_decoding(struct pci_dev *dev) /* * Only one function can register for NMI_UNKNOWN */ - retval = register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt", NMIS_NO_SOURCE); if (retval) goto error; - retval = register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt", NMIS_NO_SOURCE); if (retval) goto error1; - retval = register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt"); + retval = register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt", NMIS_NO_SOURCE); if (retval) goto error2; From patchwork Thu Jun 12 21:48:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 895842 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D29E52E2EF7; Thu, 12 Jun 2025 21:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765034; cv=none; b=tTlrTy5vIgfR6v4QHcDDJmaCwhbmF8WuS6HlPLhHKkM7xa+3gJqbev1X3j8FahRaHJ0knT0fA5GbReVec/K90ATbEphm67bjOClVl34h1wvH4jHh+V2NAZr6MPNh1g/dUcVK+P1o/klJxxVyC4HdYlzofZJZDc5jiMgI0cCWk3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765034; c=relaxed/simple; bh=X5AazRLPkD4qnl4y51OKxsRAvMNIQ2UGhe8CaDOK3k0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IR/Jl09CGjCylqGmMpYHoq85YQNoPg+eXLXmEMmyWFRoJF6EZIH/vIwHYDUhv42Hr0Qb/f/1baytnID2dwC+l+ODaZQ5KbUW3MD3v2W+goOFv//PnnC0JT8tffHX5gXw9dfpWm4yC9akXci3CX1Ynt0bdGF95sI/gJ2FK64Qyl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U/+SALHr; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U/+SALHr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765033; x=1781301033; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X5AazRLPkD4qnl4y51OKxsRAvMNIQ2UGhe8CaDOK3k0=; b=U/+SALHr8S0F/3TC/RaiyQ/ILxn7nIm/IWES4ImDLCkpIiVsJDVw3rjK 1TK9TYzQlft89EPAG0d0sqSWMsUWMPVMrh0ZCborji7VRld0LfN/v/WBS swR0ByX0RntPpgHDddStiiSnD8hmKRAyK+EndgO4ZKjknZHkc6wojrsAA FXiAZofLOjJG9kV6kU62lf6hfl3bt0uxyY8sT39nwwwQKCN6YFvoMfYgI C1fS40Jytnh4+TlwdcPDsnNL5mR/y+eWAGv+zMQuki7FLVJ9qmO7jgRLi vmc6yiEHjU+6OwbIO86O7+DAX8ftChb+R4ev2rDKeRdAXUj8auE1IPxxT A==; X-CSE-ConnectionGUID: ToIOmlPoSB6sg6KlBxfpxA== X-CSE-MsgGUID: UqmTrZXASPmHzut3obxeMw== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065345" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065345" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:30 -0700 X-CSE-ConnectionGUID: fnbw2o0IQO6h44e7RuE5Kw== X-CSE-MsgGUID: 4sGjZK+ORt+IJPLUF4H1vw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881966" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:30 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 05/10] x86/nmi: Assign and register NMI-source vectors Date: Thu, 12 Jun 2025 14:48:44 -0700 Message-ID: <20250612214849.3950094-6-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Prior to NMI-source support, the vector information was ignored by the hardware while delivering NMIs. With NMI-source reporting, the initial architecture supports a 16-bit source bitmap to identify the source of the NMI. Upon receiving an NMI, this bitmap is delivered to the kernel as part of the FRED event delivery mechanism. Assign a vector space of 1-15 that is specific to NMI-source and independent of the IDT vector space. Though unlikely, the hardware may extend the NMI-source bitmap in the future. Add a code comment to clarify how the kernel support can be easily extended. Being a bitmap, the NMI-source vectors do not have any inherent priority associated with them. The order of executing the NMI handlers is up to the kernel. Existing NMI handling already has a priority mechanism for the NMI handlers, with CPU-specific (NMI_LOCAL) handlers executed first, followed by platform NMI handlers and unknown NMI (NMI_UNKNOWN) handlers being last. Within each of these NMI types, the handlers registered with NMI_FLAG_FIRST are given priority. NMI-source follows the same priority scheme to avoid unnecessary complexity. Therefore, the NMI-source vectors are assigned arbitrarily, except for vector 2. Vector 2 is reserved for external NMIs corresponding to the Local APIC - LINT1 pin. Some third-party chipsets may send NMI messages with a fixed vector value of 2. Using vector 2 for something else would lead to confusion about the exact source. Do not assign it to any handler. NMI-source vectors are only assigned for NMI_LOCAL type handlers. Platform NMI handlers have a single handler registered per type. They don't need additional source information to differentiate among them. Use the assigned vectors to register the respective NMI handlers. Warn if the vector values are unexpected. A couple of NMI handlers, such as the microcode rendezvous and the crash reboot, do not use the typical NMI registration interface. Leave them as-is for now. Originally-by: Jacob Pan Signed-off-by: Sohil Mehta Reviewed-by: Xin Li (Intel) --- v7: Use an enum to define the NMI-source vectors. (DaveH) Add a BUILD_BUG_ON to validate the allocated vector count. (DaveH) Add review tag from Xin. v6: Store source vector unconditionally. (PeterZ) Add a warning for unexpected source vector values. (PeterZ) v5: Move the vector defines to nmi.h. Combine vector allocation and registration into one patch. Simplify NMI vector names. Describe usage of vector 2 for external NMIs. Get rid of vector priorities. --- arch/x86/events/core.c | 2 +- arch/x86/include/asm/nmi.h | 46 ++++++++++++++++++++++++++++++++ arch/x86/kernel/apic/hw_nmi.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/kgdb.c | 2 +- arch/x86/kernel/nmi.c | 7 +++++ arch/x86/kernel/nmi_selftest.c | 2 +- arch/x86/kernel/smp.c | 2 +- 8 files changed, 59 insertions(+), 6 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index ec59837d10d1..dd42fe7bce9c 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2129,7 +2129,7 @@ static int __init init_hw_perf_events(void) x86_pmu.config_mask = X86_RAW_EVENT_MASK; perf_events_lapic_init(); - register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI", NMIS_NO_SOURCE); + register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI", NMIS_VECTOR_PMI); unconstrained = (struct event_constraint) __EVENT_CONSTRAINT(0, x86_pmu.cntr_mask64, diff --git a/arch/x86/include/asm/nmi.h b/arch/x86/include/asm/nmi.h index 42820c4f59b9..a48958a236fd 100644 --- a/arch/x86/include/asm/nmi.h +++ b/arch/x86/include/asm/nmi.h @@ -53,12 +53,58 @@ typedef int (*nmi_handler_t)(unsigned int, struct pt_regs *); * origin of an NMI and to route the NMI directly to the appropriate * handler. * + * On CPUs that support NMI-source reporting with FRED, receiving an NMI + * with a valid vector sets the corresponding bit in the NMI-source + * bitmap. The bitmap is delivered as FRED event data on the stack. + * + * Multiple NMIs are coalesced in the NMI-source bitmap until the next + * NMI delivery. If an NMI is received without a vector or beyond the + * defined range, the CPU sets bit 0 of the NMI-source bitmap. + * + * Third-party chipsets may send NMI messages with a fixed vector of 2. + * Using vector 2 for some other purpose would cause confusion between + * those external NMI messages and the other purpose. Avoid using it. + * + * The vectors are in no particular priority order. Add new vector + * assignments sequentially in the list below before the COUNT. + * * @NMIS_NO_SOURCE: Reserved for undefined or unidentified sources. + * @NMIS_VECTOR_TEST: NMI selftest. + * @NMIS_VECTOR_EXTERNAL: Reserved to match external NMI vector 2. + * @NMIS_VECTOR_SMP_STOP: Panic stop CPU. + * @NMIS_VECTOR_BT: CPU backtrace. + * @NMIS_VECTOR_KGDB: Kernel debugger. + * @NMIS_VECTOR_MCE: MCE injection. + * @NMIS_VECTOR_PMI: PerfMon counters. + * + * @NMIS_VECTOR_COUNT: Count of the defined vectors. */ enum nmi_source_vectors { NMIS_NO_SOURCE = 0, + NMIS_VECTOR_TEST, + NMIS_VECTOR_EXTERNAL = 2, + NMIS_VECTOR_SMP_STOP, + NMIS_VECTOR_BT, + NMIS_VECTOR_KGDB, + NMIS_VECTOR_MCE, + NMIS_VECTOR_PMI, + + NMIS_VECTOR_COUNT }; +/* + * The early (and likely all future) hardware implementations of + * NMI-source reporting would only support a 16-bit source bitmap, with + * 1-15 being valid source vectors. + * + * If the hardware ever supports a larger bitmap, the kernel support can + * easily be extended to 64 bits by modifying the MAX below. However, + * care must be taken to reallocate the latency sensitive NMI sources + * within the first 15 vectors. Any source vector beyond the supported + * maximum on prior systems would set bit 0 in the NMI-source bitmap. + */ +#define NMIS_VECTORS_MAX 16 + struct nmiaction { struct list_head list; nmi_handler_t handler; diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index d09e771723ed..4e04f13d2de9 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -53,7 +53,7 @@ NOKPROBE_SYMBOL(nmi_cpu_backtrace_handler); static int __init register_nmi_cpu_backtrace_handler(void) { - register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, 0, "arch_bt", NMIS_NO_SOURCE); + register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, 0, "arch_bt", NMIS_VECTOR_BT); return 0; } early_initcall(register_nmi_cpu_backtrace_handler); diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c index ba70ef8a1964..320068e01c22 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -775,7 +775,7 @@ static int __init inject_init(void) debugfs_init(); - register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify", NMIS_NO_SOURCE); + register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify", NMIS_VECTOR_MCE); mce_register_injector_chain(&inject_nb); setup_inj_struct(&i_mce); diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index ae28fdeefc7f..dfde434d2992 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -602,7 +602,7 @@ int kgdb_arch_init(void) if (retval) goto out; - retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, 0, "kgdb", NMIS_NO_SOURCE); + retval = register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, 0, "kgdb", NMIS_VECTOR_KGDB); if (retval) goto out1; diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index be93ec7255bf..8071ad32aa11 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -182,6 +182,13 @@ int __register_nmi_handler(unsigned int type, struct nmiaction *action) if (WARN_ON_ONCE(!action->handler || !list_empty(&action->list))) return -EINVAL; + /* NMI-source reporting should only be used for NMI_LOCAL */ + WARN_ON_ONCE((type != NMI_LOCAL) && (action->source_vector != NMIS_NO_SOURCE)); + + /* Check for valid vector values. See comment above NMIS_VECTORS_MAX */ + BUILD_BUG_ON(NMIS_VECTOR_COUNT > NMIS_VECTORS_MAX); + WARN_ON_ONCE(action->source_vector >= NMIS_VECTOR_COUNT); + raw_spin_lock_irqsave(&desc->lock, flags); /* diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index c4fffa868160..f3918888e494 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -64,7 +64,7 @@ static void __init test_nmi_ipi(struct cpumask *mask) unsigned long timeout; if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, NMI_FLAG_FIRST, - "nmi_selftest", NMIS_NO_SOURCE, __initdata)) { + "nmi_selftest", NMIS_VECTOR_TEST, __initdata)) { nmi_fail = FAILURE; return; } diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index c9435730dfb0..1ed065b78467 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -143,7 +143,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_reboot) static int register_stop_handler(void) { return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, NMI_FLAG_FIRST, - "smp_stop", NMIS_NO_SOURCE); + "smp_stop", NMIS_VECTOR_SMP_STOP); } static void native_stop_other_cpus(int wait) From patchwork Thu Jun 12 21:48:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 896144 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F8FF2E2EEC; Thu, 12 Jun 2025 21:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765034; cv=none; b=HIdh+VoLQl8w/jShVABUB7rdiv35aByfkHEZ2UvfMwDUKLJBsBDlPZHZnCAa/9QoBlqwNe5fqOBaXlIHzkbIHBrECqfj/GChEIrifPxcWaj0KWu56u38kgSlOXb3D5IC8F6i6m3g8MdtjnMogsSVwQBUwZyC0E617oYzomCyrqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765034; c=relaxed/simple; bh=v8IGRkYnJLZLDYhftOsNKqQqUFvuebbUgG7/IWeuerM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cHzBwOhWjj7WktkKsoqzSH8KLMQ3KirdmUj+wXglYnQ8+Z1pWGPDZ0dxE2Bsm3rX0CV0Ba/vy2IXGV5oaCIl25hYJ9I0zps2nRoTXLjtbuL52N1Xah/ADvTDxbDLXWATfDXrpSPNXlm4uiKK54n4S/oiUWrERSHcJArBRxXxcmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fAM7Zdky; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fAM7Zdky" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765033; x=1781301033; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v8IGRkYnJLZLDYhftOsNKqQqUFvuebbUgG7/IWeuerM=; b=fAM7ZdkyQNVrdJe7BGEJj9cUSASVh5ir8RwTsl5DbxvgEXKz8DGNTpJt 6qf7W57/BAoeGj2/qfHnRYyxLoA/cniZCoaI+lIJcKwHXPJs6GGhlrNzG eMPFvNf8cv8KwaCI12F4FMWaiYpzCVUWa+KQ0cvCwcQ+cVUegz98iMmul +fW7/iDuhRuv5cURLGo9H4pANkFPxdKneiTHYaJk0J+XpKhweeS39L7+q 6vrl+MuV2D1QRTGo3jgxlpWIeW8+MqYMitEcSnw4pn9Wze55fnBmg4bAs TJ+q/CLBntT9bM7pRbht6znivRsL9vwriDmrIuTIZ1eYWXIYEz586Yop7 w==; X-CSE-ConnectionGUID: qdfBvkzUSHKYhIivfnMhDw== X-CSE-MsgGUID: 0T6BYliTQiKRFxITCtTMXA== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065364" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065364" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:32 -0700 X-CSE-ConnectionGUID: 3HJwHsJBTNWKFrQsScjvfQ== X-CSE-MsgGUID: lETVjOBQRau+SbsplJ/Hfw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881973" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:31 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 06/10] x86/nmi: Add support to handle NMIs with source information Date: Thu, 12 Jun 2025 14:48:45 -0700 Message-ID: <20250612214849.3950094-7-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The NMI-source bitmap is delivered as FRED event data to the kernel. When available, use NMI-source based filtering to determine the exact handlers to run. Activate NMI-source based filtering only for Local NMIs. While handling platform NMI types (such as SERR and IOCHK), do not use the source bitmap. They have only one handler registered per type, so there is no need to disambiguate between multiple handlers. Some third-party chipsets may send NMI messages with a fixed vector of 2, which would result in bit 2 being set in the NMI-source bitmap. Skip the local NMI handlers in this situation. Bit 0 of the source bitmap is set by the hardware whenever a source vector was not used while generating an NMI, or the originator could not be reliably identified. Poll all the registered handlers in that case. When multiple handlers need to be executed, adhere to the existing priority scheme and execute the handlers registered with NMI_FLAG_FIRST before others. The logic for handling legacy NMIs is unaffected since the source bitmap would always have all bits set. Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Sohil Mehta Reviewed-by: Xin Li (Intel) --- v7: Add review tag from Xin. v6: Get rid of a separate NMI source matching function (PeterZ) Set source_bitmap to ULONG_MAX to match all sources by default v5: Significantly simplify NMI-source handling logic. Get rid of a separate lookup table for NMI-source vectors. Adhere to existing priority scheme for handling NMIs. --- arch/x86/kernel/nmi.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 8071ad32aa11..3d2b636e9379 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -130,6 +130,7 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration) static int nmi_handle(unsigned int type, struct pt_regs *regs) { struct nmi_desc *desc = nmi_to_desc(type); + unsigned long source_bitmap = ULONG_MAX; nmi_handler_t ehandler; struct nmiaction *a; int handled=0; @@ -148,16 +149,45 @@ static int nmi_handle(unsigned int type, struct pt_regs *regs) rcu_read_lock(); + /* + * Activate NMI source-based filtering only for Local NMIs. + * + * Platform NMI types (such as SERR and IOCHK) have only one + * handler registered per type, so there is no need to + * disambiguate between multiple handlers. + * + * Also, if a platform source ends up setting bit 2 in the + * source bitmap, the local NMI handlers would be skipped since + * none of them use this reserved vector. + * + * For Unknown NMIs, avoid using the source bitmap to ensure all + * potential handlers have a chance to claim responsibility. + */ + if (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE) && type == NMI_LOCAL) { + source_bitmap = fred_event_data(regs); + + /* Reset the bitmap if a valid source could not be identified */ + if (WARN_ON_ONCE(!source_bitmap) || (source_bitmap & BIT(NMIS_NO_SOURCE))) + source_bitmap = ULONG_MAX; + } + /* * NMIs are edge-triggered, which means if you have enough * of them concurrently, you can lose some because only one * can be latched at any given time. Walk the whole list * to handle those situations. + * + * However, NMI-source reporting does not have this limitation. + * When NMI sources have been identified, only run the handlers + * that match the reported vectors. */ list_for_each_entry_rcu(a, &desc->head, list) { int thishandled; u64 delta; + if (!(source_bitmap & BIT(a->source_vector))) + continue; + delta = sched_clock(); thishandled = a->handler(type, regs); handled += thishandled; From patchwork Thu Jun 12 21:48:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 895841 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A12992E3385; Thu, 12 Jun 2025 21:50:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765036; cv=none; b=ad6j2AiqlbvTiuhZEKDrMAvqamy4H9PsdXUeysf/v3aES2XMHYPYOWdYehVStF2Rb2DTKxsVuBZDd8YAkhc7LJ36zJ/bmMWjMfW7WHq1WCjb0nVe1nk1PG61ey8VTJTHi/00nFQpavvIVeR4F8BkC8jlW6qVm6UnlVAKIl6e+4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765036; c=relaxed/simple; bh=A1gwyln+xuisMeyo3d8GVwsGZcGvqUSnNcXq3phIvbc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GERbsfmccEApbbjLRWjnGpNVhjPnOqQyyOO0ifpDng0VZF12xU3uaC0eK35wvLAgqX6o0XWgSRmT3kVlhj/POGNT/6rOs1xSkQ/1tOapCHG51XAFvOx5jE9guQTu9e5mVoLXnkZdRO5ZiskcRIYN5JliYmcTdprhZWJ8PKRDDRc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gxKuidHU; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gxKuidHU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765035; x=1781301035; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A1gwyln+xuisMeyo3d8GVwsGZcGvqUSnNcXq3phIvbc=; b=gxKuidHUdOTSfMz2S65w/oXMYlLb5HdRqt9m9zmZ9SC23VcA5AVNU00R I2DSiZHKP9fBtLd6teerkVpFXhTbDTXWfLggApXI+Gt1upH0wnp5WWcq1 eSAAM3eW8v1c7j1qYFN9NJfqZ+TPvBAdaXPdwDytbWgL9YLjdqYL8hQly AzAUOPBiKAkzR4qdpNlMuhOjjJQjfHbe+VmPmL6BvqxsuKbIfBWCo6laY qr5+DlrxHE0lHsIxebyikq51q/RXwSifTxAbVH1RiwCByKYMcCns015Ko 8PUd6kPty60sNivzxtVYmWH0zIWRXPmx+dQfnKp5xM+J7rKnz3L46DRjs g==; X-CSE-ConnectionGUID: 2z/2VB+uQ4yCeNFaSD2PAQ== X-CSE-MsgGUID: VK81YY4hQg+zDb/vS0K3sw== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065383" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065383" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:33 -0700 X-CSE-ConnectionGUID: Zr0PGwJFTbWaJrG7wtXcPQ== X-CSE-MsgGUID: +oYlPMCrQuynSuqf0d5mLg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881984" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:32 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 07/10] x86/nmi: Prepare for the new NMI-source vector encoding Date: Thu, 12 Jun 2025 14:48:46 -0700 Message-ID: <20250612214849.3950094-8-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When using the send_IPI_* APIC calls, callers typically use NMI vector 0x2 to trigger NMIs. The APIC APIs convert the NMI vector to the NMI delivery mode, which is eventually used to program the APIC. Before FRED, the hardware would ignore the vector used with NMI delivery mode. However, with NMI-source reporting, the vector information is relayed to the destination CPU, which sets the corresponding bit in the NMI-source bitmap. Unfortunately, the kernel now needs to maintain a new set of NMI vectors and differentiate them from the IDT vectors. Instead of creating a parallel set of send_NMI_* APIs to handle NMI-source vectors, enhance the existing send_IPI_* APIs with a new encoding scheme to handle the NMI delivery mode along with the NMI-source vector. NMI-source vectors would be encoded as: APIC_DM_NMI (0x400) | NMI_SOURCE_VECTOR (0x1-0xF) Also, introduce a helper to prepare the ICR value with the encoded delivery mode and vector. Update the guest paravirtual APIC code to use the new helper as well. While at it, rename APIC_DM_FIXED_MASK to the more appropriate APIC_DM_MASK. Suggested-by: Sean Christopherson Co-developed-by: Xin Li (Intel) Signed-off-by: Xin Li (Intel) Signed-off-by: Sohil Mehta --- v7: No change. v6: Remove a redundant else statement. (PeterZ) v5: Use a simiplified encoding scheme for NMI-source vectors. --- arch/x86/include/asm/apic.h | 30 +++++++++++++++++++++++++++++ arch/x86/include/asm/apicdef.h | 2 +- arch/x86/kernel/apic/ipi.c | 4 ++-- arch/x86/kernel/apic/local.h | 24 ++++++++++++----------- arch/x86/kernel/kvm.c | 9 +-------- drivers/thermal/intel/therm_throt.c | 2 +- 6 files changed, 48 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 23d86c9750b9..32cdd81e5e45 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -470,6 +470,36 @@ static __always_inline bool apic_id_valid(u32 apic_id) return apic_id <= apic->max_apic_id; } +/* + * Prepare the delivery mode and vector for the ICR. + * + * NMI-source vectors have the NMI delivery mode encoded within them to + * differentiate them from the IDT vectors. IDT vector 0x2 (NMI_VECTOR) + * is treated as an NMI request but without any NMI-source information. + */ +static inline u16 __prepare_ICR_DM_vector(u16 dm_vector) +{ + u16 vector = dm_vector & APIC_VECTOR_MASK; + u16 dm = dm_vector & APIC_DM_MASK; + + if (dm == APIC_DM_NMI) { + /* + * Pre-FRED, the actual vector is ignored for NMIs, but + * zero it if NMI-source reporting is not supported to + * avoid breakage on misbehaving hardware or hypervisors. + */ + if (!cpu_feature_enabled(X86_FEATURE_NMI_SOURCE)) + vector = 0; + + return dm | vector; + } + + if (vector == NMI_VECTOR) + return APIC_DM_NMI; + + return APIC_DM_FIXED | vector; +} + #else /* CONFIG_X86_LOCAL_APIC */ static inline u32 apic_read(u32 reg) { return 0; } diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h index 094106b6a538..3fb8fa73f6aa 100644 --- a/arch/x86/include/asm/apicdef.h +++ b/arch/x86/include/asm/apicdef.h @@ -87,8 +87,8 @@ #define APIC_ICR_BUSY 0x01000 #define APIC_DEST_LOGICAL 0x00800 #define APIC_DEST_PHYSICAL 0x00000 +#define APIC_DM_MASK 0x00700 #define APIC_DM_FIXED 0x00000 -#define APIC_DM_FIXED_MASK 0x00700 #define APIC_DM_LOWEST 0x00100 #define APIC_DM_SMI 0x00200 #define APIC_DM_REMRD 0x00300 diff --git a/arch/x86/kernel/apic/ipi.c b/arch/x86/kernel/apic/ipi.c index 98a57cb4aa86..4e8bc42f3bd5 100644 --- a/arch/x86/kernel/apic/ipi.c +++ b/arch/x86/kernel/apic/ipi.c @@ -158,7 +158,7 @@ static void __default_send_IPI_shortcut(unsigned int shortcut, int vector) * issues where otherwise the system hangs when the panic CPU tries * to stop the others before launching the kdump kernel. */ - if (unlikely(vector == NMI_VECTOR)) + if (unlikely(is_nmi_vector(vector))) apic_mem_wait_icr_idle_timeout(); else apic_mem_wait_icr_idle(); @@ -175,7 +175,7 @@ void __default_send_IPI_dest_field(unsigned int dest_mask, int vector, unsigned int dest_mode) { /* See comment in __default_send_IPI_shortcut() */ - if (unlikely(vector == NMI_VECTOR)) + if (unlikely(is_nmi_vector(vector))) apic_mem_wait_icr_idle_timeout(); else apic_mem_wait_icr_idle(); diff --git a/arch/x86/kernel/apic/local.h b/arch/x86/kernel/apic/local.h index bdcf609eb283..9a54c589a4bf 100644 --- a/arch/x86/kernel/apic/local.h +++ b/arch/x86/kernel/apic/local.h @@ -24,22 +24,24 @@ extern u32 x2apic_max_apicid; /* IPI */ +u16 __prepare_ICR_DM_vector(u16 vector); + DECLARE_STATIC_KEY_FALSE(apic_use_ipi_shorthand); +/* NMI-source vectors have the delivery mode encoded within them */ +static inline bool is_nmi_vector(u16 vector) +{ + if ((vector & APIC_DM_MASK) == APIC_DM_NMI) + return true; + if ((vector & APIC_VECTOR_MASK) == NMI_VECTOR) + return true; + return false; +} + static inline unsigned int __prepare_ICR(unsigned int shortcut, int vector, unsigned int dest) { - unsigned int icr = shortcut | dest; - - switch (vector) { - default: - icr |= APIC_DM_FIXED | vector; - break; - case NMI_VECTOR: - icr |= APIC_DM_NMI; - break; - } - return icr; + return shortcut | dest | __prepare_ICR_DM_vector(vector); } void default_init_apic_ldr(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 921c1c783bc1..317d585ff3d0 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -518,14 +518,7 @@ static void __send_ipi_mask(const struct cpumask *mask, int vector) local_irq_save(flags); - switch (vector) { - default: - icr = APIC_DM_FIXED | vector; - break; - case NMI_VECTOR: - icr = APIC_DM_NMI; - break; - } + icr = __prepare_ICR_DM_vector(vector); for_each_cpu(cpu, mask) { apic_id = per_cpu(x86_cpu_to_apicid, cpu); diff --git a/drivers/thermal/intel/therm_throt.c b/drivers/thermal/intel/therm_throt.c index debc94e2dc16..5c0d2de2986e 100644 --- a/drivers/thermal/intel/therm_throt.c +++ b/drivers/thermal/intel/therm_throt.c @@ -740,7 +740,7 @@ void intel_init_thermal(struct cpuinfo_x86 *c) * BIOS has programmed on AP based on BSP's info we saved since BIOS * is always setting the same value for all threads/cores. */ - if ((h & APIC_DM_FIXED_MASK) != APIC_DM_FIXED) + if ((h & APIC_DM_MASK) != APIC_DM_FIXED) apic_write(APIC_LVTTHMR, lvtthmr_init); From patchwork Thu Jun 12 21:48:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 896143 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E5A32D8765; Thu, 12 Jun 2025 21:50:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765036; cv=none; b=LPT41nXakwqOaOzlDSUdVn81JmNH3nTbCXgqU8K48dR/55j0dgcJuO3qMSoLeJck9rD4cGuHey0ExnyEt/XtiqN8UFZzT59SW+04GBxe7W2QwVwcL/LjElijDTKhk7aKXj+p7h2MWhfsHtAQvNGokpeQJq7iEyRAT1TlYCHSVA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765036; c=relaxed/simple; bh=wx+UnzyhcoYcIjxMnfjwt2d75sp1lBjQuwAGslQWWB0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mx+AqiUJWCwBhtiPDSdmvV+ROW/OU9Rc8WlymYqnVh6zeEimfzuOJHj3mD2KGD/jAVufmuB1HQ1xbhRIa3nQF1vbAHuYaKH+58WDIyKrf1IuzSr2aQc4KuQ6hCz96nD/hQZSNpX793YAuQCNq68F9E8OwxwyL89x4MANFUzkmN0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LRqFgLXk; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LRqFgLXk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765035; x=1781301035; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wx+UnzyhcoYcIjxMnfjwt2d75sp1lBjQuwAGslQWWB0=; b=LRqFgLXkXDovoV84UmzWezIE77sG3NEOUGu6sBU9UzJNV3yJn8SAn+nG +VrYyonKxHGijDGaoa7t6YxkPakmXlSmn6WiX8+imEhlqbdMxMKQLCXS7 tLBiE5Q3X0TpLlUSjXv7z6i7LbsaOm5bjJ2lj14d8xp5NjR/NZ0Cp8H3s shmBXa/ZdPHvbljmwV0C4nSyws+gqEaYSGXWFbgcrmk5Nklr1aN5c/ZSF 481wSNj2A0rvmzqO8uOp1O3vjrQVnmibaYDUwch4arR/CX9UdknInszYx 81aXgyaFkZy2TClI8iSCDQwwICElNDji3Rr1VGD+g+CGQtsr/2oGsc3zu g==; X-CSE-ConnectionGUID: X8UxNGyeQ32NjAtMzzV6Pw== X-CSE-MsgGUID: YJpUpASxTRa+moGGWIyB6w== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065386" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065386" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:34 -0700 X-CSE-ConnectionGUID: th5cflvlQBaz4G40fNcT3g== X-CSE-MsgGUID: hkSE3mUdRl6F+sj6KPmmxw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881990" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:33 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 08/10] x86/nmi: Enable NMI-source for IPIs delivered as NMIs Date: Thu, 12 Jun 2025 14:48:47 -0700 Message-ID: <20250612214849.3950094-9-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With the IPI handling APIs ready to support the new NMI encoding, encode the NMI delivery mode directly with the NMI-source vectors to trigger NMIs. Move most of the existing NMI-based IPIs to use the new NMI-source vectors, except for the microcode rendezvous NMI and the crash reboot NMI. NMI handling for them is special-cased in exc_nmi() and does not need NMI-source reporting. However, in the future, it might be useful to assign a source vector to all NMI sources to improve isolation and debuggability. Originally-by: Jacob Pan Suggested-by: Sean Christopherson Co-developed-by: Xin Li (Intel) Signed-off-by: Xin Li (Intel) Signed-off-by: Sohil Mehta --- v7: No change. v6: Include asm/nmi.h to avoid compile errors. (LKP) v5: Encode APIC_DM_NMI directly with the NMI-source vector. --- arch/x86/include/asm/apic.h | 8 ++++++++ arch/x86/kernel/apic/hw_nmi.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/kgdb.c | 2 +- arch/x86/kernel/nmi_selftest.c | 2 +- arch/x86/kernel/smp.c | 2 +- 6 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 32cdd81e5e45..5789df1708bd 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #define ARCH_APICTIMER_STOPS_ON_C3 1 @@ -23,6 +24,13 @@ #define APIC_EXTNMI_ALL 1 #define APIC_EXTNMI_NONE 2 +/* Trigger NMIs with source information */ +#define TEST_NMI (APIC_DM_NMI | NMIS_VECTOR_TEST) +#define SMP_STOP_NMI (APIC_DM_NMI | NMIS_VECTOR_SMP_STOP) +#define BT_NMI (APIC_DM_NMI | NMIS_VECTOR_BT) +#define KGDB_NMI (APIC_DM_NMI | NMIS_VECTOR_KGDB) +#define MCE_NMI (APIC_DM_NMI | NMIS_VECTOR_MCE) + /* * Debugging macros */ diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 4e04f13d2de9..586f4b25feae 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -33,7 +33,7 @@ u64 hw_nmi_get_sample_period(int watchdog_thresh) #ifdef arch_trigger_cpumask_backtrace static void nmi_raise_cpu_backtrace(cpumask_t *mask) { - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, BT_NMI); } void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inject.c index 320068e01c22..81a04836ac74 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -270,7 +270,7 @@ static void __maybe_unused raise_mce(struct mce *m) mce_irq_ipi, NULL, 0); preempt_enable(); } else if (m->inject_flags & MCJ_NMI_BROADCAST) - __apic_send_IPI_mask(mce_inject_cpumask, NMI_VECTOR); + __apic_send_IPI_mask(mce_inject_cpumask, MCE_NMI); } start = jiffies; while (!cpumask_empty(mce_inject_cpumask)) { diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index dfde434d2992..013feeb6a9ef 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -416,7 +416,7 @@ static void kgdb_disable_hw_debug(struct pt_regs *regs) */ void kgdb_roundup_cpus(void) { - apic_send_IPI_allbutself(NMI_VECTOR); + apic_send_IPI_allbutself(KGDB_NMI); } #endif diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index f3918888e494..d5578370b47f 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -72,7 +72,7 @@ static void __init test_nmi_ipi(struct cpumask *mask) /* sync above data before sending NMI */ wmb(); - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, TEST_NMI); /* Don't wait longer than a second */ timeout = USEC_PER_SEC; diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 1ed065b78467..239e9bf97abb 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -217,7 +217,7 @@ static void native_stop_other_cpus(int wait) pr_emerg("Shutting down cpus with NMI\n"); for_each_cpu(cpu, &cpus_stop_mask) - __apic_send_IPI(cpu, NMI_VECTOR); + __apic_send_IPI(cpu, SMP_STOP_NMI); } /* * Don't wait longer than 10 ms if the caller didn't From patchwork Thu Jun 12 21:48:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 896142 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 525672E612D; Thu, 12 Jun 2025 21:50:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765037; cv=none; b=VSrAz2glgFlHipXvTLNjz7Yu1YEOoBLsLRzG9vp0iLRpSrOL00Msue2Hax4ADfcjVKMomKbZf3fCRpG6B0wAF+35+4K6H4nJXQy+y8IhHcFrIsZHo5arwodmVYt6ueYq7qsC4JK20abBoNDdxz9+XBgZqUHsgrH1XTcvUGuXU94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765037; c=relaxed/simple; bh=0yJO8+t59kyXCqaGy07bQzCzjDip1tbJbjX7B+1zjMs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UXo4mMtoz6t4/mBh24VJqV8TtFHKhgZOJkQb4lOyJIK3BCvfpyhw+VwMZFw2pXvUwRXprZM4rpdIjPeUNBHkQCndyxdZOzlavHTYAl5qaSowDOAbDjV2SZcvV56DxFIO5DkYzWT5CZ0Wi4PRIdEmkjqJCI5D1pSl3ARWJNqLWVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gwQkh9QX; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gwQkh9QX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765035; x=1781301035; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0yJO8+t59kyXCqaGy07bQzCzjDip1tbJbjX7B+1zjMs=; b=gwQkh9QXF5veqFEGK+k3JyRi1l+0f8UqyiWXk5b7BwQhGKipIeYRl5wn ua5g6zqp9q2s1aN/2uMSiDqFwvE/pkLW2RZYxPYpJh7fx2iJn/NaErbmO s9YuzVRZL7M/ISU9rxxeo2SGJhTKGnzM8tJNd6AuxBMLFEtANxecahgEt dj0JDBSMO7YbLpArn0tWAwnuFoFZzp10OvLy2dIP6uoScxFF3ZrN0H96M iKgVVUvw0Qkha8wX83BQVsrL/RGAuDm7aIs9UgYVQf+WBfMnfkJS0J0UE IobNbMxSWU2hp9u0zTlg3qZb5NzQgtTPkzf1qfvBPNO3ifyPRY+95ZYDP Q==; X-CSE-ConnectionGUID: LwkZftrJR3aNuUUjdotWng== X-CSE-MsgGUID: R1ElUbUHSrOnaTWAydoLgQ== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065400" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065400" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:35 -0700 X-CSE-ConnectionGUID: Icu8J/9VSGWuSuzQVlWxaQ== X-CSE-MsgGUID: SFH2AFFXSDyjnoZ73RK4QQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184881999" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:34 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 09/10] perf/x86: Enable NMI-source reporting for perfmon Date: Thu, 12 Jun 2025 14:48:48 -0700 Message-ID: <20250612214849.3950094-10-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jacob Pan Program the designated PMI NMI-source vector into the local vector table for the PMU. An NMI for the PMU would directly invoke the PMI handler without polling other NMI handlers, resulting in reduced PMI delivery latency. Co-developed-by: Zeng Guang Signed-off-by: Zeng Guang Signed-off-by: Jacob Pan Signed-off-by: Sohil Mehta Tested-by: Sandipan Das # AMD overlapping bits Reviewed-by: Kan Liang Reviewed-by: Xin Li (Intel) --- v7: Pick up a review tag (Xin). v6: Pick up a tested-by tag (Sandipan). v5: No significant change. --- arch/x86/events/core.c | 4 ++-- arch/x86/events/intel/core.c | 6 +++--- arch/x86/include/asm/apic.h | 1 + 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index dd42fe7bce9c..3336609288b0 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -1704,7 +1704,7 @@ int x86_pmu_handle_irq(struct pt_regs *regs) * This generic handler doesn't seem to have any issues where the * unmasking occurs so it was left at the top. */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); for_each_set_bit(idx, x86_pmu.cntr_mask, X86_PMC_IDX_MAX) { if (!test_bit(idx, cpuc->active_mask)) @@ -1746,7 +1746,7 @@ void perf_events_lapic_init(void) /* * Always use NMI for PMU */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); } static int diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 741b229f0718..000d3ab72bd2 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3318,7 +3318,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * NMI handler. */ if (!late_ack && !mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); intel_bts_disable_local(); cpuc->enabled = 0; __intel_pmu_disable_all(true); @@ -3355,7 +3355,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) done: if (mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); /* Only restore PMU state when it's active. See x86_pmu_disable(). */ cpuc->enabled = pmu_enabled; if (pmu_enabled) @@ -3368,7 +3368,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * Haswell CPUs. */ if (late_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, PERF_NMI); return handled; } diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 5789df1708bd..7287005f05a6 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -30,6 +30,7 @@ #define BT_NMI (APIC_DM_NMI | NMIS_VECTOR_BT) #define KGDB_NMI (APIC_DM_NMI | NMIS_VECTOR_KGDB) #define MCE_NMI (APIC_DM_NMI | NMIS_VECTOR_MCE) +#define PERF_NMI (APIC_DM_NMI | NMIS_VECTOR_PMI) /* * Debugging macros From patchwork Thu Jun 12 21:48:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sohil Mehta X-Patchwork-Id: 895840 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CB432E62DA; Thu, 12 Jun 2025 21:50:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765038; cv=none; b=tKBssGT8Q/Pt5KJz04pJcGHk2zHkcsCCU88tzKHmxeRfvGy4UD3zOtjXA6ezHUSs6ruOpJYr1UXGlYlb0X80F4/lBzGX7v6BpI3IJWxVC5RSWHTk4stkFJ7VUymZJQ12IyyiHSnh2fZzuVVeFx70HSBSrsVfAsNxwDMkDKvkhUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749765038; c=relaxed/simple; bh=HAJ/WUjo+26I25B8LZv9C58T13MdBKPP51Ed+aO6Hzc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ASztzlx9wtE+vt0p8j88jlfD7qYz2/oIb8aP+XlrKWmX5//IEl2m5rDtpaQXkEgwyTd6J45eGhFZcGTaOzNW+alCOLTH9+E8DgdY6obTpgh1rPd3I9G+yYJu+pOwZy16YNwX1hFxDMOBaOAgp4dONqcigTclV7FmtXRNqoATen0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LmtUbBWU; arc=none smtp.client-ip=192.198.163.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LmtUbBWU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1749765037; x=1781301037; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HAJ/WUjo+26I25B8LZv9C58T13MdBKPP51Ed+aO6Hzc=; b=LmtUbBWUPASfVQNSGYgP+FS0eqPi7EmSC6aDmkNtLrGtUBfNpSWxhM/B weos++PYbkhgJP78VLIlZhPK98vKpdFfkszpBGy/n/cha6jYvM94WBUOz 013cZm/7Dh9OhNMGAG5MyJllBuB/3gJCYU2GNG0LpAw3LlkYUaTuGx1On FgpSgYsSOu5/cjMk5yIiV2yobCgCvZxe709zlbC1EUNnN8aX2HuE3Vcjj +4Cjb6w55X2QXQ5EqycIOqzCO+sEP9l6rFSXg4BXU7U9z+SIsvwIILvSS vEWPw6vP1uEdt1DCBdAJB/NHL7MOgUzHVvwjdHbnAAGb1VWpuF3tHumvH Q==; X-CSE-ConnectionGUID: mpk+GHciSIaQcooH6X16jQ== X-CSE-MsgGUID: 9DKIFj24S7GMyR2vsMxIuA== X-IronPort-AV: E=McAfee;i="6800,10657,11462"; a="52065412" X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="52065412" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2025 14:50:36 -0700 X-CSE-ConnectionGUID: 6shFoTgzT5Oy0swil4gfEA== X-CSE-MsgGUID: YGQBP4BrTL+w7Bg/am6Chw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,231,1744095600"; d="scan'208";a="184882009" Received: from sohilmeh.sc.intel.com ([172.25.103.65]) by orviesa001.jf.intel.com with ESMTP; 12 Jun 2025 14:50:35 -0700 From: Sohil Mehta To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Xin Li , "H . Peter Anvin" , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Sean Christopherson , Adrian Hunter , Kan Liang , Tony Luck , Zhang Rui , Steven Rostedt , Sohil Mehta , Andrew Cooper , "Kirill A . Shutemov" , Jacob Pan , Andi Kleen , Kai Huang , Sandipan Das , linux-perf-users@vger.kernel.org, linux-edac@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH v7 10/10] x86/nmi: Print source information with the unknown NMI console message Date: Thu, 12 Jun 2025 14:48:49 -0700 Message-ID: <20250612214849.3950094-11-sohil.mehta@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250612214849.3950094-1-sohil.mehta@intel.com> References: <20250612214849.3950094-1-sohil.mehta@intel.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The NMI-source bitmap is a useful piece of information provided by the NMI-source reporting feature. It is very helpful for debugging unknown NMIs, as it can pinpoint the exact source that caused the NMI. Print the complete source bitmap along with the "unknown NMI" kernel log message, since unexpected sources might have triggered the NMI. Signed-off-by: Sohil Mehta --- v7: No change. v6: Drop the tracepoint modification part for now. v5: New patch --- arch/x86/kernel/nmi.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 3d2b636e9379..0b5bb20c5eb7 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -379,6 +379,9 @@ unknown_nmi_error(unsigned char reason, struct pt_regs *regs) pr_emerg_ratelimited("Uhhuh. NMI received for unknown reason %02x on CPU %d.\n", reason, smp_processor_id()); + if (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE)) + pr_emerg_ratelimited("NMI-source bitmap is 0x%lx\n", fred_event_data(regs)); + if (unknown_nmi_panic || panic_on_unrecovered_nmi) nmi_panic(regs, "NMI: Not continuing");