From patchwork Fri Jun 19 14:33:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 223818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C781C433E0 for ; Fri, 19 Jun 2020 16:19:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BAA3217A0 for ; Fri, 19 Jun 2020 16:19:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592583582; bh=I4j9cgFKmU/e3A06B8j3e0Dn3D6aWeZNdZzrQfrzmSU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=h8Ui8udzalt+5/AFOd7iXoRIWKAZ8uKKhVDTR+hfJCR9vzM45VxfSPmnGQk8vJe3P ITmNKaUs5NVA9Y7YkqSIfbSk/GUYXRZ/S/lznfs4QO+LcgLBZ6fjVk3Q/GL0Bybjt5 XC2JCiYOIUn/aRiYzn6jfhcfJpB9fLqKZjAPgs8M= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390746AbgFSQTk (ORCPT ); Fri, 19 Jun 2020 12:19:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:58286 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390742AbgFSPBv (ORCPT ); Fri, 19 Jun 2020 11:01:51 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5520F21841; Fri, 19 Jun 2020 15:01:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592578911; bh=I4j9cgFKmU/e3A06B8j3e0Dn3D6aWeZNdZzrQfrzmSU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mIwTmoAjg98Ci/46JyAwVjF/z6v95g782EsyFhQFt97csPIyxVcGQOiZl7do4GTuJ wW5OaPO9n0y6Bp1JtKIMqwFlx0WQiLdD54BtrR4Z0aEs0PXCxIZFkWkd1n/8OA/sVh 6B2etcYYPGHz8uSaYUBzUsAIqfNPxvipSf/7kt44= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Darrel Goeddel , Ashok Raj , Bjorn Helgaas , Alex Williamson , Lu Baolu , Romil Sharma , Sasha Levin , Mark Scott Subject: [PATCH 4.19 203/267] PCI: Add ACS quirk for Intel Root Complex Integrated Endpoints Date: Fri, 19 Jun 2020 16:33:08 +0200 Message-Id: <20200619141658.474621270@linuxfoundation.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200619141648.840376470@linuxfoundation.org> References: <20200619141648.840376470@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Ashok Raj [ Upstream commit 3247bd10a4502a3075ce8e1c3c7d31ef76f193ce ] All Intel platforms guarantee that all root complex implementations must send transactions up to IOMMU for address translations. Hence for Intel RCiEP devices, we can assume some ACS-type isolation even without an ACS capability. >From the Intel VT-d spec, r3.1, sec 3.16 ("Root-Complex Peer to Peer Considerations"): When DMA remapping is enabled, peer-to-peer requests through the Root-Complex must be handled as follows: - The input address in the request is translated (through first-level, second-level or nested translation) to a host physical address (HPA). The address decoding for peer addresses must be done only on the translated HPA. Hardware implementations are free to further limit peer-to-peer accesses to specific host physical address regions (or to completely disallow peer-forwarding of translated requests). - Since address translation changes the contents (address field) of the PCI Express Transaction Layer Packet (TLP), for PCI Express peer-to-peer requests with ECRC, the Root-Complex hardware must use the new ECRC (re-computed with the translated address) if it decides to forward the TLP as a peer request. - Root-ports, and multi-function root-complex integrated endpoints, may support additional peer-to-peer control features by supporting PCI Express Access Control Services (ACS) capability. Refer to ACS capability in PCI Express specifications for details. Since Linux didn't give special treatment to allow this exception, certain RCiEP MFD devices were grouped in a single IOMMU group. This doesn't permit a single device to be assigned to a guest for instance. In one vendor system: Device 14.x were grouped in a single IOMMU group. /sys/kernel/iommu_groups/5/devices/0000:00:14.0 /sys/kernel/iommu_groups/5/devices/0000:00:14.2 /sys/kernel/iommu_groups/5/devices/0000:00:14.3 After this patch: /sys/kernel/iommu_groups/5/devices/0000:00:14.0 /sys/kernel/iommu_groups/5/devices/0000:00:14.2 /sys/kernel/iommu_groups/6/devices/0000:00:14.3 <<< new group 14.0 and 14.2 are integrated devices, but legacy end points, whereas 14.3 was a PCIe-compliant RCiEP. 00:14.3 Network controller: Intel Corporation Device 9df0 (rev 30) Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00 This permits assigning this device to a guest VM. [bhelgaas: drop "Fixes" tag since this doesn't fix a bug in that commit] Link: https://lore.kernel.org/r/1590699462-7131-1-git-send-email-ashok.raj@intel.com Tested-by: Darrel Goeddel Signed-off-by: Ashok Raj Signed-off-by: Bjorn Helgaas Reviewed-by: Alex Williamson Cc: stable@vger.kernel.org Cc: Lu Baolu Cc: Mark Scott , Cc: Romil Sharma Signed-off-by: Sasha Levin --- drivers/pci/quirks.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 013b84880e1d..d6236bb26950 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4543,6 +4543,20 @@ static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags) return acs_flags ? 0 : 1; } +static int pci_quirk_rciep_acs(struct pci_dev *dev, u16 acs_flags) +{ + /* + * Intel RCiEP's are required to allow p2p only on translated + * addresses. Refer to Intel VT-d specification, r3.1, sec 3.16, + * "Root-Complex Peer to Peer Considerations". + */ + if (pci_pcie_type(dev) != PCI_EXP_TYPE_RC_END) + return -ENOTTY; + + return pci_acs_ctrl_enabled(acs_flags, + PCI_ACS_SV | PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_UF); +} + static int pci_quirk_brcm_acs(struct pci_dev *dev, u16 acs_flags) { /* @@ -4626,6 +4640,7 @@ static const struct pci_dev_acs_enabled { /* I219 */ { PCI_VENDOR_ID_INTEL, 0x15b7, pci_quirk_mf_endpoint_acs }, { PCI_VENDOR_ID_INTEL, 0x15b8, pci_quirk_mf_endpoint_acs }, + { PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_rciep_acs }, /* QCOM QDF2xxx root ports */ { PCI_VENDOR_ID_QCOM, 0x0400, pci_quirk_qcom_rp_acs }, { PCI_VENDOR_ID_QCOM, 0x0401, pci_quirk_qcom_rp_acs },