From patchwork Tue Jan 10 11:57:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksey Makarov X-Patchwork-Id: 90683 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp578493qgi; Tue, 10 Jan 2017 03:58:18 -0800 (PST) X-Received: by 10.98.78.66 with SMTP id c63mr3363317pfb.138.1484049498249; Tue, 10 Jan 2017 03:58:18 -0800 (PST) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v68si1936554pgv.129.2017.01.10.03.58.17; Tue, 10 Jan 2017 03:58:18 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S967021AbdAJL6P (ORCPT + 25 others); Tue, 10 Jan 2017 06:58:15 -0500 Received: from mail-wm0-f48.google.com ([74.125.82.48]:38577 "EHLO mail-wm0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755206AbdAJL6N (ORCPT ); Tue, 10 Jan 2017 06:58:13 -0500 Received: by mail-wm0-f48.google.com with SMTP id k184so157909508wme.1 for ; Tue, 10 Jan 2017 03:58:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=iUPWVGM6xLKVW4ZZsX9AJqlzHUGfXnP/fstM1A2Bmkc=; b=NzxrJNaYdUNSyk3t++qeCx3I42HRP3moqBnC3jE1QNz3GjWUVNmWlpMMH1vfswAhv0 O/E/5ldbYfcoeyDhtGhEWQ+y2lE9rpK0sRriXidrdrDvtLfY0GRX4k1BBF2+jpW3ZZbz e3RXdrt3DzcO61pJfRDUZknAgzUfVwd6IK7qo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=iUPWVGM6xLKVW4ZZsX9AJqlzHUGfXnP/fstM1A2Bmkc=; b=Mwk+rEGaPNpd4XwAz3PXPu3y5yHRp1YyZSKM7gXO/E+104nj8MYztKjNXsCpdzqhXi yCdRpg0IIglLylQv+fLIyb8OVhQG4m4DXWuBBiqk4AjGytROfQVUC09YAaap6ILh8qp/ gt1xK+tIgSTuryD59fi/e/8T1W2uq5IVQf56sVRDdE1fKuJoPPb2EtxnN7YG+W8zu/Gz QMra+rrgwOpNgSKA4cAhFOWPjkHrRM0BxNqIYetgm+aRxX+xXakvl7a2z6dKkpHg1iTa mDV8MWfMUq6xqouGuBWNsRYuVedCAz5yCheQs8Xds/Y8m702SlHmySdOsJ6D0sblrFJh IBJw== X-Gm-Message-State: AIkVDXKbpEPHmbNWgYmXrmWzdV5n29ry3eGRB6hTy4KbnA8PiWpu16FUulKGSeA7A0Cl3HRe X-Received: by 10.28.197.77 with SMTP id v74mr1772712wmf.98.1484049491171; Tue, 10 Jan 2017 03:58:11 -0800 (PST) Received: from localhost.localdomain ([109.172.149.148]) by smtp.gmail.com with ESMTPSA id c133sm3426246wme.12.2017.01.10.03.58.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jan 2017 03:58:10 -0800 (PST) From: Aleksey Makarov To: Will Deacon Cc: iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Aleksey Makarov , Robin Murphy , Joerg Roedel Subject: [RFC PATCH] IOMMU: SMMUv2: Support for Extended Stream ID (16 bit) Date: Tue, 10 Jan 2017 15:57:52 +0400 Message-Id: <20170110115755.19102-1-aleksey.makarov@linaro.org> X-Mailer: git-send-email 2.11.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Enable the Extended Stream ID feature when available. This patch on top of series "[PATCH v7 00/19] KVM PCIe/MSI passthrough on ARM/ARM64 and IOVA reserved regions" by Eric Auger allows to passthrough an external PCIe network card on a ThunderX server successfully. Without this patch that card caused a warning like pci 0006:90:00.0: stream ID 0x9000 out of range for SMMU (0x7fff) during boot. Signed-off-by: Aleksey Makarov --- drivers/iommu/arm-smmu.c | 53 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 37 insertions(+), 16 deletions(-) -- 2.11.0 diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 13d26009b8e0..d160c12828f4 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -24,6 +24,7 @@ * - v7/v8 long-descriptor format * - Non-secure access to the SMMU * - Context fault reporting + * - Extended Stream ID (16 bit) */ #define pr_fmt(fmt) "arm-smmu: " fmt @@ -87,6 +88,7 @@ #define sCR0_CLIENTPD (1 << 0) #define sCR0_GFRE (1 << 1) #define sCR0_GFIE (1 << 2) +#define sCR0_EXIDENABLE (1 << 3) #define sCR0_GCFGFRE (1 << 4) #define sCR0_GCFGFIE (1 << 5) #define sCR0_USFCFG (1 << 10) @@ -126,6 +128,7 @@ #define ID0_NUMIRPT_MASK 0xff #define ID0_NUMSIDB_SHIFT 9 #define ID0_NUMSIDB_MASK 0xf +#define ID0_EXIDS (1 << 8) #define ID0_NUMSMRG_SHIFT 0 #define ID0_NUMSMRG_MASK 0xff @@ -169,6 +172,7 @@ #define ARM_SMMU_GR0_S2CR(n) (0xc00 + ((n) << 2)) #define S2CR_CBNDX_SHIFT 0 #define S2CR_CBNDX_MASK 0xff +#define S2CR_EXIDVALID (1 << 10) #define S2CR_TYPE_SHIFT 16 #define S2CR_TYPE_MASK 0x3 enum arm_smmu_s2cr_type { @@ -354,6 +358,7 @@ struct arm_smmu_device { #define ARM_SMMU_FEAT_FMT_AARCH64_64K (1 << 9) #define ARM_SMMU_FEAT_FMT_AARCH32_L (1 << 10) #define ARM_SMMU_FEAT_FMT_AARCH32_S (1 << 11) +#define ARM_SMMU_FEAT_EXIDS (1 << 12) u32 features; #define ARM_SMMU_OPT_SECURE_CFG_ACCESS (1 << 0) @@ -1051,7 +1056,7 @@ static void arm_smmu_write_smr(struct arm_smmu_device *smmu, int idx) struct arm_smmu_smr *smr = smmu->smrs + idx; u32 reg = smr->id << SMR_ID_SHIFT | smr->mask << SMR_MASK_SHIFT; - if (smr->valid) + if (!(smmu->features & ARM_SMMU_FEAT_EXIDS) && smr->valid) reg |= SMR_VALID; writel_relaxed(reg, ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_SMR(idx)); } @@ -1063,6 +1068,9 @@ static void arm_smmu_write_s2cr(struct arm_smmu_device *smmu, int idx) (s2cr->cbndx & S2CR_CBNDX_MASK) << S2CR_CBNDX_SHIFT | (s2cr->privcfg & S2CR_PRIVCFG_MASK) << S2CR_PRIVCFG_SHIFT; + if (smmu->features & ARM_SMMU_FEAT_EXIDS && smmu->smrs && + smmu->smrs[idx].valid) + reg |= S2CR_EXIDVALID; writel_relaxed(reg, ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_S2CR(idx)); } @@ -1674,6 +1682,9 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu) if (smmu->features & ARM_SMMU_FEAT_VMID16) reg |= sCR0_VMID16EN; + if (smmu->features & ARM_SMMU_FEAT_EXIDS) + reg |= sCR0_EXIDENABLE; + /* Push the button */ __arm_smmu_tlb_sync(smmu); writel(reg, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); @@ -1761,7 +1772,12 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) "\t(IDR0.CTTW overridden by FW configuration)\n"); /* Max. number of entries we have for stream matching/indexing */ - size = 1 << ((id >> ID0_NUMSIDB_SHIFT) & ID0_NUMSIDB_MASK); + if (smmu->version == ARM_SMMU_V2 && id & ID0_EXIDS) { + smmu->features |= ARM_SMMU_FEAT_EXIDS; + size = (1 << 16); + } else { + size = 1 << ((id >> ID0_NUMSIDB_SHIFT) & ID0_NUMSIDB_MASK); + } smmu->streamid_mask = size - 1; if (id & ID0_SMS) { u32 smr; @@ -1774,20 +1790,25 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) return -ENODEV; } - /* - * SMR.ID bits may not be preserved if the corresponding MASK - * bits are set, so check each one separately. We can reject - * masters later if they try to claim IDs outside these masks. - */ - smr = smmu->streamid_mask << SMR_ID_SHIFT; - writel_relaxed(smr, gr0_base + ARM_SMMU_GR0_SMR(0)); - smr = readl_relaxed(gr0_base + ARM_SMMU_GR0_SMR(0)); - smmu->streamid_mask = smr >> SMR_ID_SHIFT; - - smr = smmu->streamid_mask << SMR_MASK_SHIFT; - writel_relaxed(smr, gr0_base + ARM_SMMU_GR0_SMR(0)); - smr = readl_relaxed(gr0_base + ARM_SMMU_GR0_SMR(0)); - smmu->smr_mask_mask = smr >> SMR_MASK_SHIFT; + if (smmu->features & ARM_SMMU_FEAT_EXIDS) { + smmu->smr_mask_mask = smmu->streamid_mask; + } else { + /* + * SMR.ID bits may not be preserved if the corresponding + * MASK bits are set, so check each one separately. + * We can reject masters later if they try to claim IDs + * outside these masks. + */ + smr = smmu->streamid_mask << SMR_ID_SHIFT; + writel_relaxed(smr, gr0_base + ARM_SMMU_GR0_SMR(0)); + smr = readl_relaxed(gr0_base + ARM_SMMU_GR0_SMR(0)); + smmu->streamid_mask = smr >> SMR_ID_SHIFT; + + smr = smmu->streamid_mask << SMR_MASK_SHIFT; + writel_relaxed(smr, gr0_base + ARM_SMMU_GR0_SMR(0)); + smr = readl_relaxed(gr0_base + ARM_SMMU_GR0_SMR(0)); + smmu->smr_mask_mask = smr >> SMR_MASK_SHIFT; + } /* Zero-initialised to mark as invalid */ smmu->smrs = devm_kcalloc(smmu->dev, size, sizeof(*smmu->smrs),