From patchwork Fri Jul 21 11:41:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 108474 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp689115qge; Fri, 21 Jul 2017 04:42:21 -0700 (PDT) X-Received: by 10.98.252.87 with SMTP id e84mr7199768pfh.184.1500637341338; Fri, 21 Jul 2017 04:42:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500637341; cv=none; d=google.com; s=arc-20160816; b=sMxKb4f6yRE/5H6MIo/RzeesFajFw/UK7snevBKm+jWoigLsw+c7XH/mhnTQf2Kwn8 Z+akq3Py6BCFjPqB6cmyuUtgwPRKC+PV84NSgYTcpidLaOp2DzDviCW1RADEBtlBPmMT Ud+o2mkFzn+6TfGV6p4ca+gxVVtZb/tdc+/hNEe4tlmtVWe5zxSj7pVTHNvyqVKKPUgi j5jBVanplWXXUoa58oA+pJ+FQR3gsT+T4waDj+KDSFRvWMOzchy6b7rt+pYFGPQvPEaQ cye6nGWNSdjV5l8VyITigbLp62/brj1SPEkzjwCyNAXkjHPAv9B4rNBs0c7ERy68BCcz /fsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=+9phoOCn5XnHpV2oRdA8m1OrJDFyGbOyWx+3HU5n8pU=; b=0IzymyT4BS+FPCDf7oiA4mY6GF64I/tspc/XU+P5O1W1/MnEJOo/x7bjMtUJ+zTbSD s+3PwtVQ7L1jMRV6iveyvc418qnlP4vOsyozn3RO7DjK95R5ANxWg9V3pk2gVNwKToBN 7TeMWL+1ISJ4DyUIuXf/fTp/UXioar/am9Tv6bUsmxIeg90m6/MWNq89DNkjIjRWJBlc 0+bcKMeoenOelstkytr5tdL1QAFMSIy2mxRcs8ZlDLmlnNjuH3oDXbl1RB06BinwWzig mFOubY4KYMhue+wThSc3+/WoPZh/pVfS5PXxRsmcaeog1YZqxJ+4+47ddM/QnJJ5suj/ uAlQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d21si3083793pgn.292.2017.07.21.04.42.21; Fri, 21 Jul 2017 04:42:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753823AbdGULmS (ORCPT + 25 others); Fri, 21 Jul 2017 07:42:18 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:35248 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750802AbdGULmL (ORCPT ); Fri, 21 Jul 2017 07:42:11 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB4121596; Fri, 21 Jul 2017 04:42:10 -0700 (PDT) Received: from e110467-lin.cambridge.arm.com (e110467-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2ABCC3F578; Fri, 21 Jul 2017 04:42:09 -0700 (PDT) From: Robin Murphy To: joro@8bytes.org Cc: iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, dwmw2@infradead.org, thunder.leizhen@huawei.com, lorenzo.pieralisi@arm.com, ard.biesheuvel@linaro.org, Jonathan.Cameron@huawei.com, nwatters@codeaurora.org, ray.jui@broadcom.com Subject: [PATCH v2 1/4] iommu/iova: Optimise rbtree searching Date: Fri, 21 Jul 2017 12:41:58 +0100 Message-Id: X-Mailer: git-send-email 2.12.2.dirty In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Zhen Lei Checking the IOVA bounds separately before deciding which direction to continue the search (if necessary) results in redundantly comparing both pfns twice each. GCC can already determine that the final comparison op is redundant and optimise it down to 3 in total, but we can go one further with a little tweak of the ordering (which makes the intent of the code that much cleaner as a bonus). Signed-off-by: Zhen Lei Tested-by: Ard Biesheuvel Tested-by: Zhen Lei [rm: rewrote commit message to clarify] Signed-off-by: Robin Murphy --- v2: No change drivers/iommu/iova.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) -- 2.12.2.dirty diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 246f14c83944..8f7552dc4e04 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -289,15 +289,12 @@ private_find_iova(struct iova_domain *iovad, unsigned long pfn) while (node) { struct iova *iova = rb_entry(node, struct iova, node); - /* If pfn falls within iova's range, return iova */ - if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) { - return iova; - } - if (pfn < iova->pfn_lo) node = node->rb_left; - else if (pfn > iova->pfn_lo) + else if (pfn > iova->pfn_hi) node = node->rb_right; + else + return iova; /* pfn falls within iova's range */ } return NULL; From patchwork Fri Jul 21 11:41:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 108476 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp689528qge; Fri, 21 Jul 2017 04:42:49 -0700 (PDT) X-Received: by 10.98.219.130 with SMTP id f124mr7273774pfg.168.1500637369467; Fri, 21 Jul 2017 04:42:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500637369; cv=none; d=google.com; s=arc-20160816; b=DykYHkh70OAg79JY4XwL3OqE6U1fCjeDA6R+nhSQt/TCgZc3Vl9XtYbPQE1IRlCJpJ mLxU5Ll/A+QPHKVsFsbRmgmyPHMrFpVD4o2H2pgDnip2v5af42fEf9+m5RajXwG1Pkp1 OfDzPtfFYRJ3sHQn7KDCdvd3ole8JDonIdtBpjLIgiOHKZ50ZJdZXSu4ZJiPzK0Roh42 X6pLrTrHr/qoHfEzQooj6L+gao8DrzNWFMZUUF/ukvviPEes/2oIeJ0N0svWlssqU3NO F9Btp8pNNTNS6oV72GAxk4edfbW+hzBsvuOc+oMlZRBe8DHbAhDlUM8xTPxurHyZKRPs x+CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=9euJSnES7EyqK9U0FFQJIKYgyudHO+QyvY6T1UV6oZM=; b=Jv/ZKRUpIGPVWX1lGR50KAFqU3Kn4XvHfSUdkPdwYsEFNKH8bMFY56eztHHc3lIx9f tSgc5QGyxU+JFnDNF+zySgU2uu35eEgUxeHFVYZVzovL/g+/u+tah/EXFFJYQAgePImA 2893Oe0ktzHDhp3JY3eiuL++8s6obNxfn6vnmLaU2JBc+xEux0AIK3YXVh50YV6Kj4dz 2tVJCcu2jlZlnyZBK+4sIj/J4rYFwl3/4hPFOJD8BE34KgL+/gOh7JWx6/AtxM0yQwKj D9jUI/c92zSrZMAk6HN/hAx2UOby6aq9d1FgmPU1v2nHbnPXbyZb1eyTrT+CNuNgFYGZ mftA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m8si3008131pgc.582.2017.07.21.04.42.49; Fri, 21 Jul 2017 04:42:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753424AbdGULmR (ORCPT + 25 others); Fri, 21 Jul 2017 07:42:17 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:35270 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751065AbdGULmO (ORCPT ); Fri, 21 Jul 2017 07:42:14 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D67D515AD; Fri, 21 Jul 2017 04:42:13 -0700 (PDT) Received: from e110467-lin.cambridge.arm.com (e110467-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 152093F578; Fri, 21 Jul 2017 04:42:11 -0700 (PDT) From: Robin Murphy To: joro@8bytes.org Cc: iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, dwmw2@infradead.org, thunder.leizhen@huawei.com, lorenzo.pieralisi@arm.com, ard.biesheuvel@linaro.org, Jonathan.Cameron@huawei.com, nwatters@codeaurora.org, ray.jui@broadcom.com Subject: [PATCH v2 2/4] iommu/iova: Optimise the padding calculation Date: Fri, 21 Jul 2017 12:41:59 +0100 Message-Id: <4f964e56fe39c6ce1c84b8458c3a27dcb51077d4.1500636791.git.robin.murphy@arm.com> X-Mailer: git-send-email 2.12.2.dirty In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Zhen Lei The mask for calculating the padding size doesn't change, so there's no need to recalculate it every loop iteration. Furthermore, Once we've done that, it becomes clear that we don't actually need to calculate a padding size at all - by flipping the arithmetic around, we can just combine the upper limit, size, and mask directly to check against the lower limit. For an arm64 build, this alone knocks 16% off the size of the entire alloc_iova() function! Signed-off-by: Zhen Lei Tested-by: Ard Biesheuvel Tested-by: Zhen Lei [rm: simplified more of the arithmetic, rewrote commit message] Signed-off-by: Robin Murphy --- v2: No change drivers/iommu/iova.c | 40 ++++++++++++++-------------------------- 1 file changed, 14 insertions(+), 26 deletions(-) -- 2.12.2.dirty diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 8f7552dc4e04..d094d1ca8f23 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -129,16 +129,6 @@ iova_insert_rbtree(struct rb_root *root, struct iova *iova, rb_insert_color(&iova->node, root); } -/* - * Computes the padding size required, to make the start address - * naturally aligned on the power-of-two order of its size - */ -static unsigned int -iova_get_pad_size(unsigned int size, unsigned int limit_pfn) -{ - return (limit_pfn - size) & (__roundup_pow_of_two(size) - 1); -} - static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, struct iova *new, bool size_aligned) @@ -146,7 +136,10 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, struct rb_node *prev, *curr = NULL; unsigned long flags; unsigned long saved_pfn; - unsigned int pad_size = 0; + unsigned long align_mask = ~0UL; + + if (size_aligned) + align_mask <<= __fls(size); /* Walk the tree backwards */ spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); @@ -156,31 +149,26 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, while (curr) { struct iova *curr_iova = rb_entry(curr, struct iova, node); - if (limit_pfn <= curr_iova->pfn_lo) { + if (limit_pfn <= curr_iova->pfn_lo) goto move_left; - } else if (limit_pfn > curr_iova->pfn_hi) { - if (size_aligned) - pad_size = iova_get_pad_size(size, limit_pfn); - if ((curr_iova->pfn_hi + size + pad_size) < limit_pfn) - break; /* found a free slot */ - } + + if (((limit_pfn - size) & align_mask) > curr_iova->pfn_hi) + break; /* found a free slot */ + limit_pfn = curr_iova->pfn_lo; move_left: prev = curr; curr = rb_prev(curr); } - if (!curr) { - if (size_aligned) - pad_size = iova_get_pad_size(size, limit_pfn); - if ((iovad->start_pfn + size + pad_size) > limit_pfn) { - spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); - return -ENOMEM; - } + if (limit_pfn < size || + (!curr && ((limit_pfn - size) & align_mask) < iovad->start_pfn)) { + spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); + return -ENOMEM; } /* pfn_lo will point to size aligned address if size_aligned is set */ - new->pfn_lo = limit_pfn - (size + pad_size); + new->pfn_lo = (limit_pfn - size) & align_mask; new->pfn_hi = new->pfn_lo + size - 1; /* If we have 'prev', it's a valid place to start the insertion. */ From patchwork Fri Jul 21 11:42:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 108475 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp689225qge; Fri, 21 Jul 2017 04:42:27 -0700 (PDT) X-Received: by 10.99.126.66 with SMTP id o2mr7384234pgn.36.1500637347666; Fri, 21 Jul 2017 04:42:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500637347; cv=none; d=google.com; s=arc-20160816; b=n/rYCw9O88xYGqG5vNumO1kEtJxqs4OLY3Wh8iJ9y8H3cbvyH9bM6Bq5IQsRvaQ3gZ fHOIoTRy4S7wa2JfNsn9IOelWGDbqT/qCvFFwKVxtGQCd+ay6E/Tz7durnKVFb0yUBvJ vdPjdGXMAGJX/gS7bC4VqL4LbYkgcxN/GAzCDjMexzZBnUsxN/cs3B+HuH4UOuiNPvHb de/YEM+RW6lfX0zFcbsI07AVVX4ECA9FM8rkolP2UIJSTldC4mv2l7gnKXMAF2ugCK7p GiJBinzKEnaCGAj5vwOAgODjbKOUZ38UvNhRdDC8rwJBl8dOKDFVjKVldn5mgngqjUfG zHjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=stf7BN75bKNMQcqTXV9syDhwNywmYIrHDS4/zk9T3TM=; b=asyBBSDIBvyTjDFzU7HneYzpAMN5Z/XovqtvUJVFiYgCeLnNH44rmJRXOwUJxecKSR wfTjiIhx4bwTIXVE/VWGx8/m7euG6WBlWeFmPjgTcdASsNNpJgT6AcT/3xg0Ekt+YY7J UjxWmejGpwJAsBfmoAqyriJEg0IDTCiDC0J/o5Gvg9zxId3lT2KY1pTLrM8oOtTy9iPr s73FgZqfC62VoKai934hQ06EhmrEYn8OF2kYB1w3mHI5gb/4+pHaVyqU4xy6L9ugCiie 8L6TBpNv27pL6w0v9SI9JRLA2mGICtlcXiNQ6lD2ma46tc2t+MJJocYKpuaBIdjeYKhD VVkw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e7si2970489pgc.261.2017.07.21.04.42.27; Fri, 21 Jul 2017 04:42:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753881AbdGULmY (ORCPT + 25 others); Fri, 21 Jul 2017 07:42:24 -0400 Received: from foss.arm.com ([217.140.101.70]:35310 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750802AbdGULmU (ORCPT ); Fri, 21 Jul 2017 07:42:20 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2575F1610; Fri, 21 Jul 2017 04:42:20 -0700 (PDT) Received: from e110467-lin.cambridge.arm.com (e110467-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 93F033F578; Fri, 21 Jul 2017 04:42:17 -0700 (PDT) From: Robin Murphy To: joro@8bytes.org Cc: iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, dwmw2@infradead.org, thunder.leizhen@huawei.com, lorenzo.pieralisi@arm.com, ard.biesheuvel@linaro.org, Jonathan.Cameron@huawei.com, nwatters@codeaurora.org, ray.jui@broadcom.com, Thierry Reding , Jonathan Hunter , David Airlie , Sudeep Dutt , Ashutosh Dixit Subject: [PATCH v2 4/4] iommu/iova: Make dma_32bit_pfn implicit Date: Fri, 21 Jul 2017 12:42:01 +0100 Message-Id: <1335357886db139a51f7d0bd0728067d0d77de8d.1500636791.git.robin.murphy@arm.com> X-Mailer: git-send-email 2.12.2.dirty In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Zhen Lei Now that the cached node optimisation can apply to all allocations, the couple of users which were playing tricks with dma_32bit_pfn in order to benefit from it can stop doing so. Conversely, there is also no need for all the other users to explicitly calculate a 'real' 32-bit PFN, when init_iova_domain() can happily do that itself from the page granularity. CC: Thierry Reding CC: Jonathan Hunter CC: David Airlie CC: Sudeep Dutt CC: Ashutosh Dixit Signed-off-by: Zhen Lei Tested-by: Ard Biesheuvel Tested-by: Zhen Lei [rm: use iova_shift(), rewrote commit message] Signed-off-by: Robin Murphy --- v2: Avoid iova_pfn() overflow with 32-bit dma_addr_t drivers/gpu/drm/tegra/drm.c | 3 +-- drivers/gpu/host1x/dev.c | 3 +-- drivers/iommu/amd_iommu.c | 7 ++----- drivers/iommu/dma-iommu.c | 18 +----------------- drivers/iommu/intel-iommu.c | 11 +++-------- drivers/iommu/iova.c | 4 ++-- drivers/misc/mic/scif/scif_rma.c | 3 +-- include/linux/iova.h | 5 ++--- 8 files changed, 13 insertions(+), 41 deletions(-) -- 2.12.2.dirty diff --git a/drivers/gpu/drm/tegra/drm.c b/drivers/gpu/drm/tegra/drm.c index 518f4b69ea53..81e9ae1ee90b 100644 --- a/drivers/gpu/drm/tegra/drm.c +++ b/drivers/gpu/drm/tegra/drm.c @@ -150,8 +150,7 @@ static int tegra_drm_load(struct drm_device *drm, unsigned long flags) order = __ffs(tegra->domain->pgsize_bitmap); init_iova_domain(&tegra->carveout.domain, 1UL << order, - carveout_start >> order, - carveout_end >> order); + carveout_start >> order); tegra->carveout.shift = iova_shift(&tegra->carveout.domain); tegra->carveout.limit = carveout_end >> tegra->carveout.shift; diff --git a/drivers/gpu/host1x/dev.c b/drivers/gpu/host1x/dev.c index 2c58a390123a..57c8eed0ed71 100644 --- a/drivers/gpu/host1x/dev.c +++ b/drivers/gpu/host1x/dev.c @@ -193,8 +193,7 @@ static int host1x_probe(struct platform_device *pdev) order = __ffs(host->domain->pgsize_bitmap); init_iova_domain(&host->iova, 1UL << order, - geometry->aperture_start >> order, - geometry->aperture_end >> order); + geometry->aperture_start >> order); host->iova_end = geometry->aperture_end; } diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 688e77576e5a..a12e3e12014a 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -63,7 +63,6 @@ /* IO virtual address start page frame number */ #define IOVA_START_PFN (1) #define IOVA_PFN(addr) ((addr) >> PAGE_SHIFT) -#define DMA_32BIT_PFN IOVA_PFN(DMA_BIT_MASK(32)) /* Reserved IOVA ranges */ #define MSI_RANGE_START (0xfee00000) @@ -2010,8 +2009,7 @@ static struct dma_ops_domain *dma_ops_domain_alloc(void) if (!dma_dom->domain.pt_root) goto free_dma_dom; - init_iova_domain(&dma_dom->iovad, PAGE_SIZE, - IOVA_START_PFN, DMA_32BIT_PFN); + init_iova_domain(&dma_dom->iovad, PAGE_SIZE, IOVA_START_PFN); /* Initialize reserved ranges */ copy_reserved_iova(&reserved_iova_ranges, &dma_dom->iovad); @@ -2912,8 +2910,7 @@ static int init_reserved_iova_ranges(void) struct pci_dev *pdev = NULL; struct iova *val; - init_iova_domain(&reserved_iova_ranges, PAGE_SIZE, - IOVA_START_PFN, DMA_32BIT_PFN); + init_iova_domain(&reserved_iova_ranges, PAGE_SIZE, IOVA_START_PFN); lockdep_set_class(&reserved_iova_ranges.iova_rbtree_lock, &reserved_rbtree_key); diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 9d1cebe7f6cb..191be9c80a8a 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -292,18 +292,7 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, /* ...then finally give it a kicking to make sure it fits */ base_pfn = max_t(unsigned long, base_pfn, domain->geometry.aperture_start >> order); - end_pfn = min_t(unsigned long, end_pfn, - domain->geometry.aperture_end >> order); } - /* - * PCI devices may have larger DMA masks, but still prefer allocating - * within a 32-bit mask to avoid DAC addressing. Such limitations don't - * apply to the typical platform device, so for those we may as well - * leave the cache limit at the top of their range to save an rb_last() - * traversal on every allocation. - */ - if (dev && dev_is_pci(dev)) - end_pfn &= DMA_BIT_MASK(32) >> order; /* start_pfn is always nonzero for an already-initialised domain */ if (iovad->start_pfn) { @@ -312,16 +301,11 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, pr_warn("Incompatible range for DMA domain\n"); return -EFAULT; } - /* - * If we have devices with different DMA masks, move the free - * area cache limit down for the benefit of the smaller one. - */ - iovad->dma_32bit_pfn = min(end_pfn + 1, iovad->dma_32bit_pfn); return 0; } - init_iova_domain(iovad, 1UL << order, base_pfn, end_pfn); + init_iova_domain(iovad, 1UL << order, base_pfn); if (!dev) return 0; diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 687f18f65cea..afa3b4e765e7 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -82,8 +82,6 @@ #define IOVA_START_PFN (1) #define IOVA_PFN(addr) ((addr) >> PAGE_SHIFT) -#define DMA_32BIT_PFN IOVA_PFN(DMA_BIT_MASK(32)) -#define DMA_64BIT_PFN IOVA_PFN(DMA_BIT_MASK(64)) /* page table handling */ #define LEVEL_STRIDE (9) @@ -1874,8 +1872,7 @@ static int dmar_init_reserved_ranges(void) struct iova *iova; int i; - init_iova_domain(&reserved_iova_list, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&reserved_iova_list, VTD_PAGE_SIZE, IOVA_START_PFN); lockdep_set_class(&reserved_iova_list.iova_rbtree_lock, &reserved_rbtree_key); @@ -1933,8 +1930,7 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, int adjust_width, agaw; unsigned long sagaw; - init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN); domain_reserve_special_ranges(domain); /* calculate AGAW */ @@ -4989,8 +4985,7 @@ static int md_domain_init(struct dmar_domain *domain, int guest_width) { int adjust_width; - init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN); domain_reserve_special_ranges(domain); /* calculate AGAW */ diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index f5809a2ee6c2..f88acadeebfe 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -35,7 +35,7 @@ static void free_iova_rcaches(struct iova_domain *iovad); void init_iova_domain(struct iova_domain *iovad, unsigned long granule, - unsigned long start_pfn, unsigned long pfn_32bit) + unsigned long start_pfn) { /* * IOVA granularity will normally be equal to the smallest @@ -50,7 +50,7 @@ init_iova_domain(struct iova_domain *iovad, unsigned long granule, iovad->cached32_node = NULL; iovad->granule = granule; iovad->start_pfn = start_pfn; - iovad->dma_32bit_pfn = pfn_32bit + 1; + iovad->dma_32bit_pfn = 1UL << (32 - iova_shift(iovad)); init_iova_rcaches(iovad); } EXPORT_SYMBOL_GPL(init_iova_domain); diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c index 329727e00e97..c824329f7012 100644 --- a/drivers/misc/mic/scif/scif_rma.c +++ b/drivers/misc/mic/scif/scif_rma.c @@ -39,8 +39,7 @@ void scif_rma_ep_init(struct scif_endpt *ep) struct scif_endpt_rma_info *rma = &ep->rma_info; mutex_init(&rma->rma_lock); - init_iova_domain(&rma->iovad, PAGE_SIZE, SCIF_IOVA_START_PFN, - SCIF_DMA_64BIT_PFN); + init_iova_domain(&rma->iovad, PAGE_SIZE, SCIF_IOVA_START_PFN); spin_lock_init(&rma->tc_lock); mutex_init(&rma->mmn_lock); INIT_LIST_HEAD(&rma->reg_list); diff --git a/include/linux/iova.h b/include/linux/iova.h index 0bb8df43b393..58c2a365c45f 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -102,7 +102,7 @@ struct iova *reserve_iova(struct iova_domain *iovad, unsigned long pfn_lo, unsigned long pfn_hi); void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to); void init_iova_domain(struct iova_domain *iovad, unsigned long granule, - unsigned long start_pfn, unsigned long pfn_32bit); + unsigned long start_pfn); struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn); void put_iova_domain(struct iova_domain *iovad); struct iova *split_and_remove_iova(struct iova_domain *iovad, @@ -170,8 +170,7 @@ static inline void copy_reserved_iova(struct iova_domain *from, static inline void init_iova_domain(struct iova_domain *iovad, unsigned long granule, - unsigned long start_pfn, - unsigned long pfn_32bit) + unsigned long start_pfn) { }