From patchwork Wed Mar 22 06:27:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95696 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86517qgd; Tue, 21 Mar 2017 23:30:35 -0700 (PDT) X-Received: by 10.99.2.139 with SMTP id 133mr42201967pgc.168.1490164235383; Tue, 21 Mar 2017 23:30:35 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t188si564788pgt.331.2017.03.21.23.30.35; Tue, 21 Mar 2017 23:30:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758846AbdCVGaF (ORCPT + 16 others); Wed, 22 Mar 2017 02:30:05 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:4352 "EHLO dggrg02-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1758611AbdCVG33 (ORCPT ); Wed, 22 Mar 2017 02:29:29 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKH60773; Wed, 22 Mar 2017 14:29:21 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:09 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 1/7] iommu/iova: fix incorrect variable types Date: Wed, 22 Mar 2017 14:27:41 +0800 Message-ID: <1490164067-12552-2-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0202.58D219C3.039A, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 78cbb16d2aeef7ce94a35e29d941e132 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Keep these four variables type consistent with the paramters of function __alloc_and_insert_iova_range and the members of struct iova: 1. static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, 2. struct iova { unsigned long pfn_hi; unsigned long pfn_lo; In fact, limit_pfn is most likely larger than 32 bits on DMA64. Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index b7268a1..8ba8b496 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -104,8 +104,8 @@ __cached_rbnode_delete_update(struct iova_domain *iovad, struct iova *free) * Computes the padding size required, to make the start address * naturally aligned on the power-of-two order of its size */ -static unsigned int -iova_get_pad_size(unsigned int size, unsigned int limit_pfn) +static unsigned long +iova_get_pad_size(unsigned long size, unsigned long limit_pfn) { return (limit_pfn + 1 - size) & (__roundup_pow_of_two(size) - 1); } @@ -117,7 +117,7 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, struct rb_node *prev, *curr = NULL; unsigned long flags; unsigned long saved_pfn; - unsigned int pad_size = 0; + unsigned long pad_size = 0; /* Walk the tree backwards */ spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); From patchwork Wed Mar 22 06:27:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95694 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86304qgd; Tue, 21 Mar 2017 23:30:01 -0700 (PDT) X-Received: by 10.99.231.83 with SMTP id j19mr41570289pgk.50.1490164201235; Tue, 21 Mar 2017 23:30:01 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r5si423551pfl.34.2017.03.21.23.30.00; Tue, 21 Mar 2017 23:30:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758811AbdCVG3o (ORCPT + 16 others); Wed, 22 Mar 2017 02:29:44 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:4348 "EHLO dggrg02-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1757351AbdCVG32 (ORCPT ); Wed, 22 Mar 2017 02:29:28 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKH60775; Wed, 22 Mar 2017 14:29:21 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:10 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 2/7] iommu/iova: cut down judgement times Date: Wed, 22 Mar 2017 14:27:42 +0800 Message-ID: <1490164067-12552-3-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0203.58D219C3.04A4, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: ce6238b6d6c8f75336f7b9148b5e8178 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Below judgement can only be satisfied at the last time, which produced 2N judgements(suppose N times failed, 0 or 1 time successed) in vain. if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) { return iova; } Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) -- 2.5.0 Reviewed-by: Robin Murphy diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 8ba8b496..1c49969 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -312,15 +312,12 @@ private_find_iova(struct iova_domain *iovad, unsigned long pfn) while (node) { struct iova *iova = rb_entry(node, struct iova, node); - /* If pfn falls within iova's range, return iova */ - if ((pfn >= iova->pfn_lo) && (pfn <= iova->pfn_hi)) { - return iova; - } - if (pfn < iova->pfn_lo) node = node->rb_left; - else if (pfn > iova->pfn_lo) + else if (pfn > iova->pfn_hi) node = node->rb_right; + else + return iova; /* pfn falls within iova's range */ } return NULL; From patchwork Wed Mar 22 06:27:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95699 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86639qgd; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) X-Received: by 10.98.93.150 with SMTP id n22mr45124917pfj.103.1490164260025; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w19si399900pfa.377.2017.03.21.23.30.59; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933184AbdCVGao (ORCPT + 16 others); Wed, 22 Mar 2017 02:30:44 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:4350 "EHLO dggrg02-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1758405AbdCVG33 (ORCPT ); Wed, 22 Mar 2017 02:29:29 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKH60782; Wed, 22 Mar 2017 14:29:22 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:11 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 3/7] iommu/iova: insert start_pfn boundary of dma32 Date: Wed, 22 Mar 2017 14:27:43 +0800 Message-ID: <1490164067-12552-4-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0205.58D219C3.03B5, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: f7a4e4003358ecd7ea91be973f1218f1 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Reserve the first granule size memory(start at start_pfn) as boundary iova, to make sure that iovad->cached32_node can not be NULL in future. Meanwhile, changed the assignment of iovad->cached32_node from rb_next to rb_prev of &free->node in function __cached_rbnode_delete_update. Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 63 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 37 insertions(+), 26 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 1c49969..b5a148e 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -32,6 +32,17 @@ static unsigned long iova_rcache_get(struct iova_domain *iovad, static void init_iova_rcaches(struct iova_domain *iovad); static void free_iova_rcaches(struct iova_domain *iovad); +static void +insert_iova_boundary(struct iova_domain *iovad) +{ + struct iova *iova; + unsigned long start_pfn_32bit = iovad->start_pfn; + + iova = reserve_iova(iovad, start_pfn_32bit, start_pfn_32bit); + BUG_ON(!iova); + iovad->cached32_node = &iova->node; +} + void init_iova_domain(struct iova_domain *iovad, unsigned long granule, unsigned long start_pfn, unsigned long pfn_32bit) @@ -45,27 +56,38 @@ init_iova_domain(struct iova_domain *iovad, unsigned long granule, spin_lock_init(&iovad->iova_rbtree_lock); iovad->rbroot = RB_ROOT; - iovad->cached32_node = NULL; iovad->granule = granule; iovad->start_pfn = start_pfn; iovad->dma_32bit_pfn = pfn_32bit; init_iova_rcaches(iovad); + + /* + * Insert boundary nodes for dma32. So cached32_node can not be NULL in + * future. + */ + insert_iova_boundary(iovad); } EXPORT_SYMBOL_GPL(init_iova_domain); static struct rb_node * __get_cached_rbnode(struct iova_domain *iovad, unsigned long *limit_pfn) { - if ((*limit_pfn > iovad->dma_32bit_pfn) || - (iovad->cached32_node == NULL)) + struct rb_node *cached_node; + struct rb_node *next_node; + + if (*limit_pfn > iovad->dma_32bit_pfn) return rb_last(&iovad->rbroot); - else { - struct rb_node *prev_node = rb_prev(iovad->cached32_node); - struct iova *curr_iova = - rb_entry(iovad->cached32_node, struct iova, node); - *limit_pfn = curr_iova->pfn_lo - 1; - return prev_node; + else + cached_node = iovad->cached32_node; + + next_node = rb_next(cached_node); + if (next_node) { + struct iova *next_iova = rb_entry(next_node, struct iova, node); + + *limit_pfn = min(*limit_pfn, next_iova->pfn_lo - 1); } + + return cached_node; } static void @@ -83,20 +105,13 @@ __cached_rbnode_delete_update(struct iova_domain *iovad, struct iova *free) struct iova *cached_iova; struct rb_node *curr; - if (!iovad->cached32_node) - return; curr = iovad->cached32_node; cached_iova = rb_entry(curr, struct iova, node); if (free->pfn_lo >= cached_iova->pfn_lo) { - struct rb_node *node = rb_next(&free->node); - struct iova *iova = rb_entry(node, struct iova, node); - /* only cache if it's below 32bit pfn */ - if (node && iova->pfn_lo < iovad->dma_32bit_pfn) - iovad->cached32_node = node; - else - iovad->cached32_node = NULL; + if (free->pfn_hi <= iovad->dma_32bit_pfn) + iovad->cached32_node = rb_prev(&free->node); } } @@ -114,7 +129,7 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, struct iova *new, bool size_aligned) { - struct rb_node *prev, *curr = NULL; + struct rb_node *prev, *curr; unsigned long flags; unsigned long saved_pfn; unsigned long pad_size = 0; @@ -144,13 +159,9 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, curr = rb_prev(curr); } - if (!curr) { - if (size_aligned) - pad_size = iova_get_pad_size(size, limit_pfn); - if ((iovad->start_pfn + size + pad_size) > limit_pfn) { - spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); - return -ENOMEM; - } + if (unlikely(!curr)) { + spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); + return -ENOMEM; } /* pfn_lo will point to size aligned address if size_aligned is set */ From patchwork Wed Mar 22 06:27:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95695 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86321qgd; Tue, 21 Mar 2017 23:30:03 -0700 (PDT) X-Received: by 10.99.127.29 with SMTP id a29mr25680657pgd.11.1490164203367; Tue, 21 Mar 2017 23:30:03 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r5si423551pfl.34.2017.03.21.23.30.03; Tue, 21 Mar 2017 23:30:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758831AbdCVG3y (ORCPT + 16 others); Wed, 22 Mar 2017 02:29:54 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:4349 "EHLO dggrg02-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752391AbdCVG32 (ORCPT ); Wed, 22 Mar 2017 02:29:28 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKH60777; Wed, 22 Mar 2017 14:29:21 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:11 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 4/7] iommu/iova: adjust __cached_rbnode_insert_update Date: Wed, 22 Mar 2017 14:27:44 +0800 Message-ID: <1490164067-12552-5-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0205.58D219C3.03ED, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 6ffe392b6f421bbc1a91bc063760fe73 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org For case 2 and 3, adjust cached32_node to the new place, case 1 keep no change. For example: case1: (the right part was allocated) |------------------------------| |<-----free---->|<--new_iova-->| | | cached32_node case2: (all was allocated) |------------------------------| |<---------new_iova----------->| | | cached32_node case3: |-----------------------|......|---------| |..free..|<--new_iova-->| | | | | cached32_node(new) cached32_node(old) Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index b5a148e..87a9332 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -91,12 +91,16 @@ __get_cached_rbnode(struct iova_domain *iovad, unsigned long *limit_pfn) } static void -__cached_rbnode_insert_update(struct iova_domain *iovad, - unsigned long limit_pfn, struct iova *new) +__cached_rbnode_insert_update(struct iova_domain *iovad, struct iova *new) { - if (limit_pfn != iovad->dma_32bit_pfn) + struct iova *cached_iova; + + if (new->pfn_hi > iovad->dma_32bit_pfn) return; - iovad->cached32_node = &new->node; + + cached_iova = rb_entry(iovad->cached32_node, struct iova, node); + if (new->pfn_lo <= cached_iova->pfn_lo) + iovad->cached32_node = rb_prev(&new->node); } static void @@ -131,12 +135,10 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, { struct rb_node *prev, *curr; unsigned long flags; - unsigned long saved_pfn; unsigned long pad_size = 0; /* Walk the tree backwards */ spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); - saved_pfn = limit_pfn; curr = __get_cached_rbnode(iovad, &limit_pfn); prev = curr; while (curr) { @@ -197,11 +199,10 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, rb_link_node(&new->node, parent, entry); rb_insert_color(&new->node, &iovad->rbroot); } - __cached_rbnode_insert_update(iovad, saved_pfn, new); + __cached_rbnode_insert_update(iovad, new); spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags); - return 0; } From patchwork Wed Mar 22 06:27:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95697 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86521qgd; Tue, 21 Mar 2017 23:30:36 -0700 (PDT) X-Received: by 10.98.89.139 with SMTP id k11mr43187542pfj.196.1490164235991; Tue, 21 Mar 2017 23:30:35 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t188si564788pgt.331.2017.03.21.23.30.35; Tue, 21 Mar 2017 23:30:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758861AbdCVGaT (ORCPT + 16 others); Wed, 22 Mar 2017 02:30:19 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:4351 "EHLO dggrg02-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1758766AbdCVG3c (ORCPT ); Wed, 22 Mar 2017 02:29:32 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKH60780; Wed, 22 Mar 2017 14:29:22 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:12 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 5/7] iommu/iova: to optimize the allocation performance of dma64 Date: Wed, 22 Mar 2017 14:27:45 +0800 Message-ID: <1490164067-12552-6-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0203.58D219C3.0A46, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 73a87cc0d0a2bc9a6e22a0adebd33d36 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently we always search free iova space for dma64 begin at the last node of iovad rb-tree. In the worst case, there maybe too many nodes exist at the tail, so that we should traverse many times for the first loop in __alloc_and_insert_iova_range. As we traced, more than 10K times for the case of iperf. __alloc_and_insert_iova_range: ...... curr = __get_cached_rbnode(iovad, &limit_pfn); //--> return rb_last(&iovad->rbroot); while (curr) { ...... curr = rb_prev(curr); } So add cached64_node to take the same effect as cached32_node, and add the start_pfn boundary of dma64, to prevent a iova cross both dma32 and dma64 area. |-------------------|------------------------------| |<--cached32_node-->|<--------cached64_node------->| | | start_pfn dma_32bit_pfn + 1 Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 46 +++++++++++++++++++++++++++------------------- include/linux/iova.h | 5 +++-- 2 files changed, 30 insertions(+), 21 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 87a9332..23abe84 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -37,10 +37,15 @@ insert_iova_boundary(struct iova_domain *iovad) { struct iova *iova; unsigned long start_pfn_32bit = iovad->start_pfn; + unsigned long start_pfn_64bit = iovad->dma_32bit_pfn + 1; iova = reserve_iova(iovad, start_pfn_32bit, start_pfn_32bit); BUG_ON(!iova); iovad->cached32_node = &iova->node; + + iova = reserve_iova(iovad, start_pfn_64bit, start_pfn_64bit); + BUG_ON(!iova); + iovad->cached64_node = &iova->node; } void @@ -62,8 +67,8 @@ init_iova_domain(struct iova_domain *iovad, unsigned long granule, init_iova_rcaches(iovad); /* - * Insert boundary nodes for dma32. So cached32_node can not be NULL in - * future. + * Insert boundary nodes for dma32 and dma64. So cached32_node and + * cached64_node can not be NULL in future. */ insert_iova_boundary(iovad); } @@ -75,10 +80,10 @@ __get_cached_rbnode(struct iova_domain *iovad, unsigned long *limit_pfn) struct rb_node *cached_node; struct rb_node *next_node; - if (*limit_pfn > iovad->dma_32bit_pfn) - return rb_last(&iovad->rbroot); - else + if (*limit_pfn <= iovad->dma_32bit_pfn) cached_node = iovad->cached32_node; + else + cached_node = iovad->cached64_node; next_node = rb_next(cached_node); if (next_node) { @@ -94,29 +99,32 @@ static void __cached_rbnode_insert_update(struct iova_domain *iovad, struct iova *new) { struct iova *cached_iova; + struct rb_node **cached_node; - if (new->pfn_hi > iovad->dma_32bit_pfn) - return; + if (new->pfn_hi <= iovad->dma_32bit_pfn) + cached_node = &iovad->cached32_node; + else + cached_node = &iovad->cached64_node; - cached_iova = rb_entry(iovad->cached32_node, struct iova, node); + cached_iova = rb_entry(*cached_node, struct iova, node); if (new->pfn_lo <= cached_iova->pfn_lo) - iovad->cached32_node = rb_prev(&new->node); + *cached_node = rb_prev(&new->node); } static void __cached_rbnode_delete_update(struct iova_domain *iovad, struct iova *free) { struct iova *cached_iova; - struct rb_node *curr; + struct rb_node **cached_node; - curr = iovad->cached32_node; - cached_iova = rb_entry(curr, struct iova, node); + if (free->pfn_hi <= iovad->dma_32bit_pfn) + cached_node = &iovad->cached32_node; + else + cached_node = &iovad->cached64_node; - if (free->pfn_lo >= cached_iova->pfn_lo) { - /* only cache if it's below 32bit pfn */ - if (free->pfn_hi <= iovad->dma_32bit_pfn) - iovad->cached32_node = rb_prev(&free->node); - } + cached_iova = rb_entry(*cached_node, struct iova, node); + if (free->pfn_lo >= cached_iova->pfn_lo) + *cached_node = rb_prev(&free->node); } /* @@ -283,7 +291,7 @@ EXPORT_SYMBOL_GPL(iova_cache_put); * alloc_iova - allocates an iova * @iovad: - iova domain in question * @size: - size of page frames to allocate - * @limit_pfn: - max limit address + * @limit_pfn: - max limit address(included) * @size_aligned: - set if size_aligned address range is required * This function allocates an iova in the range iovad->start_pfn to limit_pfn, * searching top-down from limit_pfn to iovad->start_pfn. If the size_aligned @@ -402,7 +410,7 @@ EXPORT_SYMBOL_GPL(free_iova); * alloc_iova_fast - allocates an iova from rcache * @iovad: - iova domain in question * @size: - size of page frames to allocate - * @limit_pfn: - max limit address + * @limit_pfn: - max limit address(included) * This function tries to satisfy an iova allocation from the rcache, * and falls back to regular allocation on failure. */ diff --git a/include/linux/iova.h b/include/linux/iova.h index f27bb2c..844d723 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -40,10 +40,11 @@ struct iova_rcache { struct iova_domain { spinlock_t iova_rbtree_lock; /* Lock to protect update of rbtree */ struct rb_root rbroot; /* iova domain rbtree root */ - struct rb_node *cached32_node; /* Save last alloced node */ + struct rb_node *cached32_node; /* Save last alloced node, 32bits */ + struct rb_node *cached64_node; /* Save last alloced node, 64bits */ unsigned long granule; /* pfn granularity for this domain */ unsigned long start_pfn; /* Lower limit for this domain */ - unsigned long dma_32bit_pfn; + unsigned long dma_32bit_pfn; /* max dma32 limit address(included) */ struct iova_rcache rcaches[IOVA_RANGE_CACHE_MAX_SIZE]; /* IOVA range caches */ }; From patchwork Wed Mar 22 06:27:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95700 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86642qgd; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) X-Received: by 10.98.71.149 with SMTP id p21mr43661984pfi.94.1490164260337; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w19si399900pfa.377.2017.03.21.23.31.00; Tue, 21 Mar 2017 23:31:00 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758893AbdCVGal (ORCPT + 16 others); Wed, 22 Mar 2017 02:30:41 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:4446 "EHLO dggrg03-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1758387AbdCVG33 (ORCPT ); Wed, 22 Mar 2017 02:29:29 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKK61223; Wed, 22 Mar 2017 14:29:24 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:13 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 6/7] iommu/iova: move the caculation of pad mask out of loop Date: Wed, 22 Mar 2017 14:27:46 +0800 Message-ID: <1490164067-12552-7-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090201.58D219C5.001E, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 389c04abca8bd0befc69fec56652987d Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I'm not sure whether the compiler can optimize it, but move it out will be better. At least, it does not require lock protection. Signed-off-by: Zhen Lei --- drivers/iommu/iova.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 23abe84..68754e4 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -127,23 +127,16 @@ __cached_rbnode_delete_update(struct iova_domain *iovad, struct iova *free) *cached_node = rb_prev(&free->node); } -/* - * Computes the padding size required, to make the start address - * naturally aligned on the power-of-two order of its size - */ -static unsigned long -iova_get_pad_size(unsigned long size, unsigned long limit_pfn) -{ - return (limit_pfn + 1 - size) & (__roundup_pow_of_two(size) - 1); -} - static int __alloc_and_insert_iova_range(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn, struct iova *new, bool size_aligned) { struct rb_node *prev, *curr; unsigned long flags; - unsigned long pad_size = 0; + unsigned long pad_mask, pad_size = 0; + + if (size_aligned) + pad_mask = __roundup_pow_of_two(size) - 1; /* Walk the tree backwards */ spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); @@ -157,8 +150,13 @@ static int __alloc_and_insert_iova_range(struct iova_domain *iovad, else if (limit_pfn < curr_iova->pfn_hi) goto adjust_limit_pfn; else { + /* + * Computes the padding size required, to make the start + * address naturally aligned on the power-of-two order + * of its size + */ if (size_aligned) - pad_size = iova_get_pad_size(size, limit_pfn); + pad_size = (limit_pfn + 1 - size) & pad_mask; if ((curr_iova->pfn_hi + size + pad_size) <= limit_pfn) break; /* found a free slot */ } From patchwork Wed Mar 22 06:27:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 95698 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp86525qgd; Tue, 21 Mar 2017 23:30:36 -0700 (PDT) X-Received: by 10.84.177.164 with SMTP id x33mr52515718plb.75.1490164236893; Tue, 21 Mar 2017 23:30:36 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t188si564788pgt.331.2017.03.21.23.30.36; Tue, 21 Mar 2017 23:30:36 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758877AbdCVGac (ORCPT + 16 others); Wed, 22 Mar 2017 02:30:32 -0400 Received: from szxga03-in.huawei.com ([45.249.212.189]:4447 "EHLO dggrg03-dlp.huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1758639AbdCVG3a (ORCPT ); Wed, 22 Mar 2017 02:29:30 -0400 Received: from 172.30.72.54 (EHLO DGGEML404-HUB.china.huawei.com) ([172.30.72.54]) by dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AKK61224; Wed, 22 Mar 2017 14:29:24 +0800 (CST) Received: from localhost (10.177.23.164) by DGGEML404-HUB.china.huawei.com (10.3.17.39) with Microsoft SMTP Server id 14.3.301.0; Wed, 22 Mar 2017 14:29:14 +0800 From: Zhen Lei To: Joerg Roedel , iommu , Robin Murphy , David Woodhouse , Sudeep Dutt , Ashutosh Dixit , linux-kernel CC: Zefan Li , Xinwei Hu , "Tianhong Ding" , Hanjun Guo , Zhen Lei Subject: [PATCH 7/7] iommu/iova: fix iovad->dma_32bit_pfn as the last pfn of dma32 Date: Wed, 22 Mar 2017 14:27:47 +0800 Message-ID: <1490164067-12552-8-git-send-email-thunder.leizhen@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.0 In-Reply-To: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> References: <1490164067-12552-1-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.23.164] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090205.58D219C5.002F, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: e52df2ee40a32aafd6a9f72a8db99c90 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To make sure iovad->cached32_node and iovad->cached64_node can exactly control dma32 and dma64 area. It also help us to remove the parameter pfn_32bit of init_iova_domain. Signed-off-by: Zhen Lei --- drivers/iommu/amd_iommu.c | 7 ++----- drivers/iommu/dma-iommu.c | 22 +++++----------------- drivers/iommu/intel-iommu.c | 11 +++-------- drivers/iommu/iova.c | 4 ++-- drivers/misc/mic/scif/scif_rma.c | 3 +-- include/linux/iova.h | 2 +- 6 files changed, 14 insertions(+), 35 deletions(-) -- 2.5.0 diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 98940d1..78c8b93 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -61,7 +61,6 @@ /* IO virtual address start page frame number */ #define IOVA_START_PFN (1) #define IOVA_PFN(addr) ((addr) >> PAGE_SHIFT) -#define DMA_32BIT_PFN IOVA_PFN(DMA_BIT_MASK(32)) /* Reserved IOVA ranges */ #define MSI_RANGE_START (0xfee00000) @@ -1776,8 +1775,7 @@ static struct dma_ops_domain *dma_ops_domain_alloc(void) if (!dma_dom->domain.pt_root) goto free_dma_dom; - init_iova_domain(&dma_dom->iovad, PAGE_SIZE, - IOVA_START_PFN, DMA_32BIT_PFN); + init_iova_domain(&dma_dom->iovad, PAGE_SIZE, IOVA_START_PFN); /* Initialize reserved ranges */ copy_reserved_iova(&reserved_iova_ranges, &dma_dom->iovad); @@ -2747,8 +2745,7 @@ static int init_reserved_iova_ranges(void) struct pci_dev *pdev = NULL; struct iova *val; - init_iova_domain(&reserved_iova_ranges, PAGE_SIZE, - IOVA_START_PFN, DMA_32BIT_PFN); + init_iova_domain(&reserved_iova_ranges, PAGE_SIZE, IOVA_START_PFN); lockdep_set_class(&reserved_iova_ranges.iova_rbtree_lock, &reserved_rbtree_key); diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 48d36ce..7064d32 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -223,18 +223,7 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, /* ...then finally give it a kicking to make sure it fits */ base_pfn = max_t(unsigned long, base_pfn, domain->geometry.aperture_start >> order); - end_pfn = min_t(unsigned long, end_pfn, - domain->geometry.aperture_end >> order); } - /* - * PCI devices may have larger DMA masks, but still prefer allocating - * within a 32-bit mask to avoid DAC addressing. Such limitations don't - * apply to the typical platform device, so for those we may as well - * leave the cache limit at the top of their range to save an rb_last() - * traversal on every allocation. - */ - if (pci) - end_pfn &= DMA_BIT_MASK(32) >> order; /* start_pfn is always nonzero for an already-initialised domain */ if (iovad->start_pfn) { @@ -243,16 +232,15 @@ int iommu_dma_init_domain(struct iommu_domain *domain, dma_addr_t base, pr_warn("Incompatible range for DMA domain\n"); return -EFAULT; } - /* - * If we have devices with different DMA masks, move the free - * area cache limit down for the benefit of the smaller one. - */ - iovad->dma_32bit_pfn = min(end_pfn, iovad->dma_32bit_pfn); } else { - init_iova_domain(iovad, 1UL << order, base_pfn, end_pfn); + init_iova_domain(iovad, 1UL << order, base_pfn); if (pci) iova_reserve_pci_windows(to_pci_dev(dev), iovad); } + + if (end_pfn < iovad->dma_32bit_pfn) + dev_dbg(dev, "ancient device or dma range missed some bits?"); + return 0; } EXPORT_SYMBOL(iommu_dma_init_domain); diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 238ad34..de467c1 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -82,8 +82,6 @@ #define IOVA_START_PFN (1) #define IOVA_PFN(addr) ((addr) >> PAGE_SHIFT) -#define DMA_32BIT_PFN IOVA_PFN(DMA_BIT_MASK(32)) -#define DMA_64BIT_PFN IOVA_PFN(DMA_BIT_MASK(64)) /* page table handling */ #define LEVEL_STRIDE (9) @@ -1869,8 +1867,7 @@ static int dmar_init_reserved_ranges(void) struct iova *iova; int i; - init_iova_domain(&reserved_iova_list, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&reserved_iova_list, VTD_PAGE_SIZE, IOVA_START_PFN); lockdep_set_class(&reserved_iova_list.iova_rbtree_lock, &reserved_rbtree_key); @@ -1928,8 +1925,7 @@ static int domain_init(struct dmar_domain *domain, struct intel_iommu *iommu, int adjust_width, agaw; unsigned long sagaw; - init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN); domain_reserve_special_ranges(domain); /* calculate AGAW */ @@ -4965,8 +4961,7 @@ static int md_domain_init(struct dmar_domain *domain, int guest_width) { int adjust_width; - init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN, - DMA_32BIT_PFN); + init_iova_domain(&domain->iovad, VTD_PAGE_SIZE, IOVA_START_PFN); domain_reserve_special_ranges(domain); /* calculate AGAW */ diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 68754e4..43fd8e9 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -50,7 +50,7 @@ insert_iova_boundary(struct iova_domain *iovad) void init_iova_domain(struct iova_domain *iovad, unsigned long granule, - unsigned long start_pfn, unsigned long pfn_32bit) + unsigned long start_pfn) { /* * IOVA granularity will normally be equal to the smallest @@ -63,7 +63,7 @@ init_iova_domain(struct iova_domain *iovad, unsigned long granule, iovad->rbroot = RB_ROOT; iovad->granule = granule; iovad->start_pfn = start_pfn; - iovad->dma_32bit_pfn = pfn_32bit; + iovad->dma_32bit_pfn = DMA_BIT_MASK(32) >> ilog2(granule); init_iova_rcaches(iovad); /* diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c index 329727e..c824329 100644 --- a/drivers/misc/mic/scif/scif_rma.c +++ b/drivers/misc/mic/scif/scif_rma.c @@ -39,8 +39,7 @@ void scif_rma_ep_init(struct scif_endpt *ep) struct scif_endpt_rma_info *rma = &ep->rma_info; mutex_init(&rma->rma_lock); - init_iova_domain(&rma->iovad, PAGE_SIZE, SCIF_IOVA_START_PFN, - SCIF_DMA_64BIT_PFN); + init_iova_domain(&rma->iovad, PAGE_SIZE, SCIF_IOVA_START_PFN); spin_lock_init(&rma->tc_lock); mutex_init(&rma->mmn_lock); INIT_LIST_HEAD(&rma->reg_list); diff --git a/include/linux/iova.h b/include/linux/iova.h index 844d723..6f037b2 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -101,7 +101,7 @@ struct iova *reserve_iova(struct iova_domain *iovad, unsigned long pfn_lo, unsigned long pfn_hi); void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to); void init_iova_domain(struct iova_domain *iovad, unsigned long granule, - unsigned long start_pfn, unsigned long pfn_32bit); + unsigned long start_pfn); struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn); void put_iova_domain(struct iova_domain *iovad); struct iova *split_and_remove_iova(struct iova_domain *iovad,