From patchwork Wed Feb 21 12:22:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 129058 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp562110ljc; Wed, 21 Feb 2018 04:24:36 -0800 (PST) X-Google-Smtp-Source: AH8x227ZdyGtCj57qGCd12WCmuuEtvKG8YKCjEQRo0FdPCoJuSonVf+JCUai2YegT3+Xn0/LS4Jt X-Received: by 2002:a17:902:2006:: with SMTP id n6-v6mr2993541pla.149.1519215876728; Wed, 21 Feb 2018 04:24:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519215876; cv=none; d=google.com; s=arc-20160816; b=pdGyPqqagiRZwUxYNG19+H7D0NFinXNFSDVF0OcsPWW+mo1bVhJwxGXTZccW8aJj62 nNrgyaV/f2F0tZVwFTOgEk7PmtmIJL++KkgA2JmneAwZvD++19WBja28Y10RTtvH0/yd ROHqB7vO7Q5sv53dBFBgp6l3HpTlHBkhHtD7oOzkaa0sobrTKdwVx1AML60jOxdd2asm I/U1JgdRBiHRTxaHS64vHy3sWY39gQUQ/w7gpk7USgsWMK+e0Iya0XlHtc5oTm96T13m M6nsBaAtnBKSunOOMEk4v5W+52MD/5IMoW3nUBs6zLutyp193o5ThAhmSWok7Jfcb95m nRsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=7IiRlOmivD2W5GkXFDLmgEJyrb1YA6mfb29IOKdoa2w=; b=HrHEgRsAvPhFbEWkgjfmfXO0Ki1YsGYb4LgyEoaM533JP+xQmqoDJapXr0S617/+cS 7rvWPIyQ+Yl/zSnRfgyPVDtbXye2VQCL4Jh4jGLHtYeRBOWRlm6pLmrF/ryDHAwggV7w QSnuDWYbir2DPNJLfauNUDzWhw1nEJPO/MhpT2EnNUvpj4lyi8qUwVJ6gHCD9QnlGIMm lyGryhg7epC6KAzhdd4c185FtK84LmxwQT79bmB8l8VMWk25TbhOHxhUPoHNNnhAkmgL 6vqWTqCOpbsHT3sNIQ1vvjGAfqz8Z5+cWEzYLk/7qvGWh0RUiFVGNi5aGaN0vRAJGpdo c7qA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u6-v6si9168144pld.31.2018.02.21.04.24.36; Wed, 21 Feb 2018 04:24:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754192AbeBUMYT (ORCPT + 28 others); Wed, 21 Feb 2018 07:24:19 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:5255 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752350AbeBUMXk (ORCPT ); Wed, 21 Feb 2018 07:23:40 -0500 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id D2DF01E0430C9; Wed, 21 Feb 2018 20:23:14 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.361.1; Wed, 21 Feb 2018 20:23:09 +0800 From: Shameer Kolothum To: , , CC: , , , , , Shameer Kolothum Subject: [PATCH v4 1/6] vfio/type1: Introduce iova list and add iommu aperture validity check Date: Wed, 21 Feb 2018 12:22:04 +0000 Message-ID: <20180221122209.9292-2-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180221122209.9292-1-shameerali.kolothum.thodi@huawei.com> References: <20180221122209.9292-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This introduces an iova list that is valid for dma mappings. Make sure the new iommu aperture window doesn't conflict with the current one or with any existing dma mappings during attach. Signed-off-by: Shameer Kolothum --- drivers/vfio/vfio_iommu_type1.c | 183 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 180 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index e30e29a..ac6772d 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -60,6 +60,7 @@ MODULE_PARM_DESC(disable_hugepages, struct vfio_iommu { struct list_head domain_list; + struct list_head iova_list; struct vfio_domain *external_domain; /* domain for external user */ struct mutex lock; struct rb_root dma_list; @@ -92,6 +93,12 @@ struct vfio_group { struct list_head next; }; +struct vfio_iova { + struct list_head list; + dma_addr_t start; + dma_addr_t end; +}; + /* * Guest RAM pinning working set or DMA target */ @@ -1192,6 +1199,149 @@ static bool vfio_iommu_has_sw_msi(struct iommu_group *group, phys_addr_t *base) return ret; } +/* + * This is a helper function to insert an address range to iova list. + * The list starts with a single entry corresponding to the IOMMU + * domain geometry to which the device group is attached. The list + * aperture gets modified when a new domain is added to the container + * if the new aperture doesn't conflict with the current one or with + * any existing dma mappings. The list is also modified to exclude + * any reserved regions associated with the device group. + */ +static int vfio_iommu_iova_insert(struct list_head *head, + dma_addr_t start, dma_addr_t end) +{ + struct vfio_iova *region; + + region = kmalloc(sizeof(*region), GFP_KERNEL); + if (!region) + return -ENOMEM; + + INIT_LIST_HEAD(®ion->list); + region->start = start; + region->end = end; + + list_add_tail(®ion->list, head); + return 0; +} + +/* + * Check the new iommu aperture conflicts with existing aper or with any + * existing dma mappings. + */ +static bool vfio_iommu_aper_conflict(struct vfio_iommu *iommu, + dma_addr_t start, dma_addr_t end) +{ + struct vfio_iova *first, *last; + struct list_head *iova = &iommu->iova_list; + + if (list_empty(iova)) + return false; + + /* Disjoint sets, return conflict */ + first = list_first_entry(iova, struct vfio_iova, list); + last = list_last_entry(iova, struct vfio_iova, list); + if ((start > last->end) || (end < first->start)) + return true; + + /* Check for any existing dma mappings outside the new start */ + if (start > first->start) { + if (vfio_find_dma(iommu, first->start, start - first->start)) + return true; + } + + /* Check for any existing dma mappings outside the new end */ + if (end < last->end) { + if (vfio_find_dma(iommu, end + 1, last->end - end)) + return true; + } + + return false; +} + +/* + * Resize iommu iova aperture window. This is called only if the new + * aperture has no conflict with existing aperture and dma mappings. + */ +static int vfio_iommu_aper_resize(struct list_head *iova, + dma_addr_t start, + dma_addr_t end) +{ + struct vfio_iova *node, *next; + + if (list_empty(iova)) + return vfio_iommu_iova_insert(iova, start, end); + + /* Adjust iova list start */ + list_for_each_entry_safe(node, next, iova, list) { + if (start < node->start) + break; + if ((start >= node->start) && (start < node->end)) { + node->start = start; + break; + } + /* Delete nodes before new start */ + list_del(&node->list); + kfree(node); + } + + /* Adjust iova list end */ + list_for_each_entry_safe(node, next, iova, list) { + if (end > node->end) + continue; + if ((end > node->start) && (end <= node->end)) { + node->end = end; + continue; + } + /* Delete nodes after new end */ + list_del(&node->list); + kfree(node); + } + + return 0; +} + +static void vfio_iommu_iova_free(struct list_head *iova) +{ + struct vfio_iova *n, *next; + + list_for_each_entry_safe(n, next, iova, list) { + list_del(&n->list); + kfree(n); + } +} + +static int vfio_iommu_iova_get_copy(struct vfio_iommu *iommu, + struct list_head *iova_copy) +{ + + struct list_head *iova = &iommu->iova_list; + struct vfio_iova *n; + int ret; + + list_for_each_entry(n, iova, list) { + ret = vfio_iommu_iova_insert(iova_copy, n->start, n->end); + if (ret) + goto out_free; + } + + return 0; + +out_free: + vfio_iommu_iova_free(iova_copy); + return ret; +} + +static void vfio_iommu_iova_insert_copy(struct vfio_iommu *iommu, + struct list_head *iova_copy) +{ + struct list_head *iova = &iommu->iova_list; + + vfio_iommu_iova_free(iova); + + list_splice_tail(iova_copy, iova); +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group) { @@ -1202,6 +1352,8 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, int ret; bool resv_msi, msi_remap; phys_addr_t resv_msi_base; + struct iommu_domain_geometry geo; + LIST_HEAD(iova_copy); mutex_lock(&iommu->lock); @@ -1271,6 +1423,25 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, if (ret) goto out_domain; + /* Get aperture info */ + iommu_domain_get_attr(domain->domain, DOMAIN_ATTR_GEOMETRY, &geo); + + if (vfio_iommu_aper_conflict(iommu, geo.aperture_start, + geo.aperture_end)) { + ret = -EINVAL; + goto out_detach; + } + + /* Get a copy of the current iova list and work on it */ + ret = vfio_iommu_iova_get_copy(iommu, &iova_copy); + if (ret) + goto out_detach; + + ret = vfio_iommu_aper_resize(&iova_copy, geo.aperture_start, + geo.aperture_end); + if (ret) + goto out_detach; + resv_msi = vfio_iommu_has_sw_msi(iommu_group, &resv_msi_base); INIT_LIST_HEAD(&domain->group_list); @@ -1304,8 +1475,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, list_add(&group->next, &d->group_list); iommu_domain_free(domain->domain); kfree(domain); - mutex_unlock(&iommu->lock); - return 0; + goto done; } ret = iommu_attach_group(domain->domain, iommu_group); @@ -1328,7 +1498,9 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } list_add(&domain->next, &iommu->domain_list); - +done: + /* Delete the old one and insert new iova list */ + vfio_iommu_iova_insert_copy(iommu, &iova_copy); mutex_unlock(&iommu->lock); return 0; @@ -1337,6 +1509,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, iommu_detach_group(domain->domain, iommu_group); out_domain: iommu_domain_free(domain->domain); + vfio_iommu_iova_free(&iova_copy); out_free: kfree(domain); kfree(group); @@ -1475,6 +1648,7 @@ static void *vfio_iommu_type1_open(unsigned long arg) } INIT_LIST_HEAD(&iommu->domain_list); + INIT_LIST_HEAD(&iommu->iova_list); iommu->dma_list = RB_ROOT; mutex_init(&iommu->lock); BLOCKING_INIT_NOTIFIER_HEAD(&iommu->notifier); @@ -1517,6 +1691,9 @@ static void vfio_iommu_type1_release(void *iommu_data) list_del(&domain->next); kfree(domain); } + + vfio_iommu_iova_free(&iommu->iova_list); + kfree(iommu); }