From patchwork Tue Feb 25 19:39:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 868425 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2069.outbound.protection.outlook.com [40.107.243.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB08D20E310; Tue, 25 Feb 2025 19:39:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740512393; cv=fail; b=SlJoCfGh/DZNZUDdAKFf4ao8ime9zcEgyWSBMyaai02CmS0FTdQ7FOHhI2tuycHM+MqaTiHRk/CxaFWmaroKPKJfifnN1aTn8vYdbXiM0FEltdZrzte4xTG0vDmeXpMRnQyVylExzaOZIW5ZI2GyPYEjr8K71DLN7+cdDRnax1g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740512393; c=relaxed/simple; bh=MAQJyHIFJE0fZwdG5kb4SOY8UnhphrutU4CA3Qm28Ns=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=obgQjEqYr1aUYBW4T5PXPamVDgjKWvx53gSi/0ocI/Urd040mSSdJcur8X+XAoFDSAy+9WaUEmicXnJU/s+d2GLl0kCGTEGa5EjcSPw+Hmv37uAVQHdnJOspxZwk9f6lLsYuMd8PJrXgOwr5B4/Wt3VQ4HL92F98XXoMhzDk7vI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=awH3WoUY; arc=fail smtp.client-ip=40.107.243.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="awH3WoUY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VaFHtjdSBLSQXzGAXoDI+29QqlB6VijAl/R3g3g5kFp87b55tKiImf5okjeESa33Jk6PRYY9x6DWqo1IGJOhlY9oYUpSRlI6LwTyJLvXoFkvv+QOW8l5yTfEseqNaYdAZE4Cw0/Vb9g3YKLqkhss/CmA5XytGRD7CC+bO/21nQqDMo6ki7qXEP8Obvwu0EiDsDGINY5cNIj9/9VXE7wIX8LnF03YhxMUKP3PhlLpH/m2NEKletmhr8wgbgCvnb8LZwTAUnfJDwcNgABdzM2Z0bCnhzUBE34q7mwvM170goPQLXx5nyuHhyZl2V13L+ZNnpufVoaG42Dopenq4BYprA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yV1vI6+BTwDkONdyALdON90gjpfpGH6NQjDVCww9SRY=; b=OVxByPW4gC+Nt6N1zZrMc/pW5V3X6IT/mAc3I36J4bdyc9lPxtkZQo7iWOnncFqNj1tpSk9sb2LE4NhTu+2evucONhHohQ/y7xV55fltGPLT/cqPWA20e2H2XLXH9z+KPU8axp2diuRaqXcHQHHr2GIBkUq0gXxdT/JrAEH5nSOvkPzX/KUVW40+MNy3zLy/kafmpPWLkR1BXw30/N+gBwjs/6xtJN87AnX68Y/Rc8PGHrLvRFGiQ+ul/w77kKxgCtM+i+lFrHVJKvjS0452i07Hnh/msEDG67drpdXqkFYRpC/x1qIdotJo31tTYKIBXAqHiJBiV2KsmVNMNG0Yvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yV1vI6+BTwDkONdyALdON90gjpfpGH6NQjDVCww9SRY=; b=awH3WoUYQ2M+AIaVl9I038Pl0I3HfR5EQgXTLICfBZn4vK6HdBpy8eZqprmtsT4GDRO1gBiIzCn6Jz/2LKTMPP1xGcQB9o2dpMi9uB29rFh1sTwhPSciWKmtPgc898jqTkYG1QYghacdwF8Hw7eot6DEZQw0WvHQMnQd38p60OjiynQzCdmswePjfF7qRyAQ4Fj3Te+7nG9qiXv5sou18H+r00F49pUjE2yNUqgDndhkOzaP8KFcdDo6ViGwtatMtCSt7rD41Xjmn1Y4cPnQIUX69yZhqLXhpsHdgvxAGptJJKzz5VoB3mrLYL2H7Z8+FzMXy9IXDEklxlSUV8q9Zg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by SA1PR12MB5616.namprd12.prod.outlook.com (2603:10b6:806:22a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.18; Tue, 25 Feb 2025 19:39:44 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8466.016; Tue, 25 Feb 2025 19:39:44 +0000 From: Jason Gunthorpe To: Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, Lu Baolu , David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Hector Martin , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Bagas Sanjaya , Joerg Roedel , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Matthew Wilcox Subject: [PATCH v3 14/23] iommu/pages: Move from struct page to struct ioptdesc and folio Date: Tue, 25 Feb 2025 15:39:31 -0400 Message-ID: <14-v3-e797f4dc6918+93057-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v3-e797f4dc6918+93057-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: YQBPR01CA0005.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01::13) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|SA1PR12MB5616:EE_ X-MS-Office365-Filtering-Correlation-Id: 8b12caa8-7dd8-446e-4ead-08dd55d42687 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|366016|1800799024|921020; X-Microsoft-Antispam-Message-Info: untI9P5ODbilD7i2BCmgVrFefAn81uFGLJp6va/v9RQb4LMdAYa8av+C5gtw4YbNRDI//Q7UVos2vQfKcmlvQUS4K/uwWnFWk7E5kBP13r1YYzizLAuDWGyYUp3Pgnsyuiax1B9jbA9cRvAsHXriw48GFLwLfme+MHj7Ozxmv+o13ofhciaNq2oFGqYZ1aoNMBpwU5W3JqQvvCKVum6AaXTznROqI22fU+3Mw/Uhb04IEoZGFTC/qLNShkAjWtRQ23H1GNrerc3wr72r9kodtnjr+xCH4Wdmo3M0Eq7u/2FtrlJZ9MYvaDdCAaDOtgWFPwxfnNulCvyQI/n+TOr027VAAoPMmNQZcdtWhClUjWAJV/5HRHx88etfG6OmlzuYnxi+HRDEjdDLiJNGb9caeBJ2iH6Ru0O3imw72v9NZW1iSD6MSyDdTYiuOcjvkTIZsAltl84ORnm+ByU+mbOgRGAXUe5QXR1niU4hFLOq2fOdZx6Quytig1mWxT8VOcVtIk5hcxQDszoNKzz7nynXYzAs/iifMWu+UsG34vaTo7d6pXPH3OdkOtaOVFq7eydz09rpKfhocKXP6hCXub0oEBkt/uQFQGyFv1YYNOa9TIWQFrMdNo4KEhRD40wNMTVGqPSo893yaTmwCYa4sEfVjbTaTCbTi87vIyUSR5PCrQb0/abgnkhWEPd9mpUelNVrVgIVjINjWqzvV4vQBLSzO7uzejW9R6kwzo68BL47je20maXIoUpAAgvHmJjB9aARzfhqivaqW0j9/aUvqSEbwVe9dZKeRk1LAUbGfBaEe9039AwUxwHFBn5z9wL8+Jcf/0dJpPG49UsXD5ZKJQ5+XA30M1+DELp74/vH3N78CxKdrndAx5Zx2kzhZjX5B7RHzAD9QViXXDD6YWzbzDZzGSnsqdpbUwNkzuwoIBSP+voDMusQBuk7JS7XvweNLAgozAEEvxOXe5b/90khLwjVUZ5KSiZQre/M3mCKNfvWsMTdJZ/fCwQDBQdxnn5PuJIDn/2yr0kwSrWIOGWCbDOiGYSm+xvXKICBimfZ5VcGWh0Xxc2bQFvZyIIEC6x9DuvHFJgZXit5TauFHflATry2jAB6VJ3k1/w3CYHx8IDHQl7e9cQ9nxvj4spexudeUr55H/up5Kc4a7ou8Cg5cX/iAhinsmc+M7CH+4yV36PzP8b3qDTyb+NFuQuot+j5WArVdH89jC7qCiTqC9VlMGyxM1ewhGHi3NweZTycW5j++zIPDRmcEnXxvezXgqaTAUfT4rEs6+/OYbig0bUf78GwhQT5O9SBR9GMPhb+pRA2f3z3ngS2tYSWvSOQFpQ8J8MiuujFVfrHracndCZHH3L4uwM3cMqsimQZmKpfpnDGKHiNiQ95iYzbt+VezquYzHBqBdL5Y/ZPPNwDyp8ScI/pdIKcDGVGJuRcX1G7pZTbLao= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(7416014)(376014)(366016)(1800799024)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6rNnkpIQC9zQ6qzvOgi0jfG6Xh1On0CZran+LOO9VIryHETXNb+K1RH1ZuFvJOkwOq9oeBdJRovzmY7KDdalaWGB6MATYxv3ThimJK8uhOCB87JmmiKRcbJZgl09fnSWGTqVtNItUas0AjKnAYO8DfZAZFKndp9snj2IwfJNEtSIQMu1IKd8Eztar0Z5Pn/D5gDpsYaWy7ZqFAl0K/4ZWwEHoSN9Do0nu5FsEQH6IT+Nn1zQ94wwSMm7qSf1Xp9lF+UqJLGEHO1Hpkai3lIDOJoiTS7ewurivFEJg8Pslg9X71vZ8kOGJ9OVDBhes265MvT3JoFKSol1wymPyGjZxP4uDbwJ9YGB1EWk6eCJFEg4/TO6hINrXeUn/OGdR8TaXNP4K6KGIFpQpoI5JKZwklf2Gs9SWr2asbq5UrExTEQHY7PAEU7vQWV6Zh/YqnChXlci0ghSka4dwipeNz+ZQH5Im5tJbPCxLAA039KG3hTnp8LvnQN3BIMhCqU2vpGVkQkGpnR0wxp6wUr47gpz4Hi+drnjKF/6JxW03ll1t+p9CvIQEjRHmKVaYuAtzSCEd07Od1f467Drr6+nEnoC08iXaGdrjww0rEkPtUmJAQCbqn+u2F3/45gLtQG2fBl9A2jVCdn1ZUUZlUK81jYi3teWqj8zl9xGFVaVIgFBLHP51leqOCkt8CRLiVWzFcUO3vVM0y9cqjh9rOJmPqXa8VkR4e7ely4vS6PKf0SKclwYy0lyEeMaDsUQrzVDTmBugzbF2s3obBpHHaIgjSneu+7XmjmxNBcFZYREsGndybuVkJT92bgsbDktA+Txdpu/CYGX5TlDcEXjOItIg7jJ5Nve5urHVOmUFDwcng90yLnBCvsJDRm8mj1MDUWPVvbWnPWO8R8njHyKEcoAOg9SJ2eWsekg3aUsfJTwt5bmJsDT2O7FZHxRiTomnZCaLuZv6P7dYZl4NPs+pgKTY4eE4vmJ5xnbAMWwIs8wMLn9sZBvr6Hdd0IN1ar+kNxGAvxBHZXnFu6XpyfuPOPPmSzjmTGqjxdPWV/elvLtETAPaPn1M/BnP7NXR2NmLwEQVRRtA4JKoLVBarKljY04w15ch4x+V5W1Nu7rm103GCVf7qfWgsmbzCZDMwVPPMCRzq5GvJn6lqvsxhdmqXmEOfjs+AeTZ8Upq1dik/r9EmcwDpQWRRSaFTN9XW8W0byclNGxLMAi7V5EXkfsRthAcobSZ/1vHPQZ9XT9cF2386HWUVpiNDpqkaXa6iY2VtegQU0qtEKsf4ZQggDCi9i6ZBwK8w6YOAIoludmr6K5X5zMkCEcw7GFP3gc0eme1vSe9wGbIWaZalNX76bExgM/rkhWLwcewIF0qWB+U7RZoXd+3+0hJkrZJk+yUo2J85eyZqvdKJmC18Vj7K+pmVcfCaEW3tr8lJ4Zo/NHQ44x0Ey+YAVu+eAqRTw+2UedBeY7GS0zre1hbpGyVnTZA+ZdADr/nwGtWLOYSFDEEnXTKaKcr7j3NjvmKSal4+YwJzFRmIxzasLn3fkDT1+pIKvcjirw7KDNfHHgxHrNc6rTOeceCE2Y9cNi1lD8HHrllKNQjt82 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8b12caa8-7dd8-446e-4ead-08dd55d42687 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2025 19:39:42.6868 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IpiXO6+rAaYmYtzQ8D5YUhjAm5ERCz/D7IV9dx7gIIDYO8kFY51LhjNunkUPq69l X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB5616 This brings the iommu page table allocator into the modern world of having its own private page descriptor and not re-using fields from struct page for its own purpose. It follows the basic pattern of struct ptdesc which did this transformation for the CPU page table allocator. Currently iommu-pages is pretty basic so this isn't a huge benefit, however I see a coming need for features that CPU allocator has, like sub PAGE_SIZE allocations, and RCU freeing. This provides the base infrastructure to implement those cleanly. Remove numa_node_id() calls from the inlines and instead use NUMA_NO_NODE which will get switched to numa_mem_id(), which seems to be the right ID to use for memory allocations. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.c | 54 ++++++++++++++++++++++++++----------- drivers/iommu/iommu-pages.h | 43 ++++++++++++++++++++++++++--- 2 files changed, 78 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index 6eacb6a34586a6..3077df642adb1f 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -7,6 +7,21 @@ #include #include +#define IOPTDESC_MATCH(pg_elm, elm) \ + static_assert(offsetof(struct page, pg_elm) == \ + offsetof(struct ioptdesc, elm)) +IOPTDESC_MATCH(flags, __page_flags); +IOPTDESC_MATCH(lru, iopt_freelist_elm); /* Ensure bit 0 is clear */ +IOPTDESC_MATCH(mapping, __page_mapping); +IOPTDESC_MATCH(private, _private); +IOPTDESC_MATCH(page_type, __page_type); +IOPTDESC_MATCH(_refcount, __page_refcount); +#ifdef CONFIG_MEMCG +IOPTDESC_MATCH(memcg_data, memcg_data); +#endif +#undef IOPTDESC_MATCH +static_assert(sizeof(struct ioptdesc) <= sizeof(struct page)); + /** * iommu_alloc_pages_node - Allocate a zeroed page of a given order from * specific NUMA node @@ -20,10 +35,17 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) { const unsigned long pgcnt = 1UL << order; - struct page *page; + struct folio *folio; - page = alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); - if (unlikely(!page)) + /* + * __folio_alloc_node() does not handle NUMA_NO_NODE like + * alloc_pages_node() did. + */ + if (nid == NUMA_NO_NODE) + nid = numa_mem_id(); + + folio = __folio_alloc_node(gfp | __GFP_ZERO, order, nid); + if (unlikely(!folio)) return NULL; /* @@ -35,21 +57,21 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) * This is necessary for the proper accounting as IOMMU state can be * rather large, i.e. multiple gigabytes in size. */ - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, pgcnt); + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, pgcnt); + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, pgcnt); - return page_address(page); + return folio_address(folio); } EXPORT_SYMBOL_GPL(iommu_alloc_pages_node); -static void __iommu_free_page(struct page *page) +static void __iommu_free_desc(struct ioptdesc *iopt) { - unsigned int order = folio_order(page_folio(page)); - const unsigned long pgcnt = 1UL << order; + struct folio *folio = ioptdesc_folio(iopt); + const unsigned long pgcnt = 1UL << folio_order(folio); - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, -pgcnt); - put_page(page); + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, -pgcnt); + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, -pgcnt); + folio_put(folio); } /** @@ -62,7 +84,7 @@ void iommu_free_pages(void *virt) { if (!virt) return; - __iommu_free_page(virt_to_page(virt)); + __iommu_free_desc(virt_to_ioptdesc(virt)); } EXPORT_SYMBOL_GPL(iommu_free_pages); @@ -74,9 +96,9 @@ EXPORT_SYMBOL_GPL(iommu_free_pages); */ void iommu_put_pages_list(struct iommu_pages_list *list) { - struct page *p, *tmp; + struct ioptdesc *iopt, *tmp; - list_for_each_entry_safe(p, tmp, &list->pages, lru) - __iommu_free_page(p); + list_for_each_entry_safe(iopt, tmp, &list->pages, iopt_freelist_elm) + __iommu_free_desc(iopt); } EXPORT_SYMBOL_GPL(iommu_put_pages_list); diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 8dc0202bf108e4..f4578f252e2580 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -9,6 +9,43 @@ #include +/** + * struct ioptdesc - Memory descriptor for IOMMU page tables + * @iopt_freelist_elm: List element for a struct iommu_pages_list + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct ioptdesc { + unsigned long __page_flags; + + struct list_head iopt_freelist_elm; + unsigned long __page_mapping; + pgoff_t __index; + void *_private; + + unsigned int __page_type; + atomic_t __page_refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; + +static inline struct ioptdesc *folio_ioptdesc(struct folio *folio) +{ + return (struct ioptdesc *)folio; +} + +static inline struct folio *ioptdesc_folio(struct ioptdesc *iopt) +{ + return (struct folio *)iopt; +} + +static inline struct ioptdesc *virt_to_ioptdesc(void *virt) +{ + return folio_ioptdesc(virt_to_folio(virt)); +} + void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order); void iommu_free_pages(void *virt); void iommu_put_pages_list(struct iommu_pages_list *list); @@ -21,7 +58,7 @@ void iommu_put_pages_list(struct iommu_pages_list *list); static inline void iommu_pages_list_add(struct iommu_pages_list *list, void *virt) { - list_add_tail(&virt_to_page(virt)->lru, &list->pages); + list_add_tail(&virt_to_ioptdesc(virt)->iopt_freelist_elm, &list->pages); } /** @@ -56,7 +93,7 @@ static inline bool iommu_pages_list_empty(struct iommu_pages_list *list) */ static inline void *iommu_alloc_pages(gfp_t gfp, int order) { - return iommu_alloc_pages_node(numa_node_id(), gfp, order); + return iommu_alloc_pages_node(NUMA_NO_NODE, gfp, order); } /** @@ -79,7 +116,7 @@ static inline void *iommu_alloc_page_node(int nid, gfp_t gfp) */ static inline void *iommu_alloc_page(gfp_t gfp) { - return iommu_alloc_pages_node(numa_node_id(), gfp, 0); + return iommu_alloc_pages_node(NUMA_NO_NODE, gfp, 0); } #endif /* __IOMMU_PAGES_H */