From patchwork Tue Apr 8 16:53:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880053 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2079.outbound.protection.outlook.com [40.107.94.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88A1423645F; Tue, 8 Apr 2025 16:54:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131262; cv=fail; b=cf/XniCtOREGVoFOR2czqpAdEp14fNjxPoPljnxDYErc84lSZQsARbS2so5Uh+/EP1cilclB094q9jDVR8gAe/4pq0OmtqDjcQXCaOAsEkP5nDVkijeFsR6qHRehEJaL/rnA2g6SZ2ohBdunfUrf5ZfgRCc+xLfW5rCBsLGCKAM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131262; c=relaxed/simple; bh=Pj98uWU8/aBqFXLf6U44nb4QAic+SncYWee6K+ipXdg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Iu7yjzR70sOaMbhsNALR87NcYcBvoJr5x4lFHepOGMcOo+VXBfxgjHdSdEt4PjH74xN/7XbCelMSTrRX3nbsHGDQAuQgNS/EnipbcRqWTIkkNIiwUgtgJessH06LLQFZ9K/O62hDtRf30nbQXXvQg99wHzFA1prkDxpnjIwoQw4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=b6iAFueF; arc=fail smtp.client-ip=40.107.94.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="b6iAFueF" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gtmq+2JsOF+d90CuS1KP+gOHv3TpTDDmb2ggsdBuH1jO0PFyiNCnR/Ca7SK0r9SQHsqgIrwqmsCN90davKl6WjVhClEUdAafqtQZcKBFiVm21gd4oILkbqd76FJs32MiiL9QJMksxLpVFlvlRe9BgvoeCm6CSShfXr1GWb0Ggs03ltK7XCvanG7zPSS/diI/VP10s4CH7zEo+QTAlqSplb+RAV+IgzJHE4j9Ewq6pJ+1+r5J/mL5fdxu+XfsaZYscs9cA+ZdPYx6sOxW3+IAShrBQvv1M98DCETMQD0rygBzU3peoB6ZeaGa+q028lgzcPr1PyWwj7scwLiJgXCWJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H8Sx4u+V1hmtNT/5wI7YFShWV6OQmD25KxCjs0xg2Ck=; b=bMRPS7CCVLPIvSUJsis9VUbHLfpigjEmOrscwrqpoDXeDOY2yQrtZRSPxXVlh6VePLPyXxT7Plc7j41/+9XlcB6ptn3Q0/3OSBYb6uxe5R3NC3IFT2KAxsilIsSri/JiYA8cln6RIgsVNi7aFjbgPVx/u5VjprlnRWDZqOElNWaxT42hIPYKvfITMtvn+AWOWMdzlkwiw38H6/4kiFKdlbbR/BXkvRpuh3zNI9m1zmqlNyBHCJ4xFA77VKX9N7yemx3y2OuXtJw5lVJQ7jJHlRq3vDybBpVaSXq1i/0UNQlxZswWbyBF5iAjAJ3Kd02rrdv3Sb5JXi+fSBp+axTPiw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H8Sx4u+V1hmtNT/5wI7YFShWV6OQmD25KxCjs0xg2Ck=; b=b6iAFueFTvKBap3K3qfZNZ9jQiEf2h2tpAPstxTfeHc7ewJvqvvC0rxUhS7mRgf6tOQziXJSuR7YB61b2gx43kHX5jvIImvjqBsCf7K/2kYatR2SQxDYnuA0FuohpVB0aGNg9+cX1qmDIhqEUIHit09QSmC38zuaqSsv+heHF9ZBENB7vmcirw0hVLAlsN2dugNlx/4qz12pNHx4wO3pJ4jqtv2/3VXsxGKfeRbmGH1Cbg56wRAWwfeRxVA/XrvrEPn4tvFiibFL14G4kL0lI712zxolLVx8IECT+4C+rhcG7Ud8CncyjRt7uZBMtbrjFgB752HJlck3uWuNwkOSUA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:14 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:14 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 01/23] iommu/terga: Do not use struct page as the handle for as->pd memory Date: Tue, 8 Apr 2025 13:53:49 -0300 Message-ID: <1-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0039.namprd03.prod.outlook.com (2603:10b6:208:32d::14) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 2465f73e-60ce-423f-8654-08dd76bdfd83 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: Qt7zIo5lfqz4sbprk+J14DHhiwjq0LRxE+TDV5Gkj4TLkNxQ7trM+PyZcf4aSFrFYCrhVIjPMguz7VzmlrYmtAJkSOjqHIW80nVvqJDobTakUg8RiH3EGLF1yxCDmFX2gDmqO8cXnjvbQmMD+32tgrWo+1i16UI4aLoK6BrvEPCB86UZ/nhu4J5Gxn6xkB/nUN/+oywkxHBrA9Iu8mfCL4fA0gDi+lI/XZKI+LEh7EJahYyox5CUpH0TvpE54ra6QvvJoAvEFya+F+Q3uWckkMqAxBJs84x4p1qk6QArWezTJVRwqZYFJ9n5dzzOO2KUtixDQPEeBmRtNDPhgBfnOPbZO/w7Nhg/5ZW9T0hCDWTdo4k8qtOM8T+HmoxxWeL4QSt/MVF+qBVUdDG7V4oBLFK1oHWrbKDMFX8yOB0JPhScWNA6I4KS/6xeWOOsW2DKrTlWy366GMpuCu8cMGnepKiWytKPYfS9L/xrKsp6ZdbbSNiV+s4lrKNCLyOCr16El/xJyt98VNLHcLND+JzQDchYSZlLftsSS8yhsYAC2DFi11EoGiyjrlIzXN8nN976FSnEfWuTqQsKLGc9W5eKFcDTuqqMt5c7gQi9ofpUF2s0u7PQNnBqFKdtb1LZ6DUN47cKMpuBzJ/nh9ReV9KfOMzm75+lbR0Cz3p/uMUMeEemBU8e+/+wY3oo+QWceKZ06b+j2TRRDVliV04SBvmuVeyxwtkTb30ypimrTA7eldgSu9cH2FJN3BWnlFbXGehnPgIis25k3Z9H4z9jOazA6Bg7ZGZdZMQFtP3kw2ymucPEVlvO86UTg4x70SGQgrfc+/UY4oURJ/zwhJ94jybCZkfgOX//+a9e3EwJE+fdDUAmdkj0zYEO+pVDFvRBLzeS8od6/oTMb5hUGQPDcTUTPlFjTM5r8hXUHQt3Wc+CdZbK8PjLY7stzX80ORJzoS3NGGsJXCOWeZsLCa39hQtmPqHgvZcidaCsvWJiaAtWlXyXH4PxFp6CddIKkMdYgQ3otaKYjUgyB50xHnY2VoJsSIUr3xkpYDUZME2vyjHMqBH9/7oQ6LeKei5GFIrNU/Otx3lbMnPaiJddc+Ib5/+u9oK1r+d6IpLI8N9G5q6cR2sg2AOGfCVHMorbLUbOOYhW4XSpDtxNopU5DLLGpnIkeo84qDPD2oWkpLPy70LNe47WO9+qTV8UOiJLZjqSj8Sc5jU+56oNHMxFWpMi+2IWkg0mf2dLrBH33mmZ3sfyi80fc5KIne1S5Z9CrfSYa47/W54idB8KY3nRUlCaIjlguMVD4zWZ2nSliHvxQtzoK6DGyQPFpgi4foo5xopn9W+oPwLBvmXOe3LNSYnG8IKYya6oRIHzrn1YTSawfo+l+XyTD14pkgwH0tJyVoZOYar3FkcAWON0rluc7WhDtE0yvtXBPdlrT1TqpRHgLS0KyWwT5CyKZk+bl64Rnjel6kee+YA/qJGuCSs9Cz3DJh81+g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wDjEcW406VLo2KuB2eh1LB4u4JjbEwhGVTfbQ4Q4XGU4szMcZVTkktSBd8pKxe+53CtOXSFBk04Ro/Ep1T7hRD9OEashpJqwcOJ5ZkQnKLJXwLHOpRQY79zJQ2LmIhe5LHhHWMVmMp60+GS4D4nGfhQnhpZA7KbxY9G2eF7LNxfkPGBw2KqsCRt2j/+XsOCNu+nlquGtFfigEEaevVFYOTqqSVCesr1mj152xk8yFfsbu0w3R5oTRukJeKaaOcnwOj2ScwJzsrr7opNvx1E1upDpXhf0J6MTdvoXeS4XcRfSJSuvGtWzviqbuqMhLxqchvsmcgZg3K01YFHInWRzVueDxpYRGQln7zHvFcjtmcGyVODx7gYITYrUbdwWp+SJ+9iuAACfuammr119rYivv1R6LuosP4+cC7d8xtOmCXRHndoJYl+L8TPqkzBell758EHAdYmU0SdffvLmm2bOp/1qOjVD3doXUiZrjXAPcyzghDQH/62gxStEpNXNzbQe4/Ijit5lFjySjXT0m95gdXAwqtafPOcG0cWCBfMITClLDgQs8dxHIQC24kqsjOKDn8/v+wDYMtHoMkZSYvOmRTltJHmOf/MgQsADBRFaxAIIyjfdU/HNFgzTkAPKruyD09mQtKYqffIQCkTaak53RSWLd2eqLhM5kR9bvAPv80ipzk6gRJ/hHarP4HvR/0/1iW9HP/9znjhlIMOrkbL/71BeEJeFF0rZIZnG3jtJGjASHWcsEY5Q/Ux4EXNuJZu+jqWdj9trH6nmQD5/rVzJBGwW//8pIT4IC1imUmfFMjGoV01es0xPWpnJ+tpExFRnEAQgFi0KduQYYNtY3OOLsYrfc1/yhulLqqiXxYHoNVzctzCMbnoOm6X+oFw8L8144iWqewVae6nxTW40wPo0FoO0jlt8HPynbe3PxbZ57mQ0kZDyKxGH73Nxvul1sJo2XDt+zqdqSy8jqxpuTFMUykYo0pBMtmbAz5rg2h8HVeoMoe5PigMqu4fDSXbQdEdvi5MEpv3JFTH/JnJJS6ZgbnxY9IVBeYLnLduXC3UY+21sHBAX81mvauSSqVTxEF34aRMXu3MgLg5/IS6h+0qEAELbmHnAdgpBJm9IyWvCVz/gAFiuGGnsjIxQtVfXTtMlmVIbjbwKXjALNC/E0MVibs7RgcA0eGlxEgs2KgjV0KQZpHFl5H/8ESNQAvL23zA+5LZESs9zIe97XO0kisacizilJAq3cdO/AVcq/lgxBiEPFWHt0zhGceKhhdB+csT8s3TD9h+ZpJlYEd6v3sj2LyFZzGeqjcakcWjAe/6Ejv181bsUBvgRdbBRNy/DTe/v4gkgErk1pW7xU/oQNTFz2CnM48aSlYIuUgstKJ80sUb57DXoOhtuadUsDUnN9eM6+KaSedYBz3mbQjeFQjrA+Nb0dCrB5dUT+EEKVFe6F2ftUgrz9R9xdIPZJ0oMQLCceHZheyzPjxfd8tl/ndzZy+b1s4mNtXPkVjBgiiX0nuLpH831KbA4kSYaLxQnPK78hvW8PVAA0Rlhbr/kNQnlWM87EGy/LldvyXHWCJdiebxqA+yfmUFFZ3YE+QipKfvU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2465f73e-60ce-423f-8654-08dd76bdfd83 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:13.4155 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VLkCWqcRCZNj+yyZS6y64MhXyc0hyHmlOejSduq9M6zpuQWbw4/oUwRbr4VJdfQg X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Instead use the virtual address. Change from dma_map_page() to dma_map_single() which works directly on a KVA. Add a type for the pd table level for clarity. Signed-off-by: Jason Gunthorpe --- drivers/iommu/tegra-smmu.c | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/tegra-smmu.c b/drivers/iommu/tegra-smmu.c index 69d353e1df8439..d040400ee34ac7 100644 --- a/drivers/iommu/tegra-smmu.c +++ b/drivers/iommu/tegra-smmu.c @@ -51,6 +51,8 @@ struct tegra_smmu { struct iommu_device iommu; /* IOMMU Core code handle */ }; +struct tegra_pd; + struct tegra_smmu_as { struct iommu_domain domain; struct tegra_smmu *smmu; @@ -58,7 +60,7 @@ struct tegra_smmu_as { spinlock_t lock; u32 *count; struct page **pts; - struct page *pd; + struct tegra_pd *pd; dma_addr_t pd_dma; unsigned id; u32 attr; @@ -155,6 +157,10 @@ static inline u32 smmu_readl(struct tegra_smmu *smmu, unsigned long offset) #define SMMU_PDE_ATTR (SMMU_PDE_READABLE | SMMU_PDE_WRITABLE | \ SMMU_PDE_NONSECURE) +struct tegra_pd { + u32 val[SMMU_NUM_PDE]; +}; + static unsigned int iova_pd_index(unsigned long iova) { return (iova >> SMMU_PDE_SHIFT) & (SMMU_NUM_PDE - 1); @@ -284,7 +290,7 @@ static struct iommu_domain *tegra_smmu_domain_alloc_paging(struct device *dev) as->attr = SMMU_PD_READABLE | SMMU_PD_WRITABLE | SMMU_PD_NONSECURE; - as->pd = __iommu_alloc_pages(GFP_KERNEL | __GFP_DMA, 0); + as->pd = iommu_alloc_page(GFP_KERNEL | __GFP_DMA); if (!as->pd) { kfree(as); return NULL; @@ -292,7 +298,7 @@ static struct iommu_domain *tegra_smmu_domain_alloc_paging(struct device *dev) as->count = kcalloc(SMMU_NUM_PDE, sizeof(u32), GFP_KERNEL); if (!as->count) { - __iommu_free_pages(as->pd, 0); + iommu_free_page(as->pd); kfree(as); return NULL; } @@ -300,7 +306,7 @@ static struct iommu_domain *tegra_smmu_domain_alloc_paging(struct device *dev) as->pts = kcalloc(SMMU_NUM_PDE, sizeof(*as->pts), GFP_KERNEL); if (!as->pts) { kfree(as->count); - __iommu_free_pages(as->pd, 0); + iommu_free_page(as->pd); kfree(as); return NULL; } @@ -417,8 +423,8 @@ static int tegra_smmu_as_prepare(struct tegra_smmu *smmu, goto unlock; } - as->pd_dma = dma_map_page(smmu->dev, as->pd, 0, SMMU_SIZE_PD, - DMA_TO_DEVICE); + as->pd_dma = + dma_map_single(smmu->dev, as->pd, SMMU_SIZE_PD, DMA_TO_DEVICE); if (dma_mapping_error(smmu->dev, as->pd_dma)) { err = -ENOMEM; goto unlock; @@ -450,7 +456,7 @@ static int tegra_smmu_as_prepare(struct tegra_smmu *smmu, return 0; err_unmap: - dma_unmap_page(smmu->dev, as->pd_dma, SMMU_SIZE_PD, DMA_TO_DEVICE); + dma_unmap_single(smmu->dev, as->pd_dma, SMMU_SIZE_PD, DMA_TO_DEVICE); unlock: mutex_unlock(&smmu->lock); @@ -469,7 +475,7 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, tegra_smmu_free_asid(smmu, as->id); - dma_unmap_page(smmu->dev, as->pd_dma, SMMU_SIZE_PD, DMA_TO_DEVICE); + dma_unmap_single(smmu->dev, as->pd_dma, SMMU_SIZE_PD, DMA_TO_DEVICE); as->smmu = NULL; @@ -548,11 +554,11 @@ static void tegra_smmu_set_pde(struct tegra_smmu_as *as, unsigned long iova, { unsigned int pd_index = iova_pd_index(iova); struct tegra_smmu *smmu = as->smmu; - u32 *pd = page_address(as->pd); + struct tegra_pd *pd = as->pd; unsigned long offset = pd_index * sizeof(*pd); /* Set the page directory entry first */ - pd[pd_index] = value; + pd->val[pd_index] = value; /* The flush the page directory entry from caches */ dma_sync_single_range_for_device(smmu->dev, as->pd_dma, offset, @@ -577,14 +583,12 @@ static u32 *tegra_smmu_pte_lookup(struct tegra_smmu_as *as, unsigned long iova, unsigned int pd_index = iova_pd_index(iova); struct tegra_smmu *smmu = as->smmu; struct page *pt_page; - u32 *pd; pt_page = as->pts[pd_index]; if (!pt_page) return NULL; - pd = page_address(as->pd); - *dmap = smmu_pde_to_dma(smmu, pd[pd_index]); + *dmap = smmu_pde_to_dma(smmu, as->pd->val[pd_index]); return tegra_smmu_pte_offset(pt_page, iova); } @@ -619,9 +623,7 @@ static u32 *as_get_pte(struct tegra_smmu_as *as, dma_addr_t iova, *dmap = dma; } else { - u32 *pd = page_address(as->pd); - - *dmap = smmu_pde_to_dma(smmu, pd[pde]); + *dmap = smmu_pde_to_dma(smmu, as->pd->val[pde]); } return tegra_smmu_pte_offset(as->pts[pde], iova); @@ -645,8 +647,7 @@ static void tegra_smmu_pte_put_use(struct tegra_smmu_as *as, unsigned long iova) */ if (--as->count[pde] == 0) { struct tegra_smmu *smmu = as->smmu; - u32 *pd = page_address(as->pd); - dma_addr_t pte_dma = smmu_pde_to_dma(smmu, pd[pde]); + dma_addr_t pte_dma = smmu_pde_to_dma(smmu, as->pd->val[pde]); tegra_smmu_set_pde(as, iova, 0); From patchwork Tue Apr 8 16:53:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880050 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2084.outbound.protection.outlook.com [40.107.93.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3300F23370E; Tue, 8 Apr 2025 16:54:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131270; cv=fail; b=lvInxDLSeqoJ8D6wrfLcEZ65NPeD5uVZPavi+oG2gn4jQ6O3fgTh0IvoAyhxi0aeF5ZKrLvNlnPFMpnMjFtm6V4kvcGBK44bLwoQ0dA+sO1pJJxlX3QCdtRHUnPc0tOzibs1pkc+twGrMhVuXfOms2d4vdyX+tFcjJSO+LQj9KM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131270; c=relaxed/simple; bh=p/DxFK7+NFcx8ypmeCso7SNU0qNGt1jJ2gVg6XfLoXQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=KCs9zwNBD02FIpMzb5g57AZESRx4u/1lb/frl0lIsC/+KHwrWtV0AXK6OYoDJo6nsgsLv5/XFweMFBUn99xyMDAMrKFkOk7YsJVgy165OvaGDMqb1hf1U0PrMpluKrRVbDc6PDbGm9DymLTAuByVy2RZ0m9rL28IjQ+jnEvvjms= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=l1EK8bIW; arc=fail smtp.client-ip=40.107.93.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="l1EK8bIW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=K9UEWnoajhnKEzC1q39RY0gb0uvGyyNShaEfR708x34CzU+O03UQ04LkOfPkYr2/t1W7AOfPSYS2fXIuJotOi85krOTuSxPMaaHV7XYkFZhnnyuCNrK1DASzXxNo7D0W3ReASlw05Xi/HiV5J87QehjG/XD28FiYkOALjz/8Y02bxdY6wvAuoKAVThGwDuustwzYn+ZSTnURZBaL5AOWQ5FNZk2Pky7asDmciyxHdPYs2LGbqRqv9NHXYUsJe1l/QNP4MFhUGOfeN88Om6fhHkrNT/ltgFo4hj55sdPr/TKUvTqBaEgsB8ylg8m/ucbJrvTbQa9OpF3L6Kx2Rf1F3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/TZ+x7gJ5UYyl3U4d0W7G7snnPyKONscK3UWinwyBxM=; b=WY11YVA0aM1EhnxWfUFwcvPxM6wyrHlyXjRnE3g0g6r/U7Q6UNqGiH9ozNOVwtNdx+1uL4ywdtuDdYPUPt/aFwOzNpPIiQqWV2+tTLQHgOy/lfj1R6xVElwJsZ+/ECOvsfQqYld4kmTFQ29STGzzaRGv4k5QZ66moALk7QMrLcwws+uS+1+9ZFLx8Ay/m9xWTTDzr7mIR/oTnMVgTUZDVP3JCfX8eIkFsIJ1cZf/QUJq2GDg2iTtW734PTk7BA3EgrPV5CTd+PF9aFn3O0dz4wB6kNSJhLDEDc/cyHw5l2kMeU/QwFSEi0zicib6WZh1rsQwOam5StcVErlZivRgog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/TZ+x7gJ5UYyl3U4d0W7G7snnPyKONscK3UWinwyBxM=; b=l1EK8bIWScn4ophQl3LE62j9XMgmihR76YdZ3bZLB9/JMqrtwavmJnClKK0QZT/hjA+dliQpQxTQj6aNbKrTHw2bkA7VqR3y4I2kMm+GlK0Y6Bet6+y5Kvpy3zo4/SFz6L3w7uVYNx2zsYA0FDtZYNNkuJqdwc+Q48QylUcmB6nktno7ee1ccCzBeIyQhYZU04b4i6VsiBpdkipJVWbk6Zntvl5qfnwN90R5HxANwrhJmw6GlN0QB4Uy9fAKtvRTQXPbhR5PatHnaIq6c6jLELb/bK8uXl1FjwbffdalekQh5t7ME6PkuuU11LsikFcreZaMHgKfHA7v/C8rCgIl3w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:18 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:18 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 03/23] iommu/pages: Remove __iommu_alloc_pages()/__iommu_free_pages() Date: Tue, 8 Apr 2025 13:53:51 -0300 Message-ID: <3-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0047.namprd03.prod.outlook.com (2603:10b6:408:fb::22) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: fd3cb10e-6cac-4ebf-53ec-08dd76bdfe4e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: KnzCBJfwznu7aMmateBoGXR7ZWfeScETv3/0HPTXUCjl2EXruVrwUVH6xCnui+Gt+/ZmvVAfjMDhPdKI0maCDvYRejwmK70mSS6XcJxZjGw79gNVxZ3xz1qef2krtMy3WMFiSSwKDaqQcoOx56OZaJ29j19AwTpRIoXgAsWTsCKqKushq25D/LwggSy2TCznHZrKTJyvrc6DjoiV7d/bo3g2FEhpRCNnKL4+ACleKZEZl6RiYaj5403CB1YeEoTqNWljTbksxU3TOOb1WegItz2vTmQ8d7kAIY82EJ2HxT62O73Lh7ak4i3QqDyyoeJ5kyAvIIEvmkKbAIJucVmaGx71juE1/sVWgPVJJlqmTNhHN0U3FhAfqx6r7Zq1n8xE47qvT4B5txtgREzRnYIA0H55Liu6HHzMItjncAoAvIomne78IGvA0eiDScaRfAjxDf/hrO/2fgumLW8ghOvlh2Q998cQjBAENKXzLb+cBczxvBTC3B5XXYbPssjzgbgkyttpgFc2LNnrcbaVEjNUxHMHuYrz77NNTE+sFA91ZV35aClzX9jDWgFbGSpuUVfhfOUgwSsX4+lJqVeVb474Vs2EedOxS2NE2/p6xwzp/CSLBsgeEB5Fr1TXjx9IBxlsoF4ClLeRNu9UkMOP/0Ynp6Qv+hwZJCE61e03b6xYMxWbKsqqamhs1WwG1+MHLyEstjrA8QJuz8HNGU5TqCDnPH1U8tmDorw0pAsV67s84XycIGSZOA2LAEIukUOX7P2gyjv5Wf0PEe5JfnN6ESNL8gt2PKUHZM5dArR3zIJaMjlVDxWonHHK3h/lhTomipDXCTMVhZI0Luj3dA9z5W6UKrLpX5Z+NCD1rLTyCmwiKiW+6ASB+/qpJIIvywWTqkw+/NYPvZdtIVnIJeUF0OP1ATFO6wLNFaKcbNqxVv8OH2pC2H0NIhqsmh3hRrplKs3MbBGn3m6OpojKyBogFK0BMOyn5XM+XfDlvzxxb/NJehWgoFcv7pIYoBfdUyzS+KfXQwOwqcYRtwXlkI7Jk6mi8CIovFVJ13QxhErmq6hsDFtVODZERougsZS8bf+VjvOvlEHpQ+7zkv9moRht6SLtP/mPgNOzk1uoVpI5qkeNHNCFag2iXhjHCkKngkywh286nwRPnyNhKAv9Oo62ICKkMwfSveB/i7mXKAsZQAvdkjuGyhBHVbQ1MY/J/FCv+hqewQKgBL0ROuiwiKDjp5nLTSbxwdvh5ndqyDqZferZGKxy8GP/BoWN2VtGnfQ4gMfenv6QtcB4GaTu+9A2kJKjx3bNxQjtCBr8ZXLZI9sVy/ky8rpXsPt1zCrm5LqITZO2PcwGiAUcwjRaVHzI6hKaS1bU3OYJIVOHbHSfJg/wd3zilPqbf/ewPBQCBRXGzJ+UU2CGuIYItjtFEXKI8eqUKfA8knKY3E/NyDLxtgkx30z7w09wHMHPGB3+z6Wd5Bur X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 315tOBdivZ401gp9Zj62WhOE597kFgKq2Od9aARJK1Viz3N12XDWfepFYIC8SGlpJ5pCRNUwqBRX2b1egKIvNy2acUi/+SzJrnhPmMJNa44Oba9cvywtX7XvMYFFXcoc+qP4pTQUFQasuM0Ezxn8eijpfL1QQ/x9R9qKAryC6cBacEKIRKZNngpxXPOLZIbdZ3dwITffu7sAuNWQQ90ufStk8eQXFH/pAFaO8StjjLdnC+0Dz7tXIm/VugA+gHEOWyFThKzIcAvKdACRD4mB9IsgExXvkXwMfyxvwGqotv7XnhpcWBJODZbouDMWR1VR9/4U1pAv4WIsQ9Podbh69OMKJ7WqTyUkOVHd7WYcKwtGvddMfZJ3P+YUOChkUiYv5EWs533F+tTMmiAqQ2PFqAYdg2dMv7WNzLFufKJVXb1b8+Uellw0ZMydopPoP9tdsSMXZtdNfg2LTW8Dot8S8Si+TvMKMJSVFPGraRv+yWXUrxqesn6VPxH+tob7BPny+cTIp7Npd7DLciruvngz6pqHnnUyoHpUKMX2J9+7CDd01ULEVLmjh5VCZrL5jpy8FASBUHBBL4k5T4iHg/sa3ACuk70V15SYPLmYeFxMmoJ/3ms4uBprR/GGiJ+ti6JL0aYTVzigyoEoNwaF9krI9aCwyhCrpSuQ5xyGvL+Ew5rtLuiR4AYtNLcDEXXDsWgmOVZlqJecIAq4noTY5RJtcupNlrzVdm9XafUZM8E9YDlVCg6LzH7JWoBthLfs9r/Qyg/MQ+QWUKHNJ+vIMqoYKOBSaODk3beWEuMcBFvMC/gYd9/HNPbF2mKsFgxDtLu0g0sSDG0gmHreUvWb3MXHh5boU+9k16DDhOLaY1J5uzsTorqAZXhPWqVMLqsa0W0H1MUGDyFr3bytEjWNjF6GQZ6MzChU4mj/C5r6YLRwz3TYIx4T1fgVFxWSFRt3B6a/qNQCed8f84rPMoK1eWXJQvhkBYP9ly7o1l1G3Z3fEX2LpIBUXp0G1PCLv6rNGV9Lj6zel8aMfrCWSNRtQbHFz2tAT8XaoyslVMTwvK4SNEjyy59LL0ZPxIKqnKtWirQBPfo8a7oVYBMEXg51ca+OOOFObygjSLotZvVC0GznWcuqJtWxzJl08BzAAfC8AsUusemSWKA2ahcU2Wh2N7NX134H0RaPWNxOmJ/sC4K/hwzbGxLfUe29TOmNXq/k5Oj/EdJ7BZjBzfhWOzNtmeS/uY15o40FfTPiLY+dPkno0xmXh5EYsTL+PMLGZnocUO8dsIKNOV1NulchyXJGHBWAt0b+fM+Zz+O4lBxuStyQYz3I0nx5+fXoKEuL/Out/p1jhNfNxVn3mdc1vONLqW6HRKNktAtSExByDgUzJ5o09d/WKHZaypIKAREoshUh8639hJn9eM31kxkiddE66+5n8/h5qdukaAa9d1XX06L3vGiTK0EwMccL0RDytuJWr/jYFhA25atiYmIKHLJrZRyNM5zUE3WvwWvbkbNzW3r3fSzhAZqVRFVxfxNAJqT36A2WcpIKY4YPoJQBLTMcoLvduVk9B0Aly/1s7WjNBxQMIlcq9ZVd+xzbZon+nEauZViJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd3cb10e-6cac-4ebf-53ec-08dd76bdfe4e X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:14.9287 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SZBAqx51RsYfmr2IzJMp/i0VUGz3h9vm1WEdET2OjkDg7gApZfM1YxEEw5oJBXtW X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 These were only used by tegra-smmu and leaked the struct page out of the API. Delete them since tega-smmu has been converted to the other APIs. In the process flatten the call tree so we have fewer one line functions calling other one line functions.. iommu_alloc_pages_node() is the real allocator and everything else can just call it directly. Reviewed-by: Lu Baolu Reviewed-by: Mostafa Saleh Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.h | 49 ++++++------------------------------- 1 file changed, 7 insertions(+), 42 deletions(-) diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 82ebf00330811c..0ca2437989a0e1 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -46,40 +46,6 @@ static inline void __iommu_free_account(struct page *page, int order) mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, -pgcnt); } -/** - * __iommu_alloc_pages - allocate a zeroed page of a given order. - * @gfp: buddy allocator flags - * @order: page order - * - * returns the head struct page of the allocated page. - */ -static inline struct page *__iommu_alloc_pages(gfp_t gfp, int order) -{ - struct page *page; - - page = alloc_pages(gfp | __GFP_ZERO, order); - if (unlikely(!page)) - return NULL; - - __iommu_alloc_account(page, order); - - return page; -} - -/** - * __iommu_free_pages - free page of a given order - * @page: head struct page of the page - * @order: page order - */ -static inline void __iommu_free_pages(struct page *page, int order) -{ - if (!page) - return; - - __iommu_free_account(page, order); - __free_pages(page, order); -} - /** * iommu_alloc_pages_node - allocate a zeroed page of a given order from * specific NUMA node. @@ -110,12 +76,7 @@ static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) */ static inline void *iommu_alloc_pages(gfp_t gfp, int order) { - struct page *page = __iommu_alloc_pages(gfp, order); - - if (unlikely(!page)) - return NULL; - - return page_address(page); + return iommu_alloc_pages_node(numa_node_id(), gfp, order); } /** @@ -138,7 +99,7 @@ static inline void *iommu_alloc_page_node(int nid, gfp_t gfp) */ static inline void *iommu_alloc_page(gfp_t gfp) { - return iommu_alloc_pages(gfp, 0); + return iommu_alloc_pages_node(numa_node_id(), gfp, 0); } /** @@ -148,10 +109,14 @@ static inline void *iommu_alloc_page(gfp_t gfp) */ static inline void iommu_free_pages(void *virt, int order) { + struct page *page; + if (!virt) return; - __iommu_free_pages(virt_to_page(virt), order); + page = virt_to_page(virt); + __iommu_free_account(page, order); + __free_pages(page, order); } /** From patchwork Tue Apr 8 16:53:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880045 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2046.outbound.protection.outlook.com [40.107.94.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0EAF2356D6; Tue, 8 Apr 2025 17:24:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.46 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133052; cv=fail; b=BbM1VDlwwQ+auj5gPPsYsko/8APj6wfIm3SuObrMeu9laMUPD52DagJz7Hd2uycS/8v0JVguJhKK5cNvhNjta8NxSaq3zLlSVqmdmctVk40TUAgzQzxz8OGLdsAuz3BgehLOEDwrcBGa+4etb7IoKer6qFAVP0M/wT0zOOSGnxs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133052; c=relaxed/simple; bh=jECmLwFa6zjvAYsf3UU5ilIBxXN9oh1pb+RFy6znKe8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=EDnXEL0jn9Iy+ENYekD2hx5NvHIFLznhLz0TnN6tAw07bTLWQ8x24e/bKQ6FfESqE22xLv/ges6CbzO8WwbaZgOqgM2kLgOX0nS0tbHSmdOAdt4BnauuJ+95Vg2vl7f8qxQbmt7ZWhW8FmGXj5Dgxr3gMw7E1MdpAw8tHEmtths= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hzrLUX2l; arc=fail smtp.client-ip=40.107.94.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hzrLUX2l" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=t4iW8O5YtkTRKuz5fwIq2JOCdzbBFV/q+7iDQH6FbvBoxcIKgflr9KOLXDHjKTc4XhHIB7ifmT3F72HASEhPgUn6dd1hv5fmg2AsNWeHsxzww00h/2DMNVx3SeGmceC/4BthjxFsYlC4tphWT0fSCChY6kdqUdFDPUcqYfgVc1h0RkVo73ETXZw/pTBFzx+fzkRL3eZeRntVvQalp6hwsN2A8cAzmktyYs6VpqzaujbW+aARgV8AKb5s45HGK06l3Y8Qhya0yUWkDfQdvzIJjjvkxDZDpesi8wCFTc5VHjW2tdPuXzvo6iRgxq77WEZg4/UirvYbXPjOIS9NGVlpaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U7BPIeyT3cotDXj6MRVWw7szAtV/kLylvW05uYWnCe0=; b=dkZU46AL+1DNCfcVXESyOzxMFkL5IsDRrqujaSVhaOUjinRU4maCKi1YwVHthrvAnzw+3ZiuOFapLIgwtji2AbKjHWstj/J3Q2prjlTkaHhWvQ3A8so36w0+s1LfoiUfZ+nNxIVOt6A6Nwgg2UZAoL8FgdBHu9qppF7ew1EzBMaf/+qSx1XkCJ8oYY0EtdIp3gtm7QXacYc7IeouICLZST3Zy6lBKey/+NL6s/D7i6XDMcc1PZr7f52L8SCKqBvO6JqRleNa35Zo5UwLuBjBUCtcwtRuf7ddbhiBn3IooeWz+n+6rwYUgcQTpcfSMzAhDgwmy5L8pJnwSNKZcNFuaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U7BPIeyT3cotDXj6MRVWw7szAtV/kLylvW05uYWnCe0=; b=hzrLUX2lh+3E+yQuZXh8GWvpUtJB2dQu1a958tZE0cQqy0SFYkjfPD37sdi5UFsLsinNSB4mPUy34bgUCXS9MI3uwE3xgeGuyU2Cqx/yq5xMc8rUiPj8TtYHShNu8UK15JnDDO1BlTuMJoHMkkL7QUBBVgP1aGXPjaRi6l/MET05Ky/V1k31m0mQeywAo9A5CdL7bnR5AkrpXx6cns4wJrGVF0WC6hIwmHKHSZWQ6/UBR8pBKdcjY17oitMM1pxiSz0kvs1z8sw5rGGFymNEaXT26FY7tsa5QENnN4zGlpXhTCMW6rfziqumdarvg/IzljLdNwXj3PRLx8+t7Q7UgQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by IA1PR12MB6330.namprd12.prod.outlook.com (2603:10b6:208:3e4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.33; Tue, 8 Apr 2025 17:24:05 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 17:24:05 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 07/23] iommu/pages: De-inline the substantial functions Date: Tue, 8 Apr 2025 13:53:55 -0300 Message-ID: <7-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0078.namprd13.prod.outlook.com (2603:10b6:208:2b8::23) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|IA1PR12MB6330:EE_ X-MS-Office365-Filtering-Correlation-Id: 4005723f-5ae5-4995-4ad5-08dd76c2298f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|376014|7416014|1800799024|921020; X-Microsoft-Antispam-Message-Info: N2iduw8wavf6sadVDsJOWmEKVWV11+xysB52r/3Ip8XIcYmk85h3oP7rPbrHRh4JIunOPTyAZaLINpJUl0jxePI7hx+orjjpdCD2JqNwSCM9x7pSBjLvDDTdp857Od6TTkEGJz9Y4hsL3eXD24TFJVph77dJjceoo4E2AQQod8SoepOalmeO461Ycgs/JVoNdtioHnB8FWZFvkPp3/Tl7sb+bm740DDJ+nGXa8LW5iHHG0SAo3sqxIwzDXYg6vccm0dTvrWUrfBYxStHrhRM1BlQf2MTmLiiFpDMoGvKO01QYSvZAduYsnyHSKegXz0gTbOW0tc6OtUnNi1mWEHXFzSjCHhP9//KLUM71St0B+bbyxXv9OxNF948XH7Xg7I770ttNvKSu6J0dOsAmsYoF1rnjEoKZH0isak1l+P4rSz+IYDugYY9o1kIhfA5NvfJx0ATKbxDbovAwRi5Nqv1dNju0HMmKwioG4cmVHjTdLYSa9xFg47Ou3zkRL6WyFmUJahMlajHNftNU14ep/Psn4iYCjlv/FBbcFVmOsfqnXEqr7DLo3VRkg1mNWb8uM/j4Hba5ygrLCJ71kkrQXNN2Z8qxBtvPWpX3m+WWjxpsJt9DJPUCsHTxyjaNdCC4XurbUoKiTsqHTqyVg2cSUKwewXg9USIC2YH79W9aTHqZ41X/JIs/jsslKe1csqVfqcyM7k744FDtvxPWkrNtfk7nBOuQwSEGIDP/7y8JBK0NsmqnFOuKPwHZLx8tICF0fhMGmG/r+dg4nNA885uI2MwAT6s+IcnqZQlIg5vPbiHaq8flesS8vdFI3wpPzKtiX1JOsTUOkuoavVJfAmpoHHjVGkj1n+69JqxzcaIXaLTjrldg0k9fwCjxs9QjO5FxSgrHZ6KdBS6ZJr0F+eq4yrJPeSdJlqU4gpMCIvvrqM03rWpSHt7r7UsItDJHwV/wkq1FFVNPGlCdSlI2flJCaucfvUI/wwisA710cH/jC8kBH1kECJyUW8n/lJM/1Rtce951x93u9z4ZYXcS01LDmy6AboSrUTTWrDmJ8KoR+G3a9sSXX/jU1gLCfXqS/DavW5CZEvScjyyDg1a2LUBTOKHWt4kyzLAn7dHr1zNMR1n853+AUebwlRuPF5E7S5RNgvmNZL3+IblPaLXte0YEJpE1JJ91gAXF2kRZDGqGeR2lN2hrKxSlZ/kJKITsZco+esCAk2OQn3haTU7UCNdHmNIiA3F1gKRbNRZAcHjQL0y153+ogXHaKbATlXZ5QIO4czuhIPxsqKX4qS25A2JNskfC6KTHD7rcie1DHpE4YE4dUUyc8k5ejjsoa2KAQJCT/RBqWaXzzukms7kjQm5UWshdLLmamR72OH/Qm7XJtLxJ7AhlpZxI/jrb60GzkWVtdBXzIDZiwxXgjTvIuZ3ss4czrHrRaEGdFTKUDK1/sD0BcEDMnibPryVU4/EgwUeP12nCj8e595ZZxFwCxzwzJpiYw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(376014)(7416014)(1800799024)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /rHjik2yHDjVGt9gTQrfO8sZUy8Kozps+x3T9KKSo4EBi2fybevJcNHHh6+9jVuo/fhiZJcc/Qz8JOHSQaxBgL2orNP7+ILrfpYlbr1rpJBzkh1hhADrE+Q//NlwPoLyNYs6bSN6wOpzl3X9GDG49+p5AEZwnSOoy2O5KIKjK3LIIzzDotZbiryn9UMk5nl/jL4+p88/KYGbwyDfVFHHldTlE8xRy2cpsVOTkeXsscBFIyynb6fYKHXpHXGXFDhnGE+2ZuWUsYr2VJvG8dFwwuc7jPF7JlEXm0ss2XYGk02jrkI4F+OVEgXxtulxUu0m52byhuvDhVdQjMnilxOw1B2X0/J5Y+4bzPsLbmmBZKenyUIxsD09yZciq+sR3g9X0hwLmd1XpyjFfag2elg3mxpLI317GY0rEnwc2xjpNQf0vzn4w7aoWc0lUsyuIPB2tIvdXTIzZrtYSFaccxbqxdtZGVKlwnp3J7qEL5IpJi9q2T6TPyLE4F0FYaTo+fgeJaL0Mb00V4rz099e/cJLA6CTSlB68nNTqDG2h5wVTL21IgLGZui+JqIsp7gdnEiwn59HriXK9cv3xGcuv972VQ1gJfRQXozJ3UdLiIXIaB8THpIWCJD35oWuo7orJS1vWaP+6vTl/lQRsmEflJdlUfIL/FsT7RSOqXjO6wE2JPeWbYfoSvamW8anxhyi3xB+7NBBB8PgNRQ+mv24tjLNYyo5vlycmzg7qx7UjU1FCZvQOLMMjSM40e2uTA8Fgn5qCR81q0ZixakUGFohGJgRUD9OWBzPKiKaCt1bIvF88Uf6TIkQ0R5NxAwXEZhcmLIzPw6cBICO9K2Ww3t4c54sr2b77ybOYUN+rJN3qKCq3VjCgy+hM8c/IcMmrijzNotc/UCQWrQfm1q1aIKJdCkEOdGB5TVCdPgP/q67F7cNDsze1SWyLxcHkZd8A1hleB4TqsM1qVGQZF3s48l9Z0P2545/ykCwf5SKZLnjsQGa1X9tzvNXFP4wvn/pnJ1Kcx7OoOsA7koG2YdQLUiqw2eEYqEOQW70CVb1R0Wa3DaFjgtUeswhW0+i/y4kIZf1twLHeNBLAh7nsACax7BpEjOFQjyrLZFkbs45rdeu4Eg6aOQ6E0JIfH9Wsrjnf3ZPfPykqS3xiQZf/75kjcR3VA+VeOvTll+v1PhqdWLX2LvnYSdQ2mS6YYw3rWWaY3W8Ok3O+zJRHo/mH556VWfKsAoxn+AHm9DXQ4grV3V46lFLPIT8FAmU350wzGyZbGiHiJEFh82KKyGeboQhDfYzn02wnjIXtIWlrWb2EQyyyNlnBwCFPGXZI97/937C8Ff8Np/B0XBO/gxQLnjBzsT+qELTsorO7/nBzUW/YUpfCOJGUJcgLG9SJxH44fo+5lWaFtjL20rwi6xnB/j7S/yCyDRVrutSE62Dfizfh+5Mhtiw5YdfgxdZ3blWdl5GgTNm7LVkY9m7J91T60XIDgy89rqgBofi30ElY9kL4L/CsSlQT5MOmkVwn0894/u4BeTx5RBLFGBubKgyA/0PwzXJBL4LDHZxNdT4REeN8Bq6hH0RgECzXOkxGZ8It2VbwaG4DZnK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4005723f-5ae5-4995-4ad5-08dd76c2298f X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 17:24:05.2499 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0z24EPWdt6YYtJ1ph3IZigWh00/SA53eettlQikFAydGeGITSyFqvI2F2oOjVy1Q X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6330 These are called in a lot of places and are not trivial. Move them to the core module. Tidy some of the comments and function arguments, fold __iommu_alloc_account() into its only caller, change __iommu_free_account() into __iommu_free_page() to remove some duplication. Reviewed-by: Lu Baolu Reviewed-by: Mostafa Saleh Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/Makefile | 1 + drivers/iommu/iommu-pages.c | 84 +++++++++++++++++++++++++++++ drivers/iommu/iommu-pages.h | 103 ++---------------------------------- 3 files changed, 90 insertions(+), 98 deletions(-) create mode 100644 drivers/iommu/iommu-pages.c diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 5e5a83c6c2aae2..fe91d770abe16c 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += amd/ intel/ arm/ iommufd/ riscv/ obj-$(CONFIG_IOMMU_API) += iommu.o +obj-$(CONFIG_IOMMU_SUPPORT) += iommu-pages.o obj-$(CONFIG_IOMMU_API) += iommu-traces.o obj-$(CONFIG_IOMMU_API) += iommu-sysfs.o obj-$(CONFIG_IOMMU_DEBUGFS) += iommu-debugfs.o diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c new file mode 100644 index 00000000000000..31ff83ffaf0106 --- /dev/null +++ b/drivers/iommu/iommu-pages.c @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2024, Google LLC. + * Pasha Tatashin + */ +#include "iommu-pages.h" +#include +#include + +/** + * iommu_alloc_pages_node - Allocate a zeroed page of a given order from + * specific NUMA node + * @nid: memory NUMA node id + * @gfp: buddy allocator flags + * @order: page order + * + * Returns the virtual address of the allocated page. The page must be + * freed either by calling iommu_free_pages() or via iommu_put_pages_list(). + */ +void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) +{ + const unsigned long pgcnt = 1UL << order; + struct page *page; + + page = alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); + if (unlikely(!page)) + return NULL; + + /* + * All page allocations that should be reported to as "iommu-pagetables" + * to userspace must use one of the functions below. This includes + * allocations of page-tables and other per-iommu_domain configuration + * structures. + * + * This is necessary for the proper accounting as IOMMU state can be + * rather large, i.e. multiple gigabytes in size. + */ + mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, pgcnt); + mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, pgcnt); + + return page_address(page); +} +EXPORT_SYMBOL_GPL(iommu_alloc_pages_node); + +static void __iommu_free_page(struct page *page) +{ + unsigned int order = folio_order(page_folio(page)); + const unsigned long pgcnt = 1UL << order; + + mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt); + mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, -pgcnt); + put_page(page); +} + +/** + * iommu_free_pages - free pages + * @virt: virtual address of the page to be freed. + * + * The page must have have been allocated by iommu_alloc_pages_node() + */ +void iommu_free_pages(void *virt) +{ + if (!virt) + return; + __iommu_free_page(virt_to_page(virt)); +} +EXPORT_SYMBOL_GPL(iommu_free_pages); + +/** + * iommu_put_pages_list - free a list of pages. + * @head: the head of the lru list to be freed. + * + * Frees a list of pages allocated by iommu_alloc_pages_node(). + */ +void iommu_put_pages_list(struct list_head *head) +{ + while (!list_empty(head)) { + struct page *p = list_entry(head->prev, struct page, lru); + + list_del(&p->lru); + __iommu_free_page(p); + } +} +EXPORT_SYMBOL_GPL(iommu_put_pages_list); diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index fcd17b94f7b830..e3c35aa14ad716 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -7,67 +7,12 @@ #ifndef __IOMMU_PAGES_H #define __IOMMU_PAGES_H -#include -#include -#include +#include +#include -/* - * All page allocations that should be reported to as "iommu-pagetables" to - * userspace must use one of the functions below. This includes allocations of - * page-tables and other per-iommu_domain configuration structures. - * - * This is necessary for the proper accounting as IOMMU state can be rather - * large, i.e. multiple gigabytes in size. - */ - -/** - * __iommu_alloc_account - account for newly allocated page. - * @page: head struct page of the page. - * @order: order of the page - */ -static inline void __iommu_alloc_account(struct page *page, int order) -{ - const long pgcnt = 1l << order; - - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, pgcnt); -} - -/** - * __iommu_free_account - account a page that is about to be freed. - * @page: head struct page of the page. - * @order: order of the page - */ -static inline void __iommu_free_account(struct page *page) -{ - unsigned int order = folio_order(page_folio(page)); - const long pgcnt = 1l << order; - - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, -pgcnt); -} - -/** - * iommu_alloc_pages_node - allocate a zeroed page of a given order from - * specific NUMA node. - * @nid: memory NUMA node id - * @gfp: buddy allocator flags - * @order: page order - * - * returns the virtual address of the allocated page - */ -static inline void *iommu_alloc_pages_node(int nid, gfp_t gfp, int order) -{ - struct page *page = - alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); - - if (unlikely(!page)) - return NULL; - - __iommu_alloc_account(page, order); - - return page_address(page); -} +void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order); +void iommu_free_pages(void *virt); +void iommu_put_pages_list(struct list_head *head); /** * iommu_alloc_pages - allocate a zeroed page of a given order @@ -104,42 +49,4 @@ static inline void *iommu_alloc_page(gfp_t gfp) return iommu_alloc_pages_node(numa_node_id(), gfp, 0); } -/** - * iommu_free_pages - free pages - * @virt: virtual address of the page to be freed. - * - * The page must have have been allocated by iommu_alloc_pages_node() - */ -static inline void iommu_free_pages(void *virt) -{ - struct page *page; - - if (!virt) - return; - - page = virt_to_page(virt); - __iommu_free_account(page); - put_page(page); -} - -/** - * iommu_put_pages_list - free a list of pages. - * @page: the head of the lru list to be freed. - * - * There are no locking requirement for these pages, as they are going to be - * put on a free list as soon as refcount reaches 0. Pages are put on this LRU - * list once they are removed from the IOMMU page tables. However, they can - * still be access through debugfs. - */ -static inline void iommu_put_pages_list(struct list_head *page) -{ - while (!list_empty(page)) { - struct page *p = list_entry(page->prev, struct page, lru); - - list_del(&p->lru); - __iommu_free_account(p); - put_page(p); - } -} - #endif /* __IOMMU_PAGES_H */ From patchwork Tue Apr 8 16:53:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880042 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2079.outbound.protection.outlook.com [40.107.100.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F175323C8C6; Tue, 8 Apr 2025 17:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133061; cv=fail; b=B1xTvEk9ZkJ0OzvtuSu4E0B5xhDQNBo6pi0IpnPy8NZrWt/tFy0rUB3GQKbUT9/1xl4BF1Acx3/LsltG+UJs50/WzvksGU5iEqBKInnNv1FoScmtz220Ss6MnuFY9EQZv+1zqUThJbY+4szPdH9Dx9W4SfqMMKhuSmE4vHeGlNI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133061; c=relaxed/simple; bh=bq2I9pizBR8/YsbWJcC+6hoeMLVm6n8oNKgJ1XmiC1c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=LuXIgw0JlCBbI/erlqY2LKHHk67zo8li0PRvzQGaqGR0y+BTEkJ5JVTWF9tYr+9vYsRk/TR2/d1QBo9mDk3ToKzADURuWT3/ib/ZE2LV/Kxq3JbYCLlAONAsOLhvMoZ7j6Qt9c9XggPGJ4OkXCKwGxKGWcQjx5Y2T9yXJywBz2Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=HLLQec8o; arc=fail smtp.client-ip=40.107.100.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="HLLQec8o" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rKU2LDE5B2Ap1uhh5Xk4FjFV/qrC16xEc7vMiM/I/7eOL6pKKfBaEnYKvpLj1qa/0gUpKPO+ZLIIVggB0sGcsXA9utAO1nrL/SoJAkYUKDTTDQspRLcUGgNWFHw8PWePDH0xDJVj8Vq8gBBeaFroThc9CZz0LB/vg4hjT+PwtRcr5u2eqBqUWF1uEQqgAmjLmIpWwcvui/pXS/rJeDC5u5VIPlYU5Qa3ZjjownPYcItsQqv2B9IShN7mIQCEWniqVVadLzfC1tyZww53gUmqlEuMy634689dfQ49tiHf0zJG6XEyzaVtPV2d8ImcF/ulq67k8FBHfYC/2HG1yBXJug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Pz8aFQru04TskK5JjiC/vlot38W7HjzfjFFPF27YHmU=; b=KCP56F+c/uSGk5lieXPThMr186PoMYroSct6y7od5isNBWRDIjJl1kJ+DObbQX8J9BBAF99du1rfzSLj/OLg8ESaOFcoozzuxWX0fa/LHd+bSYPxPFM9wWzS9j6L8A5q5uOmn3NUkzigjehLhINEvHyO/ugQ0vrbHydL0xY+B6jKXCaOcu75fF5D5C58zJ1PFr97ipDwKOaguVVDHKzwBKM+cYNz05HZw9cLWO8UDMOAYb5MOrZ7YrD0t8oWz+/TldldvyLg8pKsuZEUSqbbyVZEvdvPrKzdkiJLA9TkdoPvKMLjFKJcfHLp0o/YLqrItUEz6/Aj1MZ3RZ65JfTZ6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pz8aFQru04TskK5JjiC/vlot38W7HjzfjFFPF27YHmU=; b=HLLQec8onLgCs3m1ycYEm3idKn85W78z5VGJZ83cWDSbgylHHDvi4OfEefatNpjkM1bWbAlKjRCRMLRoPPCv7L9Ya9PTxVtgEkru5SEod1eV2dK9zkoMwvNQlAQEOY3wEkK3Wv9UqyPtChp/DY/pq3yg/yb+V0PMTp5EoNVj4bAgx/+uCa1O5zkHgWUklujeDzLTvtw2Jd+kfcsLPzrZ/BjLUN54ZHztOjkMtPXGEh3JDaozdCLTVrM2LgN3O6bN5YreDPdtcYwJf5Zh1mfdVfKhl0oQYtTrs9T6x7gZLZje9IfvZG37n8xQ5kO533lWtOpfQ8tDvwzjglGu6u/5oA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by CY3PR12MB9579.namprd12.prod.outlook.com (2603:10b6:930:10a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 17:24:13 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 17:24:13 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 08/23] iommu/pages: Formalize the freelist API Date: Tue, 8 Apr 2025 13:53:56 -0300 Message-ID: <8-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0013.namprd03.prod.outlook.com (2603:10b6:208:32b::18) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|CY3PR12MB9579:EE_ X-MS-Office365-Filtering-Correlation-Id: 68f44b27-ed7d-4259-a5cc-08dd76c22e4f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|366016|1800799024|921020; X-Microsoft-Antispam-Message-Info: 7wbEFybl9bhIDYGUkTs5Vv7rGqekx6W6hfZkxMB/v/zZZkVVamvxREE0fJb3L40lZMQ3QoyKzTpoXZl0ILTvQxP0x4epmVe51zjgJavNUOwOR+9uKvbdnn+MsA+MPL73RlXJEXiYH3V1D4SNzC/cgIodSZSE+vZrDVDW09PoR+85K2hddGPEiHDKprhFeTU4F8b3rjK6nC0ghxk6wxwQwjA7grLNXN67ObIOHZhAawX5DoSv2Ijx7iWuMMXxnFkOdQizZIufpJ6xXHjMe7ZXXq6afQD0e5oydGM3Wsi7S8klaFF0zjdv1y9CE3KaqcdzbtyhXC9g3MxO1UN094RBrmjmuQtQXQ5LnQH8YVfGxrgMY3j4UNRk3mDKks9nHYBeZz8MhTh9Oq5HkCn1n1NRC6vloH84r3xt83cbRJ0349HqrUpTuw/yRr48UoppAOID0r0CNEGtxcaIHIa2rsxbeNY2gOnq53pYm+STVhrOjhHaY8AmV7FBlJfobzetEjjuSxCcO9UyKQtKI4i75eVJ01YwAArNucAOa7PYGLbarU0Dw/T1ik6vu8/dmDbSA2XqcdoNVO89nxlNn/NA40k3oKr3aXetxzjIdTNCrB34dlhJCKlfzxOeFlXyctaSKqJ4RbvDy7bP5ZUtJtMgj2DwLExjm2larnllOl4QkMFWxaEX78EjHm3gjjA6hwgurfZiPxKH2vT7/57pVkbaiNOthTRK6Qknzo+JItRw5WNd1pOA/n4JCKfnu/mHi9FueXrRy5PoaSSLk/xkxhKmzBrc4U5BivjQpbyqCDd+Ckr9WRwMUOsCs5Vr+e/WXvgvzlBhddRLOOJ7rVxw6O4Dqkdbm+H9wlkmb0CYu4vu5X2CvxWiTOlUiozmT8P9o857gU3o3RGAtfGkJpjKkzpx+TL1igEfgX1Z5VwpGy2bo5eO9DK4TFUUxR6D4IWqAJ/PBGmeUpRGHhoFelVl3hcHb4/nqBMVMa5hqnpXygjHjRcortAbcwODbpjEeJODXumoXgKltOTDMONXFDK0y15rbGx2BaJvtHZgr8bibqCoSsK0bNLDv8l4QDw3pfmS5cPVC+uDgdYKoA6iZH4hY2B/mbAZ+YnQQ+nKbDNKykbW9u0ufQhFQcab7ZISwm887SFM7pDKxg7rztkx5OezMP6SVoXJFXjn1oqFenQs6syhqXIP0GzxFNYbP0YVtAoH1AGoAFRKRDrIW2/18LCmNBaBTw6wcvKK29aEF8tSYxwPJr07oOjl93iXj/L6A0n24g6KDOKrLMNGXmf3EqIX4GDtkR2nK8v2+h2y5ml4185ZNCmGxm5UgqxgFhqWvBfastNRshmtlHHqVcTt9gs6FT5MgUAZXrEl5SSbgw0iOvvzY88zKB1HhTshejIbqS/RHdjPH83n0q9VQQsyLPlcDhEGw18Z/vrjDWRKh8IKMeVW9hllZvaI7Hy0zO/OQDgJQr/7yWFhg17BaISSldrxg4ELLc4J7Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(366016)(1800799024)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8R9/qGmXyxdHMStQcWkj3uPBuZbnV9FB+lSbkpAe0nmYMFTD0JaDmEtEtVMnEGyVgUPdBm80bqsN9nGBxbbTT9S36QQJCa5NWTwtDH1GMAcwkq3vMwROltfk8BKfGeeLbpSuT3BYZmM2PuNi8n+ty5bD183W/E/wcRpIAo9ybBMTwri3yi1oU1f1r53Y2p3fAVsP0n7AbR108LdgguIu9t2raNS3Yx7s16Fu4sQFVIw3pRIJgs2vwsLMkMBSG7/T9SJ5nr7JuZe8IZIYjGu+GkJQltmMwBvKLxlNXnnx3IOVtKQEMUQeFZjMSwYjtf7vIOxjAt0cxpKr9cZDZi6Wdgq9/fP79ktgaJMbAoC6EFspfbd9dTUz5egEJb2D09HXQvMBA0SRY2xtDKrgwRUJgihOGh/K+m4Xu7g8hWjZtFl1a0gqlxyWVClw1WNK83K4ZrfP1PkHyO9PM5mRx+GO4GVrqOjmLcS+v4TPWd4XsrYnZTUjQSzWSZLkb7K+ReSCDK0wgKKCeZo7kre5+y729tm0fr9dj0cgmA2hp1IMQlh7gpwSysnjEgbHBxhbdCKhzKUlLJ62LpDZWF/zVJS0nzV1xBcVygJovaIkHWO/iHgmknrIhlr83DqSTJQSO7+NetOqYLbOa6Ssh70URzSQfR/O/VT/KLm4gywmJOkHdh9fSUx4JrRX/CBiLxCNDB+Vf2MvgR2rmJOX7U1prPiZgd4GRw8A99LJ1VZhXXdxtovTZr4jQNTnHAuYzzhkunvuVDAbwifrV/l1OK5QcugWatQidLRHbX0Z7hWYNvOeDhy6FtPYxDxS+GM3F7V+cuzL2xyDVj3yJvXsegBlKkc48T3wNZgsj5HkEITBoSdom3poSuxQHftVDtqI5nNI0RYD/Xdj2vDvdZ+WRqCjdyRkIfAcaoiI0BmG4i4h3LOzxEwxLPMP/XDQeKjTT/6bRhENiXz+nHmaUpuAYmWSXEQmf6nCnLFw2iUMv0s2K4eWrOz8hpI6dHphdepPX2MeOc3W06b9lX8Cv4HzR1F+Q/FE/BBQCuyIGWUYxGWLiQj6mYtyUZsLPqRKXnw7PrEfb2+ArxPY1Hl8Z/mHwJZVjhj0hHZLpVDOIkn7aomyAmdS1yXcUysh1ju1fnBZcA+BKwPXOC6uyAsYvmjW20xjOqGeFHqtT2sSdxuhGOSlGiw6reDfDdEPIjwdE2gLN1S8hi1S9gYizu+M4dvB4PzJREfRRrx7lI2/7CquXVsm1ScBf5+O51jFd/JaBFwtadq9vUbBomnlwXj597JiGFF2lrs4aXqnYXuwbrF4AWBR3wD6l/DxS7gqlPLajrnkFvcd+F1zXQmsNdpVixaey61jV8rS9pyDjmVb0nUNH2dlGx2186aGgrRtMh1f0MlWNuY0mifzVrkG+tMOMDnL/m3JgnEASonXwMZLJ9al/QzG3WdhRKmBsKqUn184EgChVXRgmoueaXVqyqwXHEIlrI4SpjHEO6H9R9WXs7eZzJ9H3U1QYr+NhsmtNpxTJxbbecQjaY/wtsJBSQe+Uh4mHA0DuIm3/zbBncz9VpnO5RFiSYbxqVA9BeczuK6AIX6Eg14E011O X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68f44b27-ed7d-4259-a5cc-08dd76c22e4f X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 17:24:13.1551 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZubnMpwHGtG2LcBWdG0+FEwLpxstmunvZaGbWvB2ZbU4Ch6zArlm8MppMNiK73lD X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY3PR12MB9579 We want to get rid of struct page references outside the internal allocator implementation. The free list has the driver open code something like: list_add_tail(&virt_to_page(ptr)->lru, freelist); Move the above into a small inline and make the freelist into a wrapper type 'struct iommu_pages_list' so that the compiler can help check all the conversion. This struct has also proven helpful in some future ideas to convert to a singly linked list to get an extra pointer in the struct page, and to signal that the pages should be freed with RCU. Use a temporary _Generic so we don't need to rename the free function as the patches progress. Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.c | 23 ++++++++++++------- drivers/iommu/iommu-pages.h | 45 ++++++++++++++++++++++++++++++++++--- include/linux/iommu.h | 12 ++++++++++ 3 files changed, 69 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index 31ff83ffaf0106..af8694b46417fa 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -67,18 +67,25 @@ void iommu_free_pages(void *virt) EXPORT_SYMBOL_GPL(iommu_free_pages); /** - * iommu_put_pages_list - free a list of pages. - * @head: the head of the lru list to be freed. + * iommu_put_pages_list_new - free a list of pages. + * @list: The list of pages to be freed * * Frees a list of pages allocated by iommu_alloc_pages_node(). */ -void iommu_put_pages_list(struct list_head *head) +void iommu_put_pages_list_new(struct iommu_pages_list *list) { - while (!list_empty(head)) { - struct page *p = list_entry(head->prev, struct page, lru); + struct page *p, *tmp; - list_del(&p->lru); + list_for_each_entry_safe(p, tmp, &list->pages, lru) __iommu_free_page(p); - } } -EXPORT_SYMBOL_GPL(iommu_put_pages_list); +EXPORT_SYMBOL_GPL(iommu_put_pages_list_new); + +void iommu_put_pages_list_old(struct list_head *head) +{ + struct page *p, *tmp; + + list_for_each_entry_safe(p, tmp, head, lru) + __iommu_free_page(p); +} +EXPORT_SYMBOL_GPL(iommu_put_pages_list_old); diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index e3c35aa14ad716..0acc26af7202df 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -7,12 +7,51 @@ #ifndef __IOMMU_PAGES_H #define __IOMMU_PAGES_H -#include -#include +#include void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order); void iommu_free_pages(void *virt); -void iommu_put_pages_list(struct list_head *head); +void iommu_put_pages_list_new(struct iommu_pages_list *list); +void iommu_put_pages_list_old(struct list_head *head); + +#define iommu_put_pages_list(head) \ + _Generic(head, \ + struct iommu_pages_list *: iommu_put_pages_list_new, \ + struct list_head *: iommu_put_pages_list_old)(head) + +/** + * iommu_pages_list_add - add the page to a iommu_pages_list + * @list: List to add the page to + * @virt: Address returned from iommu_alloc_pages_node() + */ +static inline void iommu_pages_list_add(struct iommu_pages_list *list, + void *virt) +{ + list_add_tail(&virt_to_page(virt)->lru, &list->pages); +} + +/** + * iommu_pages_list_splice - Put all the pages in list from into list to + * @from: Source list of pages + * @to: Destination list of pages + * + * from must be re-initialized after calling this function if it is to be + * used again. + */ +static inline void iommu_pages_list_splice(struct iommu_pages_list *from, + struct iommu_pages_list *to) +{ + list_splice(&from->pages, &to->pages); +} + +/** + * iommu_pages_list_empty - True if the list is empty + * @list: List to check + */ +static inline bool iommu_pages_list_empty(struct iommu_pages_list *list) +{ + return list_empty(&list->pages); +} /** * iommu_alloc_pages - allocate a zeroed page of a given order diff --git a/include/linux/iommu.h b/include/linux/iommu.h index ccce8a751e2a55..6dd3a221c5bc66 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -341,6 +341,18 @@ typedef unsigned int ioasid_t; /* Read but do not clear any dirty bits */ #define IOMMU_DIRTY_NO_CLEAR (1 << 0) +/* + * Pages allocated through iommu_alloc_pages_node() can be placed on this list + * using iommu_pages_list_add(). Note: ONLY pages from iommu_alloc_pages_node() + * can be used this way! + */ +struct iommu_pages_list { + struct list_head pages; +}; + +#define IOMMU_PAGES_LIST_INIT(name) \ + ((struct iommu_pages_list){ .pages = LIST_HEAD_INIT(name.pages) }) + #ifdef CONFIG_IOMMU_API /** From patchwork Tue Apr 8 16:53:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880052 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2084.outbound.protection.outlook.com [40.107.93.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22C51230BFE; Tue, 8 Apr 2025 16:54:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131264; cv=fail; b=pBzJGdbRlPDJfG5b6LF/uUSUJRwUOYYE+7CtEQETuIgwZAUn2te+swos69grjdegikbb6xU1kRFZ2FzVuo8IApidiKHqZMUo6GDvORfzMppL5XfmANVVJCdCp+UIznopb8cbzKB2knTz4HNHsLH9EAkQM+sjTk9KM4ncx7ggoAE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131264; c=relaxed/simple; bh=Uk/6Ciq54cbUF75u9+f144pb/13FDwBOG4xUZDgDNio=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QaQTCqeL2nVlQhlGsJJ/WFDPh+cUkzI+4/KWPd/SOjncuIf19700HdtEV1NmIVCCoL8KDN+fKzsgcSlQ2wSV2PfwEUlPHXtvEv4QPJs+9YInEXsji1Zi5I5rgDLJxNwSB7m0JjDjAEWxWbP0NLkDmSwfPW1bkQzQQGc3zJekavM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eCA7slN/; arc=fail smtp.client-ip=40.107.93.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eCA7slN/" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=smFeq8sM+Vi6+VXw0Nl2PbScXhoGUt+lfu5dGFo9SSjcq3bW4z8E/zKYVhsEkBm7lOm9kNw4g+0rcMsWswTofwLfdOSS8icYI+YPlhAIlxFJuYjiKIN7OR39a1I19Spg1n/q8kUpFfT4rcVVMH2PNxgGjg6EXT0WgG9XdkAREyMlTOhNIGNmr/ASJmNLi1WRce9nTlpD9/K+G/fQgUwIElSvUdGm9qxJVh1tRaPUO/WEEO2A8sNbO/PxJJKNbvKB5RwepRYc+qTcknT8aB3zeBBA2eH9i5q/8cFe8xSZpUUv3N/ut5c2vghRLWi+lK6CHsk75ZKLYwZomWakW8E8ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+akwFg18r3NdimMyy2y7q62VGXPwuDsWVi+yie6UZWg=; b=tkfz1fUOekvkZEIM+8fTMUd/b6ZvFx2MMnWCtAgM/y81cB+HgulgTBeczwqbXrOqk2nvdHUNBNgkAf+HeMYJLmcmTWSAX8S4oX6nIpzuieuMN0ZH3W8Ypm3XRO+iXmFts16nrVv1ZGjG0cFngFNAxrz+y5RTNXuyZ9lZX5D1s520UtDYroNmsqCMJktTLvYrBfNinWe2wJsr5pkZ99Vh0cDbNT5+bKqczqLtyrTdxihvn9YQj/Nt1DjUUzAwlYeQD/kxuiwWV98A231R9XKGIdfr7Wrkw/Xh5I/xDHzn3wKM9+X7M+/lPATKuO9FZpuMT9n5jrRcpBGl9/r5a7wNbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+akwFg18r3NdimMyy2y7q62VGXPwuDsWVi+yie6UZWg=; b=eCA7slN/vPd8XI46j5nTLF74n58tGE1/FAhFoyuiN7x9Ra7wKsKfNB6xzV6uM/2SpP2IrWDo8Y6brMy6tw8qKVLasMkuxZmSsbyAljdQTiGbqsoDEuySsvjo/AlxgCKLYaxLBpCJD1M7vIX9OIu3QJUrIoljDk48NDkYjji002lVuCab0MeeU2xrXqzXiquOY9qcQOQYMNygcnDfFfI7HLUBSO84tipPjYsU8/R83loVzQrli5Bl8uqeo7NUMffmE7j7CH/ioEFHIvwN5ABPm0m0rUpy33JW3gRAhwMuwYMBe3s9hKhZNhQAZUGJ3vZNUMVFmUTWa0E1JTYj7UoFuA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:15 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:15 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 09/23] iommu/riscv: Convert to use struct iommu_pages_list Date: Tue, 8 Apr 2025 13:53:57 -0300 Message-ID: <9-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0041.namprd03.prod.outlook.com (2603:10b6:208:32d::16) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e9f5c7c-64e7-49b7-34db-08dd76bdfde2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: MeOVKs2Wq31jKaK0JIrZEkbz8jLcRE+YjISlxVxSUv05NSD8c9VOpC6r1mdAeAJVDw42RGxzFwESmKD8UH5AILAuyoYwejSZB2H3MpRcw9nOdPDMHALb3t55BYCXFoEkuzCbGy3JN6JLv5flBwv9i4z2SLRgfdSCYg6cG0duEtFK6V0iNuGgqoHbmY4n1wKh7SNOQkk46MfSWGLffl/BsGUKVSxjhtvO497gVS/tGfo3dfnED4Q8SiwGjNbrwXmATRXLsXOFCRE/8tpvHBHzI8HZf/E4JKFbvrh66LbQ4G9odCBAG4UHkNyuB8tRSvPMS3aiXfYhayJ7M68NiGL6bxxWF5jnKoc/VS8A83MFtOztIp+Fjqy8dJaJykTY/USPgrYEyxEQ34xdhvPJ6ORGC99GJq1lLYnxhSPJh5A7egUu09FStl9P66z7HXbMht5Fdia5lYxyNYg4BSzaP27bMpPwCYbRNhyn5L/7wHmqMd4qL7Zd8+JS6mCAql5hzbxkq/gB+4z7TamDcx1UAI29/Yn5CNmW9vK2gZArlF7mbNamoTZPYwoVdwCPU1Ow637s8S1y7EOKeko1IwSvV6Lr81XbG9D4vfuIOHcPzh2C3wbU71SpdcCwMPjNOkIHEJK8M8iO8XoKjT2CKulbr7qNXklclYZOflRHwOMWyiDm3hgE7jdBb8mNkLdrT8e670yiQHX1Q6EVDfhHpYxB5MJg8l2TZ0bm3niWOqXG3FUXMLLi4vBueXT7Nad6VMa+5M2Q6rD1SJIddwE+bnkJX3CjhW5Qaty37WQbGkKBit/bYfeTazv2eHx1y63/ZYHI3ZGvxKfk1CVGLfE9+qqEONdCt+ixK6Jn5a44RFp3U4amGbmTxU45VzVcQdT4bVKj8Tq/fG0cLGFtzJzKdBi/C3L5pnoySL3MuaLwxqmpb7WPcCEb8I/q8um8RfgCb19dJa88WAnLHwsWSPkvIz4V2N0s1eQIS+4oXkv5nYutEMjbFO4xzXp/KePuIte15jswJyk6ol8WMFQ0AV8GuZf4bwGM35BzYOasFk49IVBiiCT8IIS/CbjmwrPJ02kbje3J37IcpfYBkb3pQs0X2dys4sK1XcdGOKbf0nJB9Jf0G9VrKRD9VMmq7SrzvXd3JJnfBus9AGnN8HfEvJLIPlZBskRtNSMvPKfq69I7J7gw9MVyArmBH1PDklOb4CES3g8ZA1uNWIZtxtxO8w4FYrL40J3BQ3BBhE3WMTyYWJCslXJ92Xc1AYRCD5kyDtmJ/jHvBvszExCRWms2yLyzzvKgJWT0OQbcmPdmD9uPSevnwDJjYi1v89OYHW8h23qiM/YK81njjTzrQgrpptmNpruZBhq2N48smThTSCZNrtZrI6Qb7vVeerLJ+bd23mPEOmqorvzA+IsVnSiEl+MtNEBAXwOF76NM4701GuUxoy/amPbPTzhoUHED0e7kBU/skbXCtvnj X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wmV4QmVu/mtabnw3xgoyRhkOclEp27jWJMIVX+e36yRrrSXfOG8AKoam50Bo+72vQThN4BOIkzW1Ke1D0kGpM77IzTOjvczl/fNdIYBQ3DEbDi5M+rTPst/0Go7Xf07bzneAU5DQJehbcDMUcZjYu8ZhorDEt8KGnmzIJu211n+dP/eGhNs6v1DuU5qCLUZLBiX0ywQuFvxsZCBh18DsJ7HOvwoY/KIeZXNZIrvV5bE10pSbHlJ4bQBA4VubaMcOiyYZAjSYFM5ZIAKr9xJpH9EsJoCuOvJ2eskao25hQI9ERX6o496UHeuSovD/Vangc4yFFXyaAnkI8ee0by7yCgxFgoaqS8vRVqq/i6Je1OmQatsJRbYZ+MfdMFRhyOa/bsLs5bMnzVxFtxIgA4SAC5ZnwSVbTGoah+AdX1xvSbGwR0569L6BfzPIN9VkcSO+1CkBc4VbENutAGlWjTfBHT85XG9+FgP4Se02XkiWKkb7MTu70CKP0BMHYkxLs8kIU2+oWsXyWZr+ppC2XSFZaseNwlcgKMfuMlVa3hqzwUGM8Xbc9KWSv3d0D/9U1YmvcBEaZA/wDn1d8nPghTtMv9kjYW8ksB0KvaXgCFRwPqCD2cFfRNvuHa+yGgipyx3+UsHG1nn+BeZYzPVRPyfgAEZu1GPNAyhigV2KWQulxprsT3y1gZdreikR6MYJpRYjmKJBF7CLZwAQKA//OLATQRbPyvEq88lZ2WLejUWgcOAmvnwxpOfpAniaKBPX4Cpa3mTXQICqbG4KlEpgcSfdQ2eTGubk+0yymfFfYdA6OYhm0zjfZJCs4qBHwQ4sWod2ptGdDqDgwDcvSqSjDHS8rlr8ROK4KTISDdw0B8EcwSKqdg+9WVuS40vQpdjcpqNUDvFz9b4YE/kbIQKOKSNZh1qkF76DIbsIFVRGH/CKY0sD+NKDx+M29PAeDBYM53ICEcdCCwVUlm8cl9BsJrvgfdi0/RSZmAPP41Fl1LudlaBouyAv2KXAvtZVnjBaT2z67TIRwJ7yhdctFfttm+JPGLTfEQQwijsThf0uhY1jDv+KOYc/qVI/yH66L1Wgom6gtT8pjjo34nDusffURGDsAvklI9TZFO37wnGv9RpZgJ+H2kx4uszJcTCIxOglgCuuHlM0DNHBMnhabZp3L+MqSlrgO3f5Cqzslz2M801Pap34gh+1d7VWTiuwEYc+uu3SWpYa8IBHEy+VkyHshOeJeipegY5DHthn7+2hyOPRXsnJC5oUgyF9ssTajyxAT63Fn51v8ndtUoeKbuFUCohVm/uWvRDd3bMHgSA6fZSaNxNt7QYkkBgWRrD+upKgJGders14MOj9qGyGtn5IKOMA3/d2SVz5QmldImguSIi3UGvwG79bMvfHiBamtLjfWl7U2WQpHrdrc9RQzmMjXfdMbWuckRRiKcJ+wVqYYEVegj1nxgcZfiS3IVVb0gteUN1NqkEzcYijqhPcWb0HGZ9Uxa10a51JSBjseoxjIWokwFL4DUo/ZtGtoWd53K2gNpS0HbZSRu7X8FXMcAHZ8C1iQJdw+xN6cEUK7iV0MtauSMivPsEjwupDdtpE4oiOCQuO X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3e9f5c7c-64e7-49b7-34db-08dd76bdfde2 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:13.9373 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nHsJyvdzt7LGcrwh3QKDTJaINZAF0tmBN9k1GUuEfkjIb32zJ/EeHJD5WIvs5jkK X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Change the internal freelist to use struct iommu_pages_list. riscv uses this page list to free page table levels that are replaced with leaf ptes. Reviewed-by: Tomasz Jeznach Signed-off-by: Jason Gunthorpe --- drivers/iommu/riscv/iommu.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 4fe07343d84e61..2750f2e6e01a2b 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1085,7 +1085,8 @@ static void riscv_iommu_iotlb_sync(struct iommu_domain *iommu_domain, #define _io_pte_entry(pn, prot) ((_PAGE_PFN_MASK & ((pn) << _PAGE_PFN_SHIFT)) | (prot)) static void riscv_iommu_pte_free(struct riscv_iommu_domain *domain, - unsigned long pte, struct list_head *freelist) + unsigned long pte, + struct iommu_pages_list *freelist) { unsigned long *ptr; int i; @@ -1103,7 +1104,7 @@ static void riscv_iommu_pte_free(struct riscv_iommu_domain *domain, } if (freelist) - list_add_tail(&virt_to_page(ptr)->lru, freelist); + iommu_pages_list_add(freelist, ptr); else iommu_free_pages(ptr); } @@ -1192,7 +1193,7 @@ static int riscv_iommu_map_pages(struct iommu_domain *iommu_domain, unsigned long *ptr; unsigned long pte, old, pte_prot; int rc = 0; - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); if (!(prot & IOMMU_WRITE)) pte_prot = _PAGE_BASE | _PAGE_READ; @@ -1223,7 +1224,7 @@ static int riscv_iommu_map_pages(struct iommu_domain *iommu_domain, *mapped = size; - if (!list_empty(&freelist)) { + if (!iommu_pages_list_empty(&freelist)) { /* * In 1.0 spec version, the smallest scope we can use to * invalidate all levels of page table (i.e. leaf and non-leaf) From patchwork Tue Apr 8 16:53:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880046 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2079.outbound.protection.outlook.com [40.107.94.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3BE623DEAD; Tue, 8 Apr 2025 16:54:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131276; cv=fail; b=XPwtvS5YIPAoqj242GigJPzpOnxSm53whj8sgh2o9LuGC4jSyTO8JRZWp95zTr3aSXh3pM557qWID0/7mdmPU+gZFgEWhVGxI/lG3NbR0HcsN76cYr/H83fyLFhc+Vwi4HWin2KtI2a1H4L2GIgN/PACSepbHqn7yk8P7DKFmwo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131276; c=relaxed/simple; bh=EwUz2zpDl8ttuhnt8DzDNLoEp4n5kn4u/oKRixT9+cM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=rOttpVAWM3j3BEaeCtyf8GSqstwNXXX1AkUc/fFvUkgLlPqHiSUcKxUERuew/x5nG9+2afiRrfnwhyPWchn9PxBuSTtvI2m5VJmbtpwauKT73SxKV7YorIdOGhqaad09QCqIoQklpOtU8vkEs1C5frADxZxKn9sxYBY5KeyoZfs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Kkr6nPb4; arc=fail smtp.client-ip=40.107.94.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Kkr6nPb4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lEp3PcVo9RzWfkjGikgbJc4QsuFSRf09PIpZnd2kYKsV8LEL7St/O6FgPa2fQS74iX20wqIIBXYnyKk41oQermSnLSRje79dt5eDT6q9mRpvygHDeqGEsFAyJxiBV6Z+sV/kDsvFcE2AbtGO33KlDTF88EFDbjL8CqftAmNN7X6z1Hl65jx9Uw3QZW3KV4epQhGZ+or1IigbxYsL6Ive9NCYIeV9TYFLmPdMFIPhLXfv7trUPMpLIWBsXye9QFaVqSttEDIzwI2zNSJlP8iNvOPMFB+UcLANtb/9CxoYoU05Zthpy5HEmyhhWviPH6ImaJeieK7OM0H20D5xf3XsRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SOMbIpYmrZnccYIV89P8MSgtHZzA8Ua/GI47Fo/WeoQ=; b=nbI9fgGHe1gPwzllkhDmo7t9ITqcCUtOR/xg/ptf+3SrRYQeW0oLCID0H6kx1Vy0sW864UvqIcyLClih2KUB9z9w64CVMDaHi6qJt6NRNtAyvNzGdwcerrqc5AhyRyyaaWMnDsbH1tCVCTox8ORv5Y+BIBsO6oymqxO3u7fwW35Vf9tACt/q7/DZTzuB6B/BfLWfEDmzzA++EpttqUwSie2GnFX1DZ7xlfN0v3lMvUkp1xLtqak3nIbDZfR2Mwzrjv6QmbB6xtrLoNKngcjNDRWXE3uHxFxMxgddKg8ybRRPLPMq/acvrY+hYvAdVqqXtRoD3lUz7Fu2VmEeI8Qpfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SOMbIpYmrZnccYIV89P8MSgtHZzA8Ua/GI47Fo/WeoQ=; b=Kkr6nPb4QkceM60udPp8rNFrBQogT3C+wn39MHm5DDbQjO6C1yIW7S2ubJcmQq3mcyB90zgZ3MumFhhMbGp+0757smGR2ZwWmPlxDp9atCj+ti/9Z9GALj/3N/f3FKmPCHG5klpYWr2IqfajDKyUkjK1y74STDzVaBx7xj1/dgTjMKtXLJUV39hwlI+LRaDu8VUUp6/zQ052cTn2EX7YGvw1akR2CaAevCgBOP3r//fTD8rb6KBSLLWfzynurh+xox0lkT7DmGyHMWxPN8DQvSA0SiEu18cDREZp5Irxaq5ukTJeeeQxdnVKZIePmbT8X1oTPS6Op9KFYpTSKuJzrQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:22 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:22 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 10/23] iommu/amd: Convert to use struct iommu_pages_list Date: Tue, 8 Apr 2025 13:53:58 -0300 Message-ID: <10-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR15CA0050.namprd15.prod.outlook.com (2603:10b6:208:237::19) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 6ff1420c-8e3d-4f6d-2093-08dd76bdff67 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: e1mgG12ZD9YD/OdZetbga5NPgflwgtOkf3nQILgl4sRE8OlNG38iVR5G6h7wma6ir+d+yHQY3BUxcprTibscw43KW7kBn/xzYQeKr1NlRvnA0Y5eUAEr1h2RPKSsOqNn0VA5b1YDMztUJKofqSLAtrtm07ClSObGgEiCxSKPXHWuD1ODlwYzCvnJ2kr9WEBdXqHzdFhNMPs3HEhhR4zPhCzlF3zvO4joNZMc1lOWCY008dXfwvyssNmsUD0JX7S9X5EvqcrWU9dYtMOFBt6bNByCPTq8z3/LtrmDN6/MZwJgALiXR5lqV76lWhPITEx6h9dkbZ3Qx3iBModAmoUmErMfYSYxtVseLTrAkd2TVUOqEZVVOE4x8NPe9GTFQQnTP3UpkQHVZWYpFbGHIZnyTpDb+EvrnVXIM2lOxZudTIidbp+fNIGk2sPD/EIrrO9ng2L5LiUMBkbN8DKlyv7e4U0it0exYUWeTTjCpiZb/lFtPqQrg6FpIc1vTnRJuycDO5PI5zLtP85p7SnoZhYkYm4TpJnOEqyGmVNpsp2532YwTQDdoLZUTGV7ZHULE8MM4inIUEOWIygOssNnG0xiTxXhcnPQczw6c1h6eouB/PfnMKX1aPPkGKopIIt6hiIxsJuEy6rdz6DcZ012sNaxirViuJPi9JeEXK+2lIbjc1cQKo6+1E5OBK3wMrZHLYBBUUB9Z6vNUfLcH2blU42+kHANzmosmEESk5KVrrYCU2ekH7Xj7aPPoUpoWsIY8bim0ZK8y9yGClkCDThBNord0LtTK/3UM8uy7by9vkN0OofFErUk7EkTPdqha2ZmRE9aNBppySYVDCdUW6lRiB2B34bNoRvbbDKX5C8fDVbRlpwhBFyfbuukmb9wAXwOqvOjwqKhpsw/Hd4/b68aKQnh5vS46lTVwnAmGBcgQSuuVtsusxQWTmRKLD9bznxdT5BP6+ivuq1U/c3n6SKOG5usS32juMtdzJ59zAeAlZQJ7OCKW4yYvJKp7ZnK+ZJ1CA/hRvdSTxvg3BnOP9JNoI5liL7ROT7SHNFKNsCyIWK59HxA6KCTLkn1CsxWtro8H14FEJ4nSRSJOsWtv1YHqIu088U1v0OQ1dRnU9YPC5TzwczmPxgk3J5BpNeFCO2KgxbUEdk4eqHP/tJLbtD8GSnSXh14kbKs5Yl2JXa6Yml4AcIiDgXpNRNpW/ypvCEPIHDiUEf4wuelYLaIWNXpWWpB6ONGighMb7adNZ+s+kdp7gkAmRDfsx68eOP01GlPUtb5HypuwBIhw6ymhbwhSxizWdiQDFR6nTlrqw4776pZ3xDUEPOnbZB71aFjFGv0RY6WKB0Vw5UVaFzokiyhqyUxmeJpV8gj2u5mIPYZMk5CghZ/w36sNLLuubcBx1WLsMPOl2KvOjOO2Ps1rp2h+R7rXq59Gc0pP1eQYbfc6alsJ5tBhBdiSq9irP4dtUzI0/lyWcianmN4RLr0L3FjVuKhTA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gAVMuCMUc/bpqZ6e45CKzgedA5gHPpzqfgZjgUp217god5GZjkIqxutHiwi7HZoTzXw0PRaha+792TKoRzH9Y3HXcDP181NXJpibVE1PQm76X7RyuFcLOUp6pbVrVSYuI9ui2vtNl732J2fJaFei5M31EQ9RV8wMD+gzJe80HM1gfp5jbJK3SdoKRyZ+uOpgVhOu8U8SX713OQQ3f7xRDL/ght0D0Vos/lhraZ/G1kPMn5XBNGl+uTCblHe9Lk0azeMq1+TNDp3BCHTDDmbZYUmfSkhUqu2FV15GmH4bUPWXGoNBInOlqu69Ut5kOcARHkmsaKTT41dhDsG5GNZQnyCR/ZdNowblHN3QkBNtRrKZeqkWiOiqI+x3Z6FEVZ37mOuzMe2K5VmfZsCFtM83+Ke1lTwNO5YHsXuZRsiBMtOlMF1xVap0CE9sWnZs7Ys779LLniLyaAQ03QLBTm/4iVcy69Vu3HcazlKVbP8HXyBBYjnSPYEzGA5xt66C/vTzdhM+IjPlIQ4MALqi7qtYSdJf81dLEUCrpUFtooIppF9bZMi4F5LnYMZKj8/TrVtl0e9s1lGLf8nTeHvd2iMyfy2JkbsR8TEEWj+APhRYxJguZ4Dn4DcH0sbU/GGDQn/VSYgdKvznks7g2jPcUFzFeylfuRZt9/sdtZpIoTHvfIQx0fQAB5Yt1gu8ewWjE+vMJt6ucwi5Krs+VpzcGhIgenTqw3fxa+HZ+S2cf4b3JYUXYH8bj2u5ymvOhDFCV38LFjeN+NHyEhoi1lE/Uuf+CfAxjTplcY1GkqxTyx+9SwGkV9nhavrSPrpnlgJ2cFhvNhoVqzfeJ11gs/M2sY9Qa98acGk7T8rUTiEghy54HbMP8WfncHaAyEwfJ17vLuCrPuaGL325WPouoemsf9edm5QX+h00YcMbOzgXUAWqzywlBDW19pxXLrbMpTIf3KMD9uip16jsdSzUlDi5yD4cEMhdRi7z8xdZe+9qbc3xq1kcj2+xDB/+qoHHd/YW629MLsAqoAJfrRjzLe57ahbkpIuYuJ6CxQgah/DT/x0CutGFW3iIIg6Qa1JV/ncpn9jwSRUwUyovHqzyidi/pKvIgApLgaIC13bgP+ovMWj9vNbt7nYGnAPipNDE5MDs6P6w64ssjWuSr/Y1N2QouJN0FZJ9u+bhBanMOtQ1dMYzm5IyYziO2h1l4kcp0JL+DSVvGip0DBUALPl72rwRis7ihsg25C/7omHYfbla+Mlg7RKCCs9P2fpTXsP1MRg3uYQPdc8hfbO4sK0ho50nynDJ/rNJC8qrGqxCjkwzovcKYI74lUuZNlXN32gbuvfEisILsmsiak9Ffvg472eq8s67AftI+F2YQPs5Ly0a/7w32rS1G09WV99mnAsOyh8oYGyocvA37fOV5qzTITNIptXPIBRe7a4o/hJhdxJbwGdumTfIiwZVagy/d85Rdmfde27XJR0yCnz9PuWB0K/XRYoo3wEkRV23DwC5GgKnKzmUBDrxmjPTWv1AGEF2/lQIDOedO1K4f6B8Le+XLh5ipUyVu6rAXWLbQMNPQc6iDi40DMs4yy4zNQpexI4gYOXR52Og X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6ff1420c-8e3d-4f6d-2093-08dd76bdff67 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:16.5679 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IYmXri0hG9r4hYfkkoFv0T2JyouWGIW29bAKuKTngD4Y4T+vy7aeJHmgr4SgGUqK X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Change the internal freelist to use struct iommu_pages_list. AMD uses the freelist to batch free the entire table during domain destruction, and to replace table levels with leafs during map. Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/amd/io_pgtable.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 42d9e6746dee28..97ddadbcfb54d6 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -47,14 +47,7 @@ static u64 *first_pte_l7(u64 *pte, unsigned long *page_size, return fpte; } -static void free_pt_page(u64 *pt, struct list_head *freelist) -{ - struct page *p = virt_to_page(pt); - - list_add_tail(&p->lru, freelist); -} - -static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) +static void free_pt_lvl(u64 *pt, struct iommu_pages_list *freelist, int lvl) { u64 *p; int i; @@ -77,20 +70,20 @@ static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) if (lvl > 2) free_pt_lvl(p, freelist, lvl - 1); else - free_pt_page(p, freelist); + iommu_pages_list_add(freelist, p); } - free_pt_page(pt, freelist); + iommu_pages_list_add(freelist, pt); } -static void free_sub_pt(u64 *root, int mode, struct list_head *freelist) +static void free_sub_pt(u64 *root, int mode, struct iommu_pages_list *freelist) { switch (mode) { case PAGE_MODE_NONE: case PAGE_MODE_7_LEVEL: break; case PAGE_MODE_1_LEVEL: - free_pt_page(root, freelist); + iommu_pages_list_add(freelist, root); break; case PAGE_MODE_2_LEVEL: case PAGE_MODE_3_LEVEL: @@ -299,7 +292,8 @@ static u64 *fetch_pte(struct amd_io_pgtable *pgtable, return pte; } -static void free_clear_pte(u64 *pte, u64 pteval, struct list_head *freelist) +static void free_clear_pte(u64 *pte, u64 pteval, + struct iommu_pages_list *freelist) { u64 *pt; int mode; @@ -328,7 +322,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, int prot, gfp_t gfp, size_t *mapped) { struct amd_io_pgtable *pgtable = io_pgtable_ops_to_data(ops); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); bool updated = false; u64 __pte, *pte; int ret, i, count; @@ -353,7 +347,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, for (i = 0; i < count; ++i) free_clear_pte(&pte[i], pte[i], &freelist); - if (!list_empty(&freelist)) + if (!iommu_pages_list_empty(&freelist)) updated = true; if (count > 1) { @@ -524,7 +518,7 @@ static int iommu_v1_read_and_clear_dirty(struct io_pgtable_ops *ops, static void v1_free_pgtable(struct io_pgtable *iop) { struct amd_io_pgtable *pgtable = container_of(iop, struct amd_io_pgtable, pgtbl); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); if (pgtable->mode == PAGE_MODE_NONE) return; From patchwork Tue Apr 8 16:54:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880048 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2079.outbound.protection.outlook.com [40.107.94.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F9B123BD16; Tue, 8 Apr 2025 16:54:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131272; cv=fail; b=Bl88jCDqFOtiGCe21cV2Hig/fD7Q4wpRvjLaEea0T+b/4oahxax78fNVGPJe0MHSDJMGtYZkpDt4kEMSzjKTqt3dftk+8b9g7c1TcSqGlqpBXjQhlO3N6YFO6KissKPbjF/NyT9SfvMGhhPL0vMlzidY+CZ5xr8nld71uhA+PY0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131272; c=relaxed/simple; bh=t4FB3NeteCB4kxcmsCtfjXRnaCDA1xumt3Ajm/w6pAw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=gMQLjebb5kqnE2V+pTkdUv2TM4YqCYPQFEnFxwXJXgZPzJywtLsZVONLW+qGeaImjQJPN1/TaggTVPD3X1jEzH9iW4kFrJBICKJCLZNJEth1UURjruOXXJ00dpmZhSl9CXXM6uvC1aaRqWJrY0f6a5K2hAi1BGZyDvTgUvJwem0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=TB5hS3gI; arc=fail smtp.client-ip=40.107.94.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="TB5hS3gI" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=N0L37fvseQydKBffuRq5Xt1Anb1gF03LqdGMYqEURPn6+FR5U6A1iZtcTBg3BXoL0m0AJb1VCkn2cTNmtG0A7GH0vItFB59fVU/MvXpbYxg6RCwVHbxKiJI2gFtVcPUxGeq6fLxX+g0TRPoRtwxVwsH3ZoT/B4Qc3ke5LFXDPIlcTJi25jlk9S/RSuBEciik0UFg96GtwjISQTMiD88ow3ZXSedgiNy0MdBdQOQP7Ty3ZAWSgBYHWTZ1ye2lySw+3N5YQGJEYclSpcUOyn6IvYhvdl80mgwak/NABZK+8TL2exHhL34GFuvbwHltiaceF+W9/YE5sSgKNgGpAZmGsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CtFfGEmratQDUmMvIACkbCBBSwxquvWKV/4GFmZ/KpY=; b=C3IRNDAu8TZ97mHhXLiUaERkvcYSo+XOC4zLYqMo02+aLBUS+CHy4DILwj4lsyIrcdMbauU2x8BwGRHCPCYQpnB29orZ/q1grbWnIpbjD3kcxXNkUdN5sxvvv0+glk2Bsss9cWOW+kmT+X1KxNfKMG1PaNj1o/DZqk04ZV253q0z0THj22o0Gx7qjP2A9/QLQlitbT0GDihIXfeWhQNz8ex+PV229s3R1GXRRuV7B9FODSkOl/YbEL6BRPOxL7Q6tA5YGMQEqfJd2RMAx451MGR+oIA8IfFsCRFkFoEwNP3Ae1sX+hsTaD8dmegHe6wLRSY731+Z9q2510hLcS0Srg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CtFfGEmratQDUmMvIACkbCBBSwxquvWKV/4GFmZ/KpY=; b=TB5hS3gIh4E6T2TfRVTpl7G57kdCNt7ziWvanJLjU6xxzzZPiDdhkwQsqQB0L14XD8cR4cxg9yEcDiEea3C83FZ8eCUfmNi3XEtfHXkxZknLo0LitFCYPtYPYq3Q3087JOwCXjoXXk0vg4L8zy5SteAmWuHGptBFYeYh0/6liitptfvfYs9Sv4TTN+s+kAYfKDbmFnPQnql7xZ/BuncE7DbnZGhoFfH+rp+f2PEII2OjdJuUZeo3Auj8oYBdcSyk3hyg/dcueQSzeFq0dCt0tYNu9CoK3kFzUPGfBL/zotYlLeBeamSkTM0mnaBHehIQ/YUlen1AvOxnVgMi7FGeUg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:19 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:19 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 12/23] iommu/pages: Remove iommu_put_pages_list_old and the _Generic Date: Tue, 8 Apr 2025 13:54:00 -0300 Message-ID: <12-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0055.namprd03.prod.outlook.com (2603:10b6:408:fb::30) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 69d17eab-43be-46d1-bee7-08dd76bdfec2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: 8IO42LQqtQvV4FjvgAIybImg9zxaTkzgs/7GVViw9f7DAf7zpZjNz1DT/c6zjh1hzHniR56uLVg6fTTu0oPT6RphxVsoQhTPhkwaBJLvJT2ayEtWZ0vucwMUgTMhP/332kokKramQYdrE00cCybolggVeXNRFngCgw2Bi8Ex7+Y5l3ZqjavefgF4x9olen9AHNP5TdfGrVJ4MynaJtpcmomLyxWjFrJdUDzoVYXKhdCG2CqaB6ZdnmiRInzjWrA8f0kd69hFpqDXQbK0r/TXpWGgtZEMhg9F46S2XqCmWV0z0xYd0UTOWskBaDPJ2/YMuwP1q/WNRMQLuoH9GhJ5zK+xxKJyENM4mqCZhvrdzYszakoGedb09SHZN0DsZROO/pxncoIlz/fHOIfKAEbqGaLqM80nsuYAq0I9wxPQKroWh+m8fHwqZlV/9+QQ8/v6vhUDa/oC6qk4qnaZx6hBH4XFBVm03RXeOBanONg40Rj4YPp+2LIB9++iqC2qFcoojkgpoCVF5lANiKldAvJaye6+vnIA2zTafFJoeTy/8yoh6jnL57gVdgRWjjK6cDEXgLgRNfbMBXnBl9AJ6htR+lmYo11k8dzYjyAzvAtKBIDb4ycYbW6eG+f+5wQtzxaCyo6LgGJbx4sKLnCKTUDEQplb7hRJUMqz57rlEUPtxz18rxZSKQfVlMxXcUc3cv3RRv/CW7Lflfo8pH5lpxOMZPCFIAqxDkl28+q0y/XvFqAtmRg92DYldN+46xcMsIhf/iGoFtstR8XFcOYqrNbkxkLdEPexfmYBf9BGwI8wIDkiLdAKKzmfR87e3/Of4ZFYyoodlqLCjoa1w/OI424jwyL+5sSDiJacD9a169jPtDoTPGhm2ic5Mr3kO9X42lPNSWRW57bRv4IGg2LYaN42NZkw9i/bPye+pbiw6AfxtkfP71w3EezZk5zr+rKIeJbZhw9LwNq1X+GfqTeoDgwFjo1PsiX5Yvs7ZgeqKvD4vheGcM6p0URMwGWuZ3+3JWc000jMNLJVbgG1IP0F2tf3dZkAHM4tMrNangC8wmucycJLQcNNwZ90hn46MEltZSnLDkP3vSfn6ECJmEYnLsFoIzj6Hd+1HFDrAL/rcXHIwTm/Y3y+JXzzR/FZeKqCOfKsrQL2iodPkUD6PjeIa4so0LfGjSaCi5ao3COjzkwWr1fa2G8j5TPXCzxYvEoJeLsouBZq+6i4s46RdMptUSTOCmB60m2FW5AaMwY+6vr8Ycezn7jTI4TU+DZEknAu2AB3zQOsq/Ve/oul0NVxjCz7kyv2fokufQxpZa0BDPya1a1XV593TS4zlU9S/o87L3BqGow3AFQjq9o0khCEJaIdIheQ6u93Cfq9T7+vQ/8hHv5iGXAkbT0qI2+BF+IfMHU9h2023F8bgrnFWxXpSJQhlsnuKXQVdWzNHM9Pgd+Z1fPjmAOLOoN52JF9Suf1gv2KCrAIrpaKx1ZHv6yD9TWv6A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1TAjXzc1d1cy10gpsPlaHxnxCzxrnyFKqg/uFG0tBrGY+tb0QFCNrSnDqkDMc125m1xIBxPDa3ZBdxFpjMR8tZJHAf1E+6U4HCyHZslcuiMgLPuPbQOcLrjg2wpHrJG1FsWkNLuC/YOpYic9jc0PEUQIR1M7ITrG7AlDsTRiGaNHTxZ5uhtsqemWVprBRIhEMlTut5+t+i60l9XvYQYCY77jgFLD40h33jr6lKP5qf0LoH8e0zwFWo9HdyxEZym+OyJrPGXdtt5K7E/fdHhAe1qEmO46zjPkHLp5qPxgNHVBzGorXhBa+Qi52ayqHe5YrZmRBkZpBx6LSs89nJHIel5rcRuNQLAaSkSTaEL0R8UvNL56AKhqw6bDEn57B0pvvGD+TLxx+VqKzAWv3Q8/usKZ4ywinc+8UQFIQV/fAlAtqnnAIOu1+DUA1nva0eQVDWPZs1OLSCxF0bkBdORS3bRouvFKow5zliD/6T6s0J6n4RAWTCDXtTEg/DG8qNhApw6P88ar1MZrvmiULISc5CbltaK9azqOv4O8jM+znIwT06bCtxj1GM0VSlEs5x+2MgCmUO4Bjxu9XTp77s7lIlxlDJT6RddTIaq6jvByY/p10ktCit4W39Yp4Z6CfqnzBphU7EBNBiJIKiisjrG7kGZIHSLZC49QNXD7VPNsHc+HYX5DVX74S39Ybb7b25hWReOuK1MULTMmRET6PWIhP+iCzHkYVHHeYEH3e9+zC44LssFDyxXvVIv/pZaEMSUAi47fea7hsnggeuC/n2LAANVZqgTMKd8HjGk7pGPpDq3SMPEmXF3YujcnSIr3Dwq3zXmpQPbVHuCBLTiPzjn38nf0A2hjyZqJNFWIato8RvsxKrHkdxd0PirP6HgJmw9+Z9j6o9FGrZ25Rjta5iFmrPmTusgfGIAX8FHKK9BwvQwZcRax+7rMupShMVxHPNAqvGXrfQ3v7wCESUFB8MkJa1p5KD7egjt0gnUAfJ8LBeB8AS5WYR7HUSp6P7ra/kkUNoxwYiMlN+KVEN2OW4fqrbPhSNtPG49gOstoyZtoiTOPLi9cL5Bdf2l49j2u4Y7N5DP7Xvay3PKysqu3TH7AyQzxQUjIkgJ9oC0DrHzrAPC7uY6FGkzxBfBP/8ZGAN6vgwvt4MGwekosNoDm5zQUvjrqdMVe9SzpbgLvgGsmLCBLFRUDpuP9U3T4G6+dMyIaHlejI4/kq2WcyND7z0YEVri3zGbqbSywzq7TLqU9sKTGeRZ1yxNiufzyX7iVyCxQUXLseQxk7qxkhX5G22cZMgKgEE/L5tk3kToXQcpk3wqx/0G7OVX67Lcjif5WEDiHEfOoSx/FRjykM4i4kaAU43k9T8vkZ7lngT04kALkIktZYlo+yhTDLwl9PVGRRLZWSw2OF8MHOO6LEyRd2ZZsGeZO1liE7z9FzUs0Hh410uNcmWBzebDPTEnUiBIqf0YjgMoCfxGdH0IRQXkf1/NnR6mrTjYgjTjAFLdrLVfe1G8yLNOtLHrZ3QmiWtCmLzUX0qPa/0PnMEgOYxSmcKOYbuHqMRYPUeNrVR+NeOXW/8dvcEMc4eW+nEh3sMFgsf9B X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 69d17eab-43be-46d1-bee7-08dd76bdfec2 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:15.4730 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: c6JMU0WJ53nA2xCebvBjtISF/fHhNTHHNZCtbzMinQoCPqGQvK6RY8r3k8j9trvH X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Nothing uses the old list_head path now, remove it. Reviewed-by: Lu Baolu Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.c | 15 +++------------ drivers/iommu/iommu-pages.h | 8 +------- 2 files changed, 4 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index af8694b46417fa..6eacb6a34586a6 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -67,25 +67,16 @@ void iommu_free_pages(void *virt) EXPORT_SYMBOL_GPL(iommu_free_pages); /** - * iommu_put_pages_list_new - free a list of pages. + * iommu_put_pages_list - free a list of pages. * @list: The list of pages to be freed * * Frees a list of pages allocated by iommu_alloc_pages_node(). */ -void iommu_put_pages_list_new(struct iommu_pages_list *list) +void iommu_put_pages_list(struct iommu_pages_list *list) { struct page *p, *tmp; list_for_each_entry_safe(p, tmp, &list->pages, lru) __iommu_free_page(p); } -EXPORT_SYMBOL_GPL(iommu_put_pages_list_new); - -void iommu_put_pages_list_old(struct list_head *head) -{ - struct page *p, *tmp; - - list_for_each_entry_safe(p, tmp, head, lru) - __iommu_free_page(p); -} -EXPORT_SYMBOL_GPL(iommu_put_pages_list_old); +EXPORT_SYMBOL_GPL(iommu_put_pages_list); diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 0acc26af7202df..8dc0202bf108e4 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -11,13 +11,7 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order); void iommu_free_pages(void *virt); -void iommu_put_pages_list_new(struct iommu_pages_list *list); -void iommu_put_pages_list_old(struct list_head *head); - -#define iommu_put_pages_list(head) \ - _Generic(head, \ - struct iommu_pages_list *: iommu_put_pages_list_new, \ - struct list_head *: iommu_put_pages_list_old)(head) +void iommu_put_pages_list(struct iommu_pages_list *list); /** * iommu_pages_list_add - add the page to a iommu_pages_list From patchwork Tue Apr 8 16:54:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880043 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2079.outbound.protection.outlook.com [40.107.100.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B91D23C361; Tue, 8 Apr 2025 17:24:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133058; cv=fail; b=MWV9ZnXJ/OjEpS0CfM5tvKEld6eOB3gzvoFPG842CVZ19sNCTBoeAggvIcN74JX4u1PKS+q2ZcjS/LdKB4VXBduCORJd6mHFa3OTTM7+kNcHYz1Sn30HoP/sVZ8fh6hj4xF2tqYSPiBF9Jw1m9gly6yzrkXfnQnE9dGDNIYi0pM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744133058; c=relaxed/simple; bh=6wNNDv/FLuAnqdhnWNs1BwH9o/SvXfM84YXhCgwSZ5w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=I2GqrWlcHZ3pvhdR+1rTMU1D/tSIPmzC3IqXU8lM0h2KtPD3KO2y1QJDB36eO0VcjS9hwHZ30DYOacW6Pt7goNg59LvSn+2AAVRbgPDznRYDdIN3+s2gReqDNycvhHARlYexeiFE/Rgu5WViQmunHsveKz81gDoCFhIfENgmwK4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ZFOYm7lT; arc=fail smtp.client-ip=40.107.100.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ZFOYm7lT" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mRtnC/Sugdn102/BVZgFUgjQa/9Yb/CF4vM/D+bMcRJP7sTbmhcN9C7UO8832ymlPDbOVT+4SXhKLyJXE8g3k8T2tlEbWNwi2MzcVGBACPjlNjZOyddl92dqBZgA9VokHdFjNgOPD2UT3D2rv7rjZ6AYGvWhOZWJdPCYtSvgkMGR9jaa4780BtCtwgd6B5BJU924aPwhwsoBpSNO/JwvPNbLrjJKCzI30KdbprR/U5Ve41IZ7J5+PXJDP4NhMtmTYeACdKTxUNQwc4dlEmoFeD9OJodnefMr9sy4WiQmRfHFJvbEWMbRjgnCkaJxDlIHtepI+/LkgfXBMZjuHWDjWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sqPksl4SKM63xlJ+mpbwQQAFMdwwGS0QSSlDFXB1CE8=; b=SX3gURGGvFmwr2HdQQ/8pjxOdAKPR5x9uy/a6zd4cydXIUwNLHvDBP0uNJ8V/xQgpUHQetcbt89FEbbVjfY44m1thowvR47ao1JrcEh7RC+uQwV/b2dCVIlfxgpgwhonw2QCH6wmUaU6vY2BNcC0pqm1VYXrM3f/prkChEV6X1SaUj41+T0LTUsySwzHcpIuUYfBwLOxrBdT7jA8FzjjGR7mos6RjkmoZblPJLH1zXdv12/eyuIP8F92LifHpbDYBIlwED85w1YDwM3PwHR+oFVG3YnxTYGRjKIybU/68zwrnpEqQ0f6oT4tCrw8Z7UbXWrRkP7nnhGXrKbRFHNVmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sqPksl4SKM63xlJ+mpbwQQAFMdwwGS0QSSlDFXB1CE8=; b=ZFOYm7lTiiDWKBKEatyXzWvWvDLFfl+tpSaDLgQblzLw4yPOzLK6EuH7XnR8tyCmFDfznP+38ATZptJ4aXVCpalInT8NIhAB4JJk04b8CORniqVtg8AIlw8RRJUtUnU27Mcicbd2LM79QfkxORnzXATtUWKuDRkh1cDoYV5GShPAvwN8ShWP3yqF7kCVSIexFFjSUUu6wV99hJmWcNp6Wp+YiNU1Sru/eFHsFgxQ70S1lDBtaoBJ4VRQe8rF2PYnGyAXGdkyuCUdfpkpKLbwsSZ3n7tma8/FmzUHC8Gm1mJRyq6x0PokywtaJch3GfCFnG9WzTmRcTDE9P5UByzSdA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by CY3PR12MB9579.namprd12.prod.outlook.com (2603:10b6:930:10a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 17:24:10 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 17:24:10 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 13/23] iommu/pages: Move from struct page to struct ioptdesc and folio Date: Tue, 8 Apr 2025 13:54:01 -0300 Message-ID: <13-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: MN0PR03CA0005.namprd03.prod.outlook.com (2603:10b6:208:52f::12) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|CY3PR12MB9579:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a1e09c0-62eb-496b-b9d9-08dd76c22ccd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|366016|1800799024|921020; X-Microsoft-Antispam-Message-Info: 692irVq9kwXx5v7zGQ9AV2orXKR9Xvt9MMWIlq5HF+QMCW74PoBYUjFXHQzQEPI8hJHKxTJg+h3+l4ACqzaeYzXp5sI1SQ5DCrTxgsGc7rbgW2W01VrFioxIm8nHou7Rp0GBraAV9gTJzjCe1cRU5o2sxXXRwtryUeauV5t2X/ZFIzW+v3da9vz1u8Bpj56bL++50tNAD9R2Qi1O0iNzCuBBBZLANzySDyQLDOiaWnCFimlzP443j9sTJMsp1nHYA3BgwC2Aq5QVUpZ52nGnjohNfKWpvWkxWwgVaHx4rx9wjX+YjB35M7S44D9KvJpCyQjXdN5XTFkElnLAd34MEUYnfxletugFJMfnNOA+AKSXr/zWwErKgQZFGzQ5lGB1lP2jp5oDLddhETDEP1CPpqI2BiATxN73FPheFzSY7ib7ljRD/wxJcTaPlH0P5TvdVYOUfr5ysqx0AgqJRT+Pv890EbW/4RX49OUlr1LbbELsKRNoUrU/IP1kJ9sG75gF35ypOe/RjI/sG9IOhQU1RUsvA5pP7XXMkZGO7I45hhrhLp+r3phNr8LhS7USpM+7juEWdODgOHE/CzJ1jXcaeEfG5PYsRU/4qR7c1WssuK2NUBSdZgqx7bB2BWbiq1xndPbkRMwlGZwXKKx4AKDAdpzBMTRguRRTSHWJk5Vdb4GTjd90UZC4qPkqFyEgOZkyvuPOpm48AXdeOZ/s3IJfD0WWD9+1UsevntflwqELgpUVue6MwLZP7JT3W/y8xhCcp/iLsNGkrh0QJGEPmudw1TUECq9M3OEgPRrI8PQ1NPn6LFjxx9aUKuQZ4pTf/zIPQ7e4z6JE9mwCXAi4Dbg3AcqHYkAwOj8yv3/oh1qF3GNTQaIMi8LbAIznz0C3TBfcfC4NsTSNJBpYrtt9CBUbbpGf3a/VbXmo3iDfLn1p7wyxppaitmkVK+CMrfmoo06nIw15ECSvcdkkvEc6Jkgbk4C5f1MerVKgIAPY1JF7N/XrOKTx9YYFvmERMd1JkHPaxg4okm3rC9/MzM4wg3e765WyB5f0rP7avsB1vVFn+dq0R7YliIlHqCfSqxEHzD+tD/YQH+bs3SanjEZ+C0HeFCeSx2aa1a01m4kChJiTSCnEiWOYNBXeZtpNw90d41Z97inRQTU80cD0NvBKaFKoIbYq9GrViqQnYOb59nK+/ETAUnAKDvsB2ZRrVFMVLDXZ22yQ7aG1jafr6SLoglKzHBgZn56NpUOiWwHdazuNkNSzaLp+BGk5vsNC9spuEoR5qOnUFBtjkCy2AoasaaqO/RGX+uUo8KaaB9TUhGZy8gNF9qtBUXkZZ6DJEuTJMcnsfWTOSXwrbRzqo6Zy2nlipe1MiN4JP5Q/47h0Q1f89t/Qv6ZF2hNaizl90V2ZU9BGNFY78NHtlW5j2qk1H0VU+OodJzqVhBTvUeue1w3+/LhZzopWUsc0wlIJWXmNBq5+hE6tiyD3/ZKRzth19WyOIQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(366016)(1800799024)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: P0GTFndX7CG/y7no9rtL8jBoW9vD9vps1h6c3hnegMwDBzHFmm4FngSe2JGBAZdSkH+iHLnMsskeDs6R1R8LgJhxFkz5fOqbteU+pmdkOzhyUOxYB7CAdk3tA1cYcb+FgmjzbTvcEPQL4J5qmRWUujDN0mmeUlGxajdvs5ZoMfLPS8qAFwCo0glgMg0HjPXsRv8pgxvyBnmn7k8mPwK9+Y1Pt4EX0b/ZMI4xnbRbBCaS1RdLm7kl7nIvSLptS7Wh8K/WdIAHoxmnCB8OgfFhI0w7jz/aQUCLjWeg7kxjYjBRhRkKYWxQvBwTwBDpAD/J33uKZYhZ1OAp4TwQkhiGWN1rEPp6oWCyQjQ2JjCR+zdxynGjS0CG3vnV5HfzxRh9Fy1DQEHl+0NTjeCIhROc0hTRO7ZffMb418liR28YcA+v5Gwkvpf4f1rdyqAVJKXXijY7oG4+Y6tVxYQVrqi5xLwniX4VIFeHYzCNKEaW/fySWQBHY6tyXZ15Bn+Lk/s3nzT/zziO2t8caC4ZnUCYG6GhE78RmAKuufbfML4abRD4Lw0B2j/FYtb55TZ7YHgOr8OwBdXFQmr5j+Sg9pLnP4hfBV+0qdnGu5wDaOboLWr+HLMOXS8ANTPHrFU1sq2DPDD48K4yHd+0OhFYglLhMBrsHrU9gurW3/MjSvW9s2EAFag2ki62EOY+ChSjjDOy5mn1WvbrW6rBEsqhCjZhHp2LgUkZuI+pbTvf3CTyUlhTNuzrlrSXm3MltyGffrkRnDyzpXbzleHoVAcTGtZJM630H+Y6wCX6v6iH0+PIoSRS0XJegar95aQMd7hdLoai8dCTINg2xickRSay1Q50aoSjG3xdkHwImG7KCNP67vuWEvHrEVK+sG3Hto6tayjGv64r0+vue/Dx+LncJjkkxhtAc/WDm5yySLrglo0+nBK0yvZEOU5AVWxH9BWhvkBk5ZBPtjTKyKixw1NcgNAG4zlGWzjpybsTrrOjUrh7QyLZS1q+c669FLOIZbmrAb8ieI2yksOI+bzgncnd++Zz4c/BfFX20cYZAZyU4cRX+ppHh1+hfVoEMWuhCLRIY+czOBfBa27HXYhTa/xq4gss5Srf6VzU3eN0oGlkWckMYmm409HJwz1pizh07NX9dFRuqWtHzAhDssQkOkf3sfu4zBDiyYZtQavNFKeB/282U6JULFlQY5tISrAVDcDHm/ipoHwncT3bJeoBQCm8OQsSblLkR33SnNOoIitspaGew/eCDIHFH/purXJZdU5gpFqQfO2Sxavg98dB8x5IB+iUOyUyvDoCCTQQILjauF4JfUgT5cgoTtZAYe8DV06s8IHp3vkFrLDRJfhyaxFVcQo2XRLzRR+Bv7UXkuR5LfU73Un35e+wQ9bW6H5IG5z7YMQ7tKX6W+bMvXtiVbUlQEPTO2W9shqPhjcBK7jdJDhjpl9Blp9VxcELsFJpEVuzj5jFjCTwKrak/QyYYzhxdI5lVD2WEpniCPO4phlK/bMcyktGyz2R74Em1eISQGHINI04p3Dqz7agOwtl129KKji9gCTVK/LvWqtJgyM2XbMv1wH8MjGrWccyXDACHW+TEbOK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a1e09c0-62eb-496b-b9d9-08dd76c22ccd X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 17:24:10.6269 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JoK7GYTRupfOlWF3rrjJgJHxeTRwAeQ4LvTfkgONnn4h+dogp9lTaBKLnQln5TMF X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY3PR12MB9579 This brings the iommu page table allocator into the modern world of having its own private page descriptor and not re-using fields from struct page for its own purpose. It follows the basic pattern of struct ptdesc which did this transformation for the CPU page table allocator. Currently iommu-pages is pretty basic so this isn't a huge benefit, however I see a coming need for features that CPU allocator has, like sub PAGE_SIZE allocations, and RCU freeing. This provides the base infrastructure to implement those cleanly. Remove numa_node_id() calls from the inlines and instead use NUMA_NO_NODE which will get switched to numa_mem_id(), which seems to be the right ID to use for memory allocations. Reviewed-by: Lu Baolu Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommu-pages.c | 54 ++++++++++++++++++++++++++----------- drivers/iommu/iommu-pages.h | 43 ++++++++++++++++++++++++++--- 2 files changed, 78 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index 6eacb6a34586a6..3077df642adb1f 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -7,6 +7,21 @@ #include #include +#define IOPTDESC_MATCH(pg_elm, elm) \ + static_assert(offsetof(struct page, pg_elm) == \ + offsetof(struct ioptdesc, elm)) +IOPTDESC_MATCH(flags, __page_flags); +IOPTDESC_MATCH(lru, iopt_freelist_elm); /* Ensure bit 0 is clear */ +IOPTDESC_MATCH(mapping, __page_mapping); +IOPTDESC_MATCH(private, _private); +IOPTDESC_MATCH(page_type, __page_type); +IOPTDESC_MATCH(_refcount, __page_refcount); +#ifdef CONFIG_MEMCG +IOPTDESC_MATCH(memcg_data, memcg_data); +#endif +#undef IOPTDESC_MATCH +static_assert(sizeof(struct ioptdesc) <= sizeof(struct page)); + /** * iommu_alloc_pages_node - Allocate a zeroed page of a given order from * specific NUMA node @@ -20,10 +35,17 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) { const unsigned long pgcnt = 1UL << order; - struct page *page; + struct folio *folio; - page = alloc_pages_node(nid, gfp | __GFP_ZERO | __GFP_COMP, order); - if (unlikely(!page)) + /* + * __folio_alloc_node() does not handle NUMA_NO_NODE like + * alloc_pages_node() did. + */ + if (nid == NUMA_NO_NODE) + nid = numa_mem_id(); + + folio = __folio_alloc_node(gfp | __GFP_ZERO, order, nid); + if (unlikely(!folio)) return NULL; /* @@ -35,21 +57,21 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) * This is necessary for the proper accounting as IOMMU state can be * rather large, i.e. multiple gigabytes in size. */ - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, pgcnt); + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, pgcnt); + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, pgcnt); - return page_address(page); + return folio_address(folio); } EXPORT_SYMBOL_GPL(iommu_alloc_pages_node); -static void __iommu_free_page(struct page *page) +static void __iommu_free_desc(struct ioptdesc *iopt) { - unsigned int order = folio_order(page_folio(page)); - const unsigned long pgcnt = 1UL << order; + struct folio *folio = ioptdesc_folio(iopt); + const unsigned long pgcnt = 1UL << folio_order(folio); - mod_node_page_state(page_pgdat(page), NR_IOMMU_PAGES, -pgcnt); - mod_lruvec_page_state(page, NR_SECONDARY_PAGETABLE, -pgcnt); - put_page(page); + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, -pgcnt); + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, -pgcnt); + folio_put(folio); } /** @@ -62,7 +84,7 @@ void iommu_free_pages(void *virt) { if (!virt) return; - __iommu_free_page(virt_to_page(virt)); + __iommu_free_desc(virt_to_ioptdesc(virt)); } EXPORT_SYMBOL_GPL(iommu_free_pages); @@ -74,9 +96,9 @@ EXPORT_SYMBOL_GPL(iommu_free_pages); */ void iommu_put_pages_list(struct iommu_pages_list *list) { - struct page *p, *tmp; + struct ioptdesc *iopt, *tmp; - list_for_each_entry_safe(p, tmp, &list->pages, lru) - __iommu_free_page(p); + list_for_each_entry_safe(iopt, tmp, &list->pages, iopt_freelist_elm) + __iommu_free_desc(iopt); } EXPORT_SYMBOL_GPL(iommu_put_pages_list); diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 8dc0202bf108e4..f4578f252e2580 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -9,6 +9,43 @@ #include +/** + * struct ioptdesc - Memory descriptor for IOMMU page tables + * @iopt_freelist_elm: List element for a struct iommu_pages_list + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. + */ +struct ioptdesc { + unsigned long __page_flags; + + struct list_head iopt_freelist_elm; + unsigned long __page_mapping; + pgoff_t __index; + void *_private; + + unsigned int __page_type; + atomic_t __page_refcount; +#ifdef CONFIG_MEMCG + unsigned long memcg_data; +#endif +}; + +static inline struct ioptdesc *folio_ioptdesc(struct folio *folio) +{ + return (struct ioptdesc *)folio; +} + +static inline struct folio *ioptdesc_folio(struct ioptdesc *iopt) +{ + return (struct folio *)iopt; +} + +static inline struct ioptdesc *virt_to_ioptdesc(void *virt) +{ + return folio_ioptdesc(virt_to_folio(virt)); +} + void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order); void iommu_free_pages(void *virt); void iommu_put_pages_list(struct iommu_pages_list *list); @@ -21,7 +58,7 @@ void iommu_put_pages_list(struct iommu_pages_list *list); static inline void iommu_pages_list_add(struct iommu_pages_list *list, void *virt) { - list_add_tail(&virt_to_page(virt)->lru, &list->pages); + list_add_tail(&virt_to_ioptdesc(virt)->iopt_freelist_elm, &list->pages); } /** @@ -56,7 +93,7 @@ static inline bool iommu_pages_list_empty(struct iommu_pages_list *list) */ static inline void *iommu_alloc_pages(gfp_t gfp, int order) { - return iommu_alloc_pages_node(numa_node_id(), gfp, order); + return iommu_alloc_pages_node(NUMA_NO_NODE, gfp, order); } /** @@ -79,7 +116,7 @@ static inline void *iommu_alloc_page_node(int nid, gfp_t gfp) */ static inline void *iommu_alloc_page(gfp_t gfp) { - return iommu_alloc_pages_node(numa_node_id(), gfp, 0); + return iommu_alloc_pages_node(NUMA_NO_NODE, gfp, 0); } #endif /* __IOMMU_PAGES_H */ From patchwork Tue Apr 8 16:54:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880047 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2079.outbound.protection.outlook.com [40.107.94.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F045F235359; Tue, 8 Apr 2025 16:54:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131274; cv=fail; b=uMLN1rd5pjEhbAbonPmrX7/uaPLev3DJWaMMs9pY0OR7Dxew7qr/JfIXnnMKztrQ/IdwK+5DqIt6Zg9HDfY0fi8/q9eOXBZf69mndsN+xPV5Eu7e8lv3W84zgZaBfinIK5XbQId7fcxTJrhNtSVVZf26Qyl/bqqJ6g0Yaw81/rc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131274; c=relaxed/simple; bh=uLo1bFs/yKgW34CDL6y+dR0YKz1Mi/5MfUXnumkjWWg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=DXnp48uHXT9+4lKdDrTTtU6zQfsWxwweZA/VTb2GOzW/6hvwE5lAWtKlGVlE4kZI2jRdvufBgE3pbFLxLZrAzfQE2dl+E3yjlhztnc7Bz7QIY39vUFNHGctZK0pjaxsnghgTcmX6B6rY2FcIkq/Ut4jVegcH6lH8qQStvfmxMu0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Q/Dv79Ya; arc=fail smtp.client-ip=40.107.94.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Q/Dv79Ya" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CbuAwrmV4W7pCsXdKSnr21en8tyGx4vdu2rQwCMv8sQ9c1+VhItzX/wmhtfMHaU7s0Nk2YSsCB1V4OlCS07XaP4LoAQHB/RuuLYR70dBKa+TVxsu437w6hGqt2v7ZPMLE9z1E4KQgZGdSsJ9HcpHgX1Udapk4Bmwujo/RILPvb20QgVDzCl7WtmGCsNvOMLaMcULKmQD42PAxCS1BcfzHxA/p50yx4fPAe9E0Jq9kQ24Pds1APs3mSYLeNHuYryrc/VrOfkctQrZnuFb9pe+Bk3t9bLiKTbfkVlmOEB5/g0N4NJ7by07e6FdIXLXNTpyqDZbqeatgeiQa/jW/ThgTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bBNST4THBS9DADg0qEI/c1+ZsFnLYj/Z34pwoczjR1c=; b=hXFxo5uBjeJFHEhFQYNHE34itW3N6LlRxllypi+qsfnUSEkBxQY9xfuqcq1ayvL/MSOjI3sEkjmcX0BrjxJLGzfScssfpJbdlCQZ8pAby8V+w0FIw6gfc7FgkDFlrSkNKwRx4fGdg78gP3/h9VfEXLL1XEPDG5m8BYWnE8QjFOG9fGQ7FVuDBCx+ZXss2h3cgJslJumrjRYVVLZvecul+pZrJAaKYzdGt+tfEjypqpVEFDHhw8yLWDhLiD2JMcm7vrqjftz/HybjUYlLzMZWCNZfeg18MpgBNiZ6rbaiofD22TNid0V8PfWYZbU1cqsisAEJl7cTy7tMkdX0PjJMZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bBNST4THBS9DADg0qEI/c1+ZsFnLYj/Z34pwoczjR1c=; b=Q/Dv79YavDQo0ZFCjpmqr7MNCdYc6h+cS09IRURhagVcmu3rnOO5F4JyF8x5M2m2mk1QLPJh3J0G6s8t87VEcXDtv76R6wNgUO3a6mH0lDPXXFRQaekTHLiXEIV/FRSxHjkTrumqD7xKt7dKmJTvjmRpFtlULs/R48LmSLiFD0ljLa/Qm7V0oiLptTyTyEsFC8j8IqEAqiUlQhXnmY3UaHLD8Dzn84a/+CYO5+OvLxS0QG/YgTafMhUjbMjvWM4eyGkgqdNlWrYyu5SRctzEq316E4MANqZUgpucfn7/odPXB9PfANViA9rQibaEkp8ZzJzyu8z5JQ/WixrgnLrb4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:20 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:20 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 14/23] iommu/pages: Move the __GFP_HIGHMEM checks into the common code Date: Tue, 8 Apr 2025 13:54:02 -0300 Message-ID: <14-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BN9PR03CA0046.namprd03.prod.outlook.com (2603:10b6:408:fb::21) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 47160a7c-d94c-4bf9-9fdd-08dd76bdfef3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: n4JwgpjF9PTwaVsUMfaf1Pea8Cjkgoc41sYQISLfHkTli7r9L0I8T+sz8E/VzjEAGQBQvTDQPftIqq+DI2n+n/bL4m4tjjy7wNs3j0ss3kl+VX29LTTF+GTxh+MVHfnegI7+0lraSiSK4aDz6O9ALs0VpQoXeD9DgLjd/qIa845G2u4D5+g6CebgNf9OPQt6ntukVvd6CE9CCz/iYrFpzHLBtxCFcYtDiz15CkYkwTi1cgA+bBSI1y9+B4HuS0TzRbk91w54NKzYTvw/yFbUOh4sDOmHL0XxZD5hZDGxak6YPkVt6R9Jsxka+PL7MnokUnNrFxlaEFWVXBtEsx3fez4ylQukMrEL5MCoQ8BZTG6N8OkLrnujJdLCHiiqwf+WFsVV+AVM+60p8iA6DIW96KWp2p0l6ucrhZS2z8Sunc/fMKX/+llCbGF2TNxE7BpsKz71p5SNpjOE3shGEo1g+HAJQUcrHca7iN8TqIy+6nBUVNRjwCPGg7TFO3TLG1aTntWT4NnHYSsojhi01OOt55aDUc0FhuEQWU48js2VkuNsUZxReyWCqGt7Xnu5qKJj0c5JGaXUxxF54BtAI6iyzZ/ZtNYiwjlTcYqwaSILBsnihZm9d/H7dm92Hy/rels92SXULis/D134EcR5beVN5ZQW2hrU66SG7W+5x09nAaueQ/5Gd0YYENuvQ61pm90AfgbVsLeaV8IHTK1zlib5Ei+0xs+ubQzXaLpAsB4HXxrPnL00M/iWNNQ7WQ1nAKnBpKekyDB+bCauTYhDDzkd9/7dXlipbInaBRlXmUZtFCJ6nMVy7vH2rAR0xd+81rznmrrY9oesvCPZuucBphWhsMzIc4eLmx9/r8SFjIjH5EkwIPG7S0EV4VWKkKpgys5nwtdGUURorJtm/EKGnzuN+fmoEI3zEvWRc2vE96GAO8L2CycvefFMHEiKNqrGeEodxFNifi99rca+LFBuEIlUofhnloVIOn0IiNWb5i2hPavluEGJrtloMBqWJTLP4r7xXQHJe9FlgULOQlw8Au2yvkZU2o6kExoohiSpG8OpIb5qBEupH7jvlVov/jrKGyTAygMTUYqei08iNBTOfPumRVwxrkZapXPqxjBU48gn+Xs23lDb4hYAFMfXAq7IA3Kd1FSIimbHlEVC8bywY1N6Te0xjO137q6mZwVPwQTQZq1PJcDcMOOsQN6qDvGCd23kfRNnf8vVs1xs6u8D9rUc72yZugjq3J5eugykj9TAFEzFPhewlQq4rsIUWElVLIK5W4BmQkXppcL4+veK6vICjhfIHz5Z+lVTI+pvmijsJzt85h6W37FNn/gYkTDjB13xH34NocJ5/ASJlX1n96VB1/w4iRs1ZFnlQmIn1lCLXHHt/vBc5NhnlQtJCwddlo+VBhobamhNWNXOhA9/MxIE2gzAuy0pr1lINVPVnfpvKvs070/CprK9FRvOgyghvCz3ciJUoKum7eyINuRSi5C8iw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: D52o+FLo1K0N4l4USLz1jWpqnkum6JnlJlzaKF/YVvTt+7B3oPkzV1X6Nz0ndQHmh8vkngPY3scLYzNE0qbXLzXwjvrQfzqq5rBiHaaZ6JViaq8cveGwjzjevZkBtybJWEtFLRc6vZEFMbbs3Mr4i/Fqh6PNmxdFNb3HgT7a2GIvgSFK24fzeph5A7fC7QB3suOFc4tspD8otS7gek9IEpcQZAIU6N3pFwocHwxRBXob0G1gd51iPuIBa5pCUsivZiOVizXt/n64GFVR09MgSfM+p9PqwoQpBb/8VlMUFCmE80HXEcqhOcoLQd8izKaxyqVPPRsQJbCgEHDAuKeOUV5wigvw7dKCUgpvBkPxT3PHXxBjEDzr5kCoLoQDhU1VNXeLcL6rbLo65iRUn/4fC2NccMd2ezLyCO1jjt9gbZLyBhzBMY3E+dbs3de3QhfmSuYe7HXbxw6NK0tzVjfN03Ks7djko1wum59MFDEhI0gZhWEL4YykVGIMhczCC/6iz8nCcCekWuFJ+bt7v1lVMsJYYuvPK5LcJaRGCalfZSuEuit0tdGSwutWsOWWOjVaZX1VkYCfybudq7lIvHD2gVwi5cOlKsf28TRaSmUijJUa8a+QF2GH2Gm55U+++s+rjfUjrq3+wnbElH7fuaTQWa/05CSW5rRYRBhylxfvoktJnNcxo3tGH5AGHEABC6e26adNfCaewjQRFACpHVHv9/teRD7B3l0fmA+f09ymkdcCwQ/tv2kyjLY8xuHUmONE+5OjPmw75cEdiiWQojtyWsnQGi2S1Cvhj2XzHXnYJwIxqmVNYlbOMj6yStyppewsKQZBv2+YkVNDQlgfZpJhtdjq0BsfYZ61dn+8alw2Fl370HxrgarS/diRlficV5eYS4Z1AFiowXT3SyTt1WXztVUYbwyHjq+7ih87vtttFjprB4seCdRzJ/X1z9+StxIZ9E0+pqFmw5l5FvMNHCYGEzHorRO2cMPOfLIsq/z5KVgm/oJhiYw96okUQygG5u9HBFIpB4dx42hJgjeG7oVMqjl6YFHW9SEzEeVQcGGCxh6CYzEoNbdEbpKDEnjXWSJVzq3GGFseMuhrP9QvlzL/AANF7O3WJJuoTzWB8DyV8WE+NE8XpPTvURbSZOukNY/C65fgZWaqe6dROnggygWJYNYGFxxuFeSnb/dB4qRdaORAG0P88GJ8LBahKrwV5/bdRTs8QG/2NE6e67ECGpPrf5+hbTTXoBT7g5XVnM3cKIYZvR82sA/ye0ljFQWLKUCKkNBaP1lkGg64ANtz/OlhN+o6Ds8ykNRkTV4fC3+IpA1DpLGlAaotr8qe7k/B21bCmx3kqkoYqNws8AQXxxVSkN5Fm4ALZeAYl40g3ZimtqjFSJ16YRVXd4Ok765P2xksRBuxs6jRYgZd2zQrPhASvtnzrtIzm8nxFDRcgvCqydq/byybUkh5lj9n5bZyk4O/aeueBIQSNunTut2kFS/R3uC/gGwPl3rsWWqbnEeR9MA9vDXme4FVjh0RxUvzvBoqTc/9RDb0g33nB1qTKh55FZmqjQ2Eiw/ISlk/za6zKE3lqsjhlfDCvSjW+Z7t2PuB X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 47160a7c-d94c-4bf9-9fdd-08dd76bdfef3 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:15.8495 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uZO8LL3OauZehwbkHBs7ruTsW6Tc1Kxy7xetDxOrv5kQKjEtF5ZisaXXClh6RcQW X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 The entire allocator API is built around using the kernel virtual address, it is illegal to pass GFP_HIGHMEM in as a GFP flag. Block it in the common code. Remove the duplicated checks from drivers. Reviewed-by: Lu Baolu Reviewed-by: Mostafa Saleh Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/io-pgtable-arm.c | 2 -- drivers/iommu/io-pgtable-dart.c | 1 - drivers/iommu/iommu-pages.c | 4 ++++ 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 62df2528d020b2..08d0f62abe8a09 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -267,8 +267,6 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, dma_addr_t dma; void *pages; - VM_BUG_ON((gfp & __GFP_HIGHMEM)); - if (cfg->alloc) pages = cfg->alloc(cookie, size, gfp); else diff --git a/drivers/iommu/io-pgtable-dart.c b/drivers/iommu/io-pgtable-dart.c index c80ccc753b5e60..8b5403aae8c0ca 100644 --- a/drivers/iommu/io-pgtable-dart.c +++ b/drivers/iommu/io-pgtable-dart.c @@ -111,7 +111,6 @@ static void *__dart_alloc_pages(size_t size, gfp_t gfp) { int order = get_order(size); - VM_BUG_ON((gfp & __GFP_HIGHMEM)); return iommu_alloc_pages(gfp, order); } diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index 3077df642adb1f..a7eed09420a231 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -37,6 +37,10 @@ void *iommu_alloc_pages_node(int nid, gfp_t gfp, unsigned int order) const unsigned long pgcnt = 1UL << order; struct folio *folio; + /* This uses page_address() on the memory. */ + if (WARN_ON(gfp & __GFP_HIGHMEM)) + return NULL; + /* * __folio_alloc_node() does not handle NUMA_NO_NODE like * alloc_pages_node() did. From patchwork Tue Apr 8 16:54:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880051 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2084.outbound.protection.outlook.com [40.107.93.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF42F237703; Tue, 8 Apr 2025 16:54:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131266; cv=fail; b=YzE/OKm17mGiaAQPam0pS7PBQt7aaBTDZ887n1IKWEdDV1crgh0O+08z9Ah7ZwJmmvMLDW/NFq9rxWpBqMyDgSPS32pZ5+lgu8tfa2lJ+eka5qm6Wo78De0bVM24i1gmFkPJy+BQm4Po9Bz/y3gcEpVoBztjmaZMf+OLlIUU9uU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131266; c=relaxed/simple; bh=y1k1jt2PbqgpOzmOj87gm8x5RZvROdkCeQEd0+akElo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=VU3JVZ2ZsuHeAJqD6nkHEXWZS9Pl1kiOt+6HjvrCvR7hXrwMRsQAAYps2OW8DfiNAdIUKkImntUrAOrcOxlxhYR1xQMg6ICfNxO2k7JCFatMVy9Yz/zanGyrUrylYA6Vgjm5DhWfu3th4qo/7xZW0l1LdGS54RzLz2ABrJ9yQdw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=AEMfWwI5; arc=fail smtp.client-ip=40.107.93.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="AEMfWwI5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SAEBhMjzW5n3+n1aLfC8f4yLResp9ho+cyBH3WHdJgyVvD1/uZVuK/BR0SRxfmAoih860tSKEU3GwZvxvVUvdHqii1LGWNZ/APc7M9C7gfmMX339MWiEt/YAJrKwX8/V7d9fen740I6rBysjOjVxaIsA4o+IkgHJOULNuK97xDS7yIOJjbSOJ8YaNLKxAryxxliGbvkcmQqLd2X+d6rxYpIaGh/HkJyzl2bSHqI2vOom95szdyAe7BrbYVMnUe1j+tckH66RudhxV1zRo7FWYzYLLt90qQGl1R+q8qwx5PkS+nD/zZWV3NMN7NYBABFp7Kt7ASXARBDwC1ttOPvGIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8yWR11U09nyhGLEmSthuTAaf3mrkamzIJMdLWHxAsZQ=; b=Sldpk1HojGw5d5/rO5III5kq9S8Qzvg1mGNnB2JkzYhzfRxjPTaMUo2OgXykTQpcG283aIaNf54hVkd+5CxJzDDu5zt4GjC7yM/tFBQ8X50o5FwbR1a9NEqvGP5LmYgyaZVgKvgWDcMwSSYCv0WZ5ejnmHACzf/yWaramZp5W95bt7pJBqe+hBe+/MOUGpJp8AxPCc1cYpQjxo+wrQdGDWc4X2nDqDl6iYQ9PaJkT05F4VtYEWj9MYmYdaS08mRgG6JhhyQPEjuWP5bzbLeh/W+RZh96Z1oUWzoBO1/zJaupBVGQfkk5JeiqvzTVS0haZKh0WlNA3VkbSvaKeSRPXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8yWR11U09nyhGLEmSthuTAaf3mrkamzIJMdLWHxAsZQ=; b=AEMfWwI5BeW/33U6uL3c09zYcgiFco/vrUXyrVr/ZQTz54cZtoBqP4smMxEozs7MJETmfJdEdYBNOHdaDuBhJIgfA26BCXW2U18Q7Ji7q+SUWWqnclFeAntj2kHADRWxuw94ha3hldVZVKrwzZM7wuep2iPvAssHurg18KJZ1o1zcbGDpXzTbhrGqhL44Spdf7SododYallXCKkegh2L1ZjsS4QmpCNnpru9J4CghcW+99I7jAEZreZBzrS0dmyqRv5XXMBccm2+aGk4vtpNKBWJAPFRF3o06IryGvKTTiDZMdVmd4WJJwhrOhjAxf+v5cNdz5HPxcUhKF0Kp00N4w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:17 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:17 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 19/23] iommu: Update various drivers to pass in lg2sz instead of order to iommu pages Date: Tue, 8 Apr 2025 13:54:07 -0300 Message-ID: <19-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR15CA0056.namprd15.prod.outlook.com (2603:10b6:208:237::25) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 35ec5830-dad7-4a26-0c87-08dd76bdfdf0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: TiJRofcbq3KLQQn+qwgp12jT8w0iS9XHuw6saDZJp0rm0F62rSsrl/2JIgFAlzPF1vJ6z29t18b5jQPOst5YGI8Skh/GbhTQMqOHRRWNuU9MB+Pjq1b1Yo559EASnnEQh83J2AFO7Vn/bp4tIn8K5OTZ39u1ZDAo3qazTpJdVvGIrLWZV3DHAfAbMHIyBNt6P1lADmai5ZWQDLPeYP9VRysyJBwdgaYBtSVDyTZU/Dq4Algunth4mjsIvSd2jvJd2/c25s5RcUFqxaiKiUljzzEfEn0JHhIdYBW6jq1bS9ObeQkhyd5MyPowvcP7xXA7W+R0jOA5xVueDIQNewEOXMaWio2Vq3R/ZG7Ag42krreqzp1XJJTceLUjpkuVLvAb4ZnIiId6bBam8lfmKRLvDUoIvcGw2wJQTVHl4nHMG9sZJII56LMLfJS9SeoxaAEpkqsAC68RykvaT2bAGRmHZSV2i5TOyd0WDBuBFK/KJoLQEzYOKKmfCFT9TaabkcYHOhmuIePBJvfEdESyQVkgXynLjBYHph4Y3dTh0EXK+zp2gletRoGE9iKO+pGzpZR3swLr4BtqLScFPPl3Hs9OzsrwmCd0F9zm81H2aJ+9eLj4jh95vb41trPMjzVKAEKjzupkJ3tpfNe0txNPNtSBsN5S+qotUi5qQtVMUS03siDftWWRnZiMFwT7I8xeVmjxhFlsO2jKzU3J8KjqSyQjkPe9QPvt+pXGWzqon7Z3kncO3qbqyNF27SGdcSBMREbHA6VeBD0aC/aYTzYiph1YIh7I32TfHANtDJrAqUE4tafxNeqP65Y9TVc6xwlAhXu9366/x4/iYxwZ2k1aNIEV4DRM71vRz5v+VrQgmt8IluR9t5lcf3ujdpVi8CWSz5KcW0Ehqjf4cDgZ5NbSYK2X00gTQIqkFtqSY++5TSJPxe7W/pzWwp66tsZlhHhfuEC3hSjTNDwEYdAEdQv6qaIoK+GxUoFKElLEphenGflR7lzyyekBMO3f3RwmRl85JYUt9hjoBfu1N/3Yujhd1ic8Vh0ReTRc9XXhzwxno6we9v/slr9j0OJz1htpJ7dBASGuDaFbzU28RQK5jpGA1bWAF9X6s8Vp1h4vcA2Qbb337RQLaLTAppYUFThwQKgmAYSUsQYmt2PySYM3NdLT7qIgB3LrB1L0CmXyeGvQ2ih5j5gLnhLt0wVtmgDT6ziwSv8EKJl59/ha6RggXSAoR9sNMGEq+ojDOHGYCtTC2ebijt2aZcl/GDSxPLBN9YEh4Qpo+0ueV5/t15YD/789+Bsarirlrx9t7aHfKKm6/JMgufQ9hVqLb00kGA51j+Q2wh6TMrs4YN84FWhBHLS9SBz/o+dOyOt0IXkXXFD16zHZhYjFC7iPEWHJHCyCqz/fFNRHV9/DKPmelarGp40QM/xYH5xghEaFTW8i5UXY7azRg3wlcN5jOEFnl/Pme+Glz22cRMY+EimNNUxtFxChkAmQvw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: rKIBDwvrKo49wCRT2/lLVx5yElcb5aFYr0UpkLmU+rCyZv793YdLlrQK4SyhQYH+3suWRiLviWgp2DOySxv+8qLN5h5xp0obiPHHaHOxIapClvH/S3btuHgHxzIodFl0LRJbvF7pwq4Vz9xbB4mm47BRDq7bll24sKJWFa22df11abEBqm7QEatZD3vejd85giBOp2XCnjTbcPp/3blgid5e7ztZDypXTZlW1HhJ7KLn6va+bWhWXBiyrsqEvXKHdYx4/pU6PJpOVb8BMvBgnNKMItWJs6iOpMDOVNuI6nmC9LZnPecBY3Y/FRLllGTFAFYUGNE7Q+3+5pZe7ivQ2Sf3SLa3sC4x8skpFnNSn4yX1ao3cWNINEryy6qVTt+KnsA521p4bV+8gDb3RXeek4iKxCWYa3ohu8gR7hcaH1cujSsyVwkPQUoyW7rnZ6IsyFd5iuHYpc0dohMol/5gFKy2FRBjiPpuk8cM1dIXLOLt73BjQz78Oc3LTtm/3yVC3RDm+jjZoqH+CO9t+tE8yXiwRy5PPs8HwOX1Qism+VMmOmRO9y+QOqYZlDZeiWzwbD0h7Oi+hgduK0E/zk8/W0+/1/HYdU4ELRLGLhXo6jnV7MOmA8jLN52eNH7SxE12RTyVQGfPbxWMBZvOWiR1FHYJ/kDfQs85WtjRwGUlzZtX9QUIy8ovAJr3NHS/M3mHJU2PQOXAoWRMH9G4f0FDReGHfhPkd2oZL84+GEdybjKFHzyrsdbUlTGU2MApSPv0W6b9KuH1Ic+hl4ygzT3O58l2QTy1V+B0cArm3YBPs1kNLiQBmGhIiU7p500uh3l/kmR1Sl5oiZJiHCXRQKlVuJ5uazPsxlZ3erPpMdGYu9m/sSkr26WkYt0R8hZ6ZsoWYMkWsVNUB3xH2WQ5nOn/iE4e4a/uGG9Y4hObWHcvd6Ca8ogSfapyJOZdYk6ASgqyKun4VMUr/hn8b8RaZOtOwTQwxcGte4lmn/GXbykYXXYNzjbDrZAN0gCFtePU8zvzZ5m5Mg6aQKOCzflPdzULFe9h8wP+pM3qrZKn7KXiSLzaYhrk/MlDrNq2KBRJ1OrVYdA60grESyduAMWd2+fHNCBlqzl1STQubQnNSQ3y7aUYkjl6ZFvURcfbCsWWq2enfHYxRK/p0a0empQJM2TFjwNPNgF3e7r4WewWXfwX12nFLyAGJB3vMGZAgJX8hn+xstBera9CXjxOsybC3GMRFSpUEluPpDz96Y58mN1W5+tN+JMkvb9Mlt9J/22EeDYXrky9g56f4j1QO21eU3KysOTjhvIqVH91eUERxERZH5RkFNHFDYAwRF8DPbBvwkspYbrEYkG49kmELivoMnpfqCVg2YOYiLQZF7VphzuXcPgejfKMvv1GQ1N1TC59RNynrBuLkFYybG7WXgcFPqSADiIb1kOpOGE0GsrA/h7SNdjIXN3scyl+3cfOUeUDY2WO0N7qs1rKynGWy3n7Ul/8mc8VkVmZ1HOTvUcdrmZIwyrKu+8f0Dzinn/3EFIiQEQ2Nj7Zup1+Gh+h9o2VO0TocRtHwkAS7rA0fl87Z/rOoWjsug0KiUqZv05J/sBmsegc X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35ec5830-dad7-4a26-0c87-08dd76bdfdf0 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:14.1851 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VTqhVCF5FYo0YH614cpzKXfBE4EOst0XPdMz7Mj7uBQ+YIXxd9c7au+I7GyAA+Tc X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Convert most of the places calling get_order() as an argument to the iommu-pages allocator into order_base_2() or the _sz flavour instead. These places already have an exact size, there is no particular reason to use order here. Reviewed-by: Lu Baolu Tested-by: Nicolin Chen Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/amd/init.c | 29 +++++++++++++++-------------- drivers/iommu/intel/dmar.c | 6 +++--- drivers/iommu/io-pgtable-arm.c | 12 +++++++++--- drivers/iommu/io-pgtable-dart.c | 12 +++--------- drivers/iommu/sun50i-iommu.c | 4 ++-- 5 files changed, 32 insertions(+), 31 deletions(-) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 13b5f83e642369..df1094893eb0d1 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -626,8 +626,8 @@ static int __init find_last_devid_acpi(struct acpi_table_header *table, u16 pci_ /* Allocate per PCI segment device table */ static inline int __init alloc_dev_table(struct amd_iommu_pci_seg *pci_seg) { - pci_seg->dev_table = iommu_alloc_pages(GFP_KERNEL | GFP_DMA32, - get_order(pci_seg->dev_table_size)); + pci_seg->dev_table = iommu_alloc_pages_sz(GFP_KERNEL | GFP_DMA32, + pci_seg->dev_table_size); if (!pci_seg->dev_table) return -ENOMEM; @@ -707,8 +707,7 @@ static void __init free_alias_table(struct amd_iommu_pci_seg *pci_seg) */ static int __init alloc_command_buffer(struct amd_iommu *iommu) { - iommu->cmd_buf = iommu_alloc_pages(GFP_KERNEL, - get_order(CMD_BUFFER_SIZE)); + iommu->cmd_buf = iommu_alloc_pages_sz(GFP_KERNEL, CMD_BUFFER_SIZE); return iommu->cmd_buf ? 0 : -ENOMEM; } @@ -811,14 +810,16 @@ static void __init free_command_buffer(struct amd_iommu *iommu) void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu, gfp_t gfp, size_t size) { - int order = get_order(size); - void *buf = iommu_alloc_pages(gfp, order); + void *buf; - if (buf && - check_feature(FEATURE_SNP) && - set_memory_4k((unsigned long)buf, (1 << order))) { + size = PAGE_ALIGN(size); + buf = iommu_alloc_pages_sz(gfp, size); + if (!buf) + return NULL; + if (check_feature(FEATURE_SNP) && + set_memory_4k((unsigned long)buf, size / PAGE_SIZE)) { iommu_free_pages(buf); - buf = NULL; + return NULL; } return buf; @@ -913,11 +914,11 @@ static int iommu_init_ga_log(struct amd_iommu *iommu) if (!AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir)) return 0; - iommu->ga_log = iommu_alloc_pages(GFP_KERNEL, get_order(GA_LOG_SIZE)); + iommu->ga_log = iommu_alloc_pages_sz(GFP_KERNEL, GA_LOG_SIZE); if (!iommu->ga_log) goto err_out; - iommu->ga_log_tail = iommu_alloc_pages(GFP_KERNEL, get_order(8)); + iommu->ga_log_tail = iommu_alloc_pages_sz(GFP_KERNEL, 8); if (!iommu->ga_log_tail) goto err_out; @@ -1012,8 +1013,8 @@ static bool __copy_device_table(struct amd_iommu *iommu) if (!old_devtb) return false; - pci_seg->old_dev_tbl_cpy = iommu_alloc_pages(GFP_KERNEL | GFP_DMA32, - get_order(pci_seg->dev_table_size)); + pci_seg->old_dev_tbl_cpy = iommu_alloc_pages_sz( + GFP_KERNEL | GFP_DMA32, pci_seg->dev_table_size); if (pci_seg->old_dev_tbl_cpy == NULL) { pr_err("Failed to allocate memory for copying old device table!\n"); memunmap(old_devtb); diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 14bc39b2cdd528..10374b684d6dbc 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1681,7 +1681,6 @@ int dmar_enable_qi(struct intel_iommu *iommu) { struct q_inval *qi; void *desc; - int order; if (!ecap_qis(iommu->ecap)) return -ENOENT; @@ -1702,8 +1701,9 @@ int dmar_enable_qi(struct intel_iommu *iommu) * Need two pages to accommodate 256 descriptors of 256 bits each * if the remapping hardware supports scalable mode translation. */ - order = ecap_smts(iommu->ecap) ? 1 : 0; - desc = iommu_alloc_pages_node(iommu->node, GFP_ATOMIC, order); + desc = iommu_alloc_pages_node_sz(iommu->node, GFP_ATOMIC, + ecap_smts(iommu->ecap) ? SZ_8K : + SZ_4K); if (!desc) { kfree(qi); iommu->qi = NULL; diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 08d0f62abe8a09..545229cf62d2d4 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -263,14 +263,20 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, void *cookie) { struct device *dev = cfg->iommu_dev; - int order = get_order(size); + size_t alloc_size; dma_addr_t dma; void *pages; + /* + * For very small starting-level translation tables the HW requires a + * minimum alignment of at least 64 to cover all cases. + */ + alloc_size = max(size, 64); if (cfg->alloc) - pages = cfg->alloc(cookie, size, gfp); + pages = cfg->alloc(cookie, alloc_size, gfp); else - pages = iommu_alloc_pages_node(dev_to_node(dev), gfp, order); + pages = iommu_alloc_pages_node_sz(dev_to_node(dev), gfp, + alloc_size); if (!pages) return NULL; diff --git a/drivers/iommu/io-pgtable-dart.c b/drivers/iommu/io-pgtable-dart.c index 8b5403aae8c0ca..679bda10479776 100644 --- a/drivers/iommu/io-pgtable-dart.c +++ b/drivers/iommu/io-pgtable-dart.c @@ -107,13 +107,6 @@ static phys_addr_t iopte_to_paddr(dart_iopte pte, return paddr; } -static void *__dart_alloc_pages(size_t size, gfp_t gfp) -{ - int order = get_order(size); - - return iommu_alloc_pages(gfp, order); -} - static int dart_init_pte(struct dart_io_pgtable *data, unsigned long iova, phys_addr_t paddr, dart_iopte prot, int num_entries, @@ -255,7 +248,7 @@ static int dart_map_pages(struct io_pgtable_ops *ops, unsigned long iova, /* no L2 table present */ if (!pte) { - cptep = __dart_alloc_pages(tblsz, gfp); + cptep = iommu_alloc_pages_sz(gfp, tblsz); if (!cptep) return -ENOMEM; @@ -412,7 +405,8 @@ apple_dart_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) cfg->apple_dart_cfg.n_ttbrs = 1 << data->tbl_bits; for (i = 0; i < cfg->apple_dart_cfg.n_ttbrs; ++i) { - data->pgd[i] = __dart_alloc_pages(DART_GRANULE(data), GFP_KERNEL); + data->pgd[i] = + iommu_alloc_pages_sz(GFP_KERNEL, DART_GRANULE(data)); if (!data->pgd[i]) goto out_free_data; cfg->apple_dart_cfg.ttbr[i] = virt_to_phys(data->pgd[i]); diff --git a/drivers/iommu/sun50i-iommu.c b/drivers/iommu/sun50i-iommu.c index 6385560dbc3fb0..76c9620af4bba8 100644 --- a/drivers/iommu/sun50i-iommu.c +++ b/drivers/iommu/sun50i-iommu.c @@ -690,8 +690,8 @@ sun50i_iommu_domain_alloc_paging(struct device *dev) if (!sun50i_domain) return NULL; - sun50i_domain->dt = iommu_alloc_pages(GFP_KERNEL | GFP_DMA32, - get_order(DT_SIZE)); + sun50i_domain->dt = + iommu_alloc_pages_sz(GFP_KERNEL | GFP_DMA32, DT_SIZE); if (!sun50i_domain->dt) goto err_free_domain; From patchwork Tue Apr 8 16:54:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880049 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2067.outbound.protection.outlook.com [40.107.92.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D6992343C5; Tue, 8 Apr 2025 16:54:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131270; cv=fail; b=a+wQ2iTLBvgzlTncX64tMMAi0mf62qeITmU0gvLH/L1CtkfSqvkSyB1FGve6ZZDbAUBC8wfQk7ljrorIyLQWjN7+qX5I41id1cZilx1m/DsfLL6n9plvwmo1LvLDBJfUi69d2Ac/4f1jCBXLwnqX1rEm0+Q0hPMPCbyvLLt8DdA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131270; c=relaxed/simple; bh=JlnbcxNqhs2Forf0o3w7wx56j95ABcPtHE9tLfD8xzQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=YBlTc/Fi8xtCCfnMJyAHQVfrIcyemZBiy7aqATxgWcXRWiQ43KDD6ZTTGMNNmMjh1Bq/Hhb1TTtAn/oINbLLgH68SedlmdSLlk0bc0bszq/IE9RSveZtLTGV5BV9tP3JDt0aLcC4DOAoxipIN2JgtnZ2ztzzpjrNjbhlc1jfO/M= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Q8gfJJhA; arc=fail smtp.client-ip=40.107.92.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Q8gfJJhA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=N2xBvxQ3hVWQ+GEHvUdXrPET78zu0QfXqivAuvCohFJqSDxmjBrwqo7pS4MPV8Y99ixbWfTobkaCgHPw5lIAQYEt8d+Yvgdk1/xkq+TdOa4hGOzpvJRoO5Ei4B2oXCpOBLSgX7iUF5YAP1nH7U+cTmDPF+ZjFwJD60CbxMoX14CWAbL83FGSdkksnzQTZEAtIW3hHPsTaFnn56u7F+s47+Iaqjm9w+AeqdyHg90CUqWx/npQkcVn6lEJIHihPm1b3gdvVMHWuJDNXnFyk0MhmdKMmNkvj+WCxZY/8jeGgiqpSbYNz+rIS/yaRG9U5CeljEKap+YLdyLydNgcHrt01g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BgTEvV7v3q40KTL5MdLmBUliyzoqF5VO/B9uqMhOrRk=; b=oKacMqbMBokix+DktkeQxKy96DHv1QwYTYbmzbLrzrwyXHJjjtwrQPNqOiEMqJQvq73EqiDGTAOMy+FZGTkguNEGJvOGJGs8D2W+ewrY5u0/8N5Vb+davsclrggNlpYjlEStYqq/r0VE3Klg+HahbohkkXyt5kfDlr4MbDxUv599g9V8m1eaGLkXR+7zMWYJgpkxJJ/rS0Rq51XuvCTd56vsE0wjYNZhk7fRg/4PWaD0rFLNzR4piZ80TRcJbwAnUhXFXNU394ZoCcgS8drMoItCwdQ6eaBxKBvx40zybgrAYxVG02lce0UvwmTomnD/mnUSTLoH7/fp8LgJ0TH0sA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgTEvV7v3q40KTL5MdLmBUliyzoqF5VO/B9uqMhOrRk=; b=Q8gfJJhAKdSmjBR7XciDHGYwPx1r8NSeIaOThNlRp65OvIBklqSvSAmKPWssxroSb/Wp5F0hu1CxcE1/daV6MJmy1KoOuYObApYtScv9z3Koc7NczjrSYTR754l2/pVbx6TXx05XIB7HazLXZ5Dg2EzzGHWkN8fH3bkZ8iU8bfVA+dPtgJ8Jua+4UXT7RCJ95lcUTFJDNNgbshGPJyXsNWXOItJz0sUfUi5CNYe7VzYxIDvczlU4IcvgVbGRDGFwdqBPL9fEZFFLmJFb+3kfZfZzPscKGeaSQtf5Tmi2aUmNVpkyTzuE88CtfTH90wF5qXBYli8OiKcaODuGWrDCew== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by SN7PR12MB8001.namprd12.prod.outlook.com (2603:10b6:806:340::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:24 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:24 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 21/23] iommu/pages: Remove iommu_alloc_page_node() Date: Tue, 8 Apr 2025 13:54:09 -0300 Message-ID: <21-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0035.namprd03.prod.outlook.com (2603:10b6:208:32d::10) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|SN7PR12MB8001:EE_ X-MS-Office365-Filtering-Correlation-Id: 9565e244-52e5-40de-b08f-08dd76be019f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|366016|1800799024|921020; X-Microsoft-Antispam-Message-Info: I+b9wUkTUZ8d+qB6Dlcw6IZCFaDLjrcyPfliNXgMaTH9BXfYT6sxADgi8Y/q9QTNOJOyfn2fgCAz00NDL4mTEaeNjHktgZVEPHCliNKNQYYAJnY4jW4YUDwtz4l9oWmkq0RpTrWuyCFTJwCbqptM15HV4qEVbwsGpgnWRlCY7F0rszbO4nlR80Ywst89ddKVVKAZzpHuuL7UU7CK9tyEsrIkpqrFwRJhRYgaNhMXB92SNnEo5TsxqdmKb/Da0UynTm+rDXwyrrWF3tH8a3GGDswOd4xSo3cKvbvpOj648jDeaPLO8mdfzCTj/fNEoTUixu3n84loEJp43tQ7zsH8M+pVqUQ3hJmODrx+qwPZSBybnqeUuD+9IIdVLv9IYbg5ZyB0bZfgJmsGvXnRIz28OJq+X8DmwYlBZXjCZEF6QqaRTdi5bf2RgtIef/DYWMwRkQumL10Dp5ru8oSm0heXEwPfIKsP2JHCV3HFpoO1rccXX2KOmns8YZFRAL0CPGFXQ/7qNvm7vmsLjPWoy+ryctRmAOkUmD2BNXkni+Hm9qDLsT5sNQN+wPmk0GjLDgF0syaQbQRoV7v6C7pbg5vNMmoTlK3N/M5vT/HZbxVa2KotkoY+C5j9E4VHjHqzD0wOctTzghozJihigZDxcQCPUPnFKeErf72bEgBE5XgGJPTgMJTZSlfaQOKXI9Th52mxgTAPuudcR0Yw2eFsCpAdKlkVHsiAqYhZPfR29VNUjVLrnPVoTs5kq46GSKYDLG/20i9j919M5RVWLnU05oZfJev21a0Nsvp1C2uSbBKg3vURdS2SlDQBVV4kivHA1GobAVfMJTCWn/2MuGD/opCfTZ+B1ttjuwv+42lwdqjGR48HXBZxNUzIUxb4JFdm1sqXDSH/flSzuKct1jyFovw9dL3tUyb6ZsgvIIJWy59K1+3KX735Jb44fzcU6HZ/lh3JKVR92nuj17sSlMPPKNiLJaXzYXNSC9tRA0/utq+QysQRe/WYCRCkz3cR9CHjvYdjQ2w49vNr6JpM9wz2o230VAiHdvDq4tmexvM+Dnw0rkouXrT26PFNyLENaSGXFUzUNjRcWhZuhkrNwPCqak4JHT3D2P43uxUJdCWDOgCCH3a73UNOqeIjKcwX5jpuT4j2ZnNfgT+UhI6QYQJgLcrQ0w9hFGGpA+dZRhdyxL7sL4qxtkVBk2ICxnXiQV1snsASesoXAsAB74SDGD3h9hoDvbceriIOBTBfLgUJzl0PcsTgvd3MalelpQSitkCKnhtVh03N4I1KXYCnglJihdCV0S7AecVxxWcsJ86UZeGwWLN/q0BBC9N/woWGz9PtL3Mj6Ra9qkhRGPQcMtO1O0e0tOJ+nEMNU1HCMEmJYJPRrzSN3ir80SlU5vsWGXn3yLYjhTV9K3YQt9KD4Y4huNM1pB8K396SoXqJPW5bOO1secpgW7kSPl8XF17akeT2p+I4Pa7AJKGW5QuFU1SPuFVW+g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(7416014)(366016)(1800799024)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: a3m05McrdF3rZQOUAVegQwKDGHgZd3yBdVsPkWQqM+nb6fPJlSNt9ZIwiwOhyQQrpAVKoui534nzu88MUsAZp8jZJ2aFtNxETNl1az/2xbt37LstbWxScCc6pmfFBVbq9Vmf2GAdsaUa+iCAhYrs2bqOG9v6Ih4S7tET1STD7HQZHoxBPQrvV9ivaS/k2G9j/1aKt8W9h40i9cXdfg2k/3U7nq4Mpx/h0uSHOnVSuYOcM5K8Aum28a7qAz+JseKHoocQ1wzYccpR2F3ci+d/NElM102E1y58P5SQE69fDSJ7q+BTj4A+SBf0PhQjwSpuWbyBfeGmHNvzBgnJCtD5VqJ7Bg/4I0pNav8hyUeIUZgmJqR+/rcR0PyWTAX0L81Iie0iv6cgGw59Y1arr4OLgFcHf3kLmyxguW0i/HpzQty+tfXye29f0OT+cy74dK5+waTYo9DNvQlWKSqXFVSC0XqWm9Mc4DGNek6hQgeOt6VvQgxvJidoGw9Rn4xP6V+QvwVCsR7re/C3Cia/RSSwNoq9+JhYqGAYiIG6bGJKwlIOJL0wLAkdvaKws03tTyYzZX/5fNQICFzFSHRQz1S/YMFim2kT4wOitLL8h0Yf2u+DU3jK2AXL8bFhFgG1uaXK0QvxzQeEI8n+jFo9MkjtAN4kNPZlQS0Wcqu/nMWiNqDg/565U3eoO4QkZRoos1K67CEzj2lvDjZ0aYqHX26zlEu06thhPU8VHRRxp+NWcv6hpaYPF/kiADHKbJVcjy2J4o1Mb7zNpHM273inZT9+98KnU0lbFg3vIEHiRjhfoJ8Hsvg6hk6ZAbnz1p/jJ8zD5RTCX58f3Ltnx4dBH90Nb/PwybzcqPnn+C3rJAna5HPeh8vWQtFF9NH2/IyJx+jQNO03AWkPqSdeNCnPGDEQumUpwZsWxFCg+N9Ytyn6LbtbHKXF1veNeiWkF71Tdwl47ZKovoUlHT0qzWYAlSZInhVO5YCkYnukOa/qNnGI23zYp8R6xk9hQZDWb79ncyi65yYbmShG9aCSIFUZz6+niGKdNr9R5LeQ1uy62tzI117pXfijFp0pfETRd06TMjnut1h6Ut07a10bCXun+QmpZ0oGSP2b2CCOpV9cGT4kpsZL8inMh+0Z1nZ1PVRkBG0z+9mSJ6cUD7KeCu5Xlof3GjB6FzFVCpyG7VSuQcdaYMRQSd/u4wWzJ/FlFETnKDqcKcnV+fdA0zpapZFDA2E0Wjknr38QYIHcXMxfxTAnzkLDBbjmMJK5ie9fepJgLzoZ/2F3E1Za1vOMUOIcAi5JbVXMJj0t/mBk8QKl7RQ5l9JPT9RkylM7ZoP3KLzGuQJGjbZQYPYsF+P/4yU4dKSywzCH7zoHaFFf6dq8Of5WvbO/5j4wTVT000Nwjuk1LQ3Eh5OAED6TJN6IcEJa976v3NRVWCnpTQuntblTMR3UdNdRQUZLMVT25Ff2YRf92yTGpyTxcj60/80zc3K98yU1ZT0GHH8T7sjo1/1OIBNjIKuoPcj8AT7BeAqNUWoOUED9DeET3MzYp3zfnFu28vvgOluvnjfFQLd6DzYez+K8Z7ko+GK8fKG8yY+D/b2ePdSE X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9565e244-52e5-40de-b08f-08dd76be019f X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:20.3136 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aIg8U78JGqKPT/YJfZ4DywU2r1IuWQgGN5MZgqOMYJI+DDjVHLJWTR8mxVdMB4gp X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8001 Use iommu_alloc_pages_node_sz() instead. AMD and Intel are both using 4K pages for these structures since those drivers only work on 4K PAGE_SIZE. riscv is also spec'd to use SZ_4K. Reviewed-by: Lu Baolu Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/amd/io_pgtable.c | 8 +++++--- drivers/iommu/amd/io_pgtable_v2.c | 4 ++-- drivers/iommu/amd/iommu.c | 2 +- drivers/iommu/intel/iommu.c | 13 ++++++++----- drivers/iommu/intel/pasid.c | 3 ++- drivers/iommu/iommu-pages.h | 13 ------------- drivers/iommu/riscv/iommu.c | 7 ++++--- 7 files changed, 22 insertions(+), 28 deletions(-) diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 97ddadbcfb54d6..4d308c07113495 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -114,7 +114,7 @@ static bool increase_address_space(struct amd_io_pgtable *pgtable, bool ret = true; u64 *pte; - pte = iommu_alloc_page_node(cfg->amd.nid, gfp); + pte = iommu_alloc_pages_node_sz(cfg->amd.nid, gfp, SZ_4K); if (!pte) return false; @@ -206,7 +206,8 @@ static u64 *alloc_pte(struct amd_io_pgtable *pgtable, if (!IOMMU_PTE_PRESENT(__pte) || pte_level == PAGE_MODE_NONE) { - page = iommu_alloc_page_node(cfg->amd.nid, gfp); + page = iommu_alloc_pages_node_sz(cfg->amd.nid, gfp, + SZ_4K); if (!page) return NULL; @@ -535,7 +536,8 @@ static struct io_pgtable *v1_alloc_pgtable(struct io_pgtable_cfg *cfg, void *coo { struct amd_io_pgtable *pgtable = io_pgtable_cfg_to_data(cfg); - pgtable->root = iommu_alloc_page_node(cfg->amd.nid, GFP_KERNEL); + pgtable->root = + iommu_alloc_pages_node_sz(cfg->amd.nid, GFP_KERNEL, SZ_4K); if (!pgtable->root) return NULL; pgtable->mode = PAGE_MODE_3_LEVEL; diff --git a/drivers/iommu/amd/io_pgtable_v2.c b/drivers/iommu/amd/io_pgtable_v2.c index f5f2f53273786d..b47941353ccbb8 100644 --- a/drivers/iommu/amd/io_pgtable_v2.c +++ b/drivers/iommu/amd/io_pgtable_v2.c @@ -152,7 +152,7 @@ static u64 *v2_alloc_pte(int nid, u64 *pgd, unsigned long iova, } if (!IOMMU_PTE_PRESENT(__pte)) { - page = iommu_alloc_page_node(nid, gfp); + page = iommu_alloc_pages_node_sz(nid, gfp, SZ_4K); if (!page) return NULL; @@ -346,7 +346,7 @@ static struct io_pgtable *v2_alloc_pgtable(struct io_pgtable_cfg *cfg, void *coo struct amd_io_pgtable *pgtable = io_pgtable_cfg_to_data(cfg); int ias = IOMMU_IN_ADDR_BIT_SIZE; - pgtable->pgd = iommu_alloc_page_node(cfg->amd.nid, GFP_KERNEL); + pgtable->pgd = iommu_alloc_pages_node_sz(cfg->amd.nid, GFP_KERNEL, SZ_4K); if (!pgtable->pgd) return NULL; diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index fe31166ced94b7..ed04d1471c2428 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1884,7 +1884,7 @@ static int setup_gcr3_table(struct gcr3_tbl_info *gcr3_info, return -ENOSPC; gcr3_info->domid = domid; - gcr3_info->gcr3_tbl = iommu_alloc_page_node(nid, GFP_ATOMIC); + gcr3_info->gcr3_tbl = iommu_alloc_pages_node_sz(nid, GFP_ATOMIC, SZ_4K); if (gcr3_info->gcr3_tbl == NULL) { pdom_id_free(domid); return -ENOMEM; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 4ca74a2b1a1109..7842e437fb170e 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -397,7 +397,8 @@ struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus, if (!alloc) return NULL; - context = iommu_alloc_page_node(iommu->node, GFP_ATOMIC); + context = iommu_alloc_pages_node_sz(iommu->node, GFP_ATOMIC, + SZ_4K); if (!context) return NULL; @@ -731,7 +732,8 @@ static struct dma_pte *pfn_to_dma_pte(struct dmar_domain *domain, if (!dma_pte_present(pte)) { uint64_t pteval, tmp; - tmp_page = iommu_alloc_page_node(domain->nid, gfp); + tmp_page = iommu_alloc_pages_node_sz(domain->nid, gfp, + SZ_4K); if (!tmp_page) return NULL; @@ -982,7 +984,7 @@ static int iommu_alloc_root_entry(struct intel_iommu *iommu) { struct root_entry *root; - root = iommu_alloc_page_node(iommu->node, GFP_ATOMIC); + root = iommu_alloc_pages_node_sz(iommu->node, GFP_ATOMIC, SZ_4K); if (!root) { pr_err("Allocating root entry for %s failed\n", iommu->name); @@ -2026,7 +2028,8 @@ static int copy_context_table(struct intel_iommu *iommu, if (!old_ce) goto out; - new_ce = iommu_alloc_page_node(iommu->node, GFP_KERNEL); + new_ce = iommu_alloc_pages_node_sz(iommu->node, + GFP_KERNEL, SZ_4K); if (!new_ce) goto out_unmap; @@ -3359,7 +3362,7 @@ static struct dmar_domain *paging_domain_alloc(struct device *dev, bool first_st domain->domain.geometry.aperture_end = __DOMAIN_MAX_ADDR(domain->gaw); /* always allocate the top pgd */ - domain->pgd = iommu_alloc_page_node(domain->nid, GFP_KERNEL); + domain->pgd = iommu_alloc_pages_node_sz(domain->nid, GFP_KERNEL, SZ_4K); if (!domain->pgd) { kfree(domain); return ERR_PTR(-ENOMEM); diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index b616aaad03ba24..728da85a910068 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -147,7 +147,8 @@ static struct pasid_entry *intel_pasid_get_entry(struct device *dev, u32 pasid) if (!entries) { u64 tmp; - entries = iommu_alloc_page_node(info->iommu->node, GFP_ATOMIC); + entries = iommu_alloc_pages_node_sz(info->iommu->node, + GFP_ATOMIC, SZ_4K); if (!entries) return NULL; diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index 4513fbc76260cd..7ece83bb0f54bb 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -114,17 +114,4 @@ static inline void *iommu_alloc_pages_sz(gfp_t gfp, size_t size) return iommu_alloc_pages_node_sz(NUMA_NO_NODE, gfp, size); } -/** - * iommu_alloc_page_node - allocate a zeroed page at specific NUMA node. - * @nid: memory NUMA node id - * @gfp: buddy allocator flags - * - * returns the virtual address of the allocated page - * Prefer to use iommu_alloc_pages_node_lg2() - */ -static inline void *iommu_alloc_page_node(int nid, gfp_t gfp) -{ - return iommu_alloc_pages_node_sz(nid, gfp, PAGE_SIZE); -} - #endif /* __IOMMU_PAGES_H */ diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 8835c82f118db4..bb57092ca90110 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1144,7 +1144,8 @@ static unsigned long *riscv_iommu_pte_alloc(struct riscv_iommu_domain *domain, * page table. This might race with other mappings, retry. */ if (_io_pte_none(pte)) { - addr = iommu_alloc_page_node(domain->numa_node, gfp); + addr = iommu_alloc_pages_node_sz(domain->numa_node, gfp, + SZ_4K); if (!addr) return NULL; old = pte; @@ -1385,8 +1386,8 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev) domain->numa_node = dev_to_node(iommu->dev); domain->amo_enabled = !!(iommu->caps & RISCV_IOMMU_CAPABILITIES_AMO_HWAD); domain->pgd_mode = pgd_mode; - domain->pgd_root = iommu_alloc_page_node(domain->numa_node, - GFP_KERNEL_ACCOUNT); + domain->pgd_root = iommu_alloc_pages_node_sz(domain->numa_node, + GFP_KERNEL_ACCOUNT, SZ_4K); if (!domain->pgd_root) { kfree(domain); return ERR_PTR(-ENOMEM);