From patchwork Tue Oct 2 11:05:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 147965 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp5007291lji; Tue, 2 Oct 2018 04:05:52 -0700 (PDT) X-Google-Smtp-Source: ACcGV63+ng7xMVlh7ANsJVNs7fPslXMVnvc4nKCub1XD3TL2RGR7gqnohrF77ipUa5DizMMCnTG9 X-Received: by 2002:aa7:80cd:: with SMTP id a13-v6mr8644426pfn.86.1538478352220; Tue, 02 Oct 2018 04:05:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538478352; cv=none; d=google.com; s=arc-20160816; b=u47YSfF6qjNQonN/tnC8PU94ZYv55bThCflTwc3pyYeuNltuRi9HR78Rm1I551SWVd ruQx0DckHyTzomhNguncpXyVJuMKDhOxvLnF7y8DCbvyyZXguWS52Orgb7JW/Z7kRd9j YyssCrkWN32Ls6FvCrNYhJ/NoAMGA64YvrZ8CZXABr3X2sZq2LP4sJvW9YT2xKU+lGXG qGg/zK1uoG17e+Nego8x4CerpvnBxIAHu9aEx+QPFHYj5xrTHBlroJQ+bzjO2FDZqdr4 ksjntOOmKt22N1Bj/2kCwfgh2hINcVkz7QS09QJG5za21Ntt9Fz3sUWGXZ1UVpwDSLjc IJag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Ka6oXZu8DJRXSCZkb2TermywvttDv1cohIKM9nr4o2s=; b=oPrHPyaPzbluu3m/gyBXfSDLFijcYpI+0swb0lLUUePIALXdLcOItqb+MQQH3qyi+W py+a3Eg5Ov8ieipcTFbdoSv3aPJsyR8aOMrOLa7uCHniJRuY3JibUabz75Ji+QDb/WJS ctP0koyUKMLQG0Ym2MO3CQVxEfeit63/0/ZIL6uN8+NRxO5DUMocQfvLXV2PCQOa9kut NL0zeWvQF7/1f8TPvDUuiiBlPgAwbxwrMYN1Rdo9W7RomB/1eyT1pKpuaPF2NasNKDK+ efkmVaTQLN78a0ZjVe1daRT9SBbLOJ0kwc5mAXqb7u/riNijQm+E+ipbJ8qiQ7sTdXSn 6awg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m7-v6si2909674pfi.286.2018.10.02.04.05.51; Tue, 02 Oct 2018 04:05:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727634AbeJBRsY (ORCPT + 32 others); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:34674 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727519AbeJBRsY (ORCPT ); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 890CD15B2; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 59E8E3F774; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 4AEB81AE3D81; Tue, 2 Oct 2018 12:06:04 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v2 1/5] ioremap: Rework pXd_free_pYd_page() API Date: Tue, 2 Oct 2018 12:05:59 +0100 Message-Id: <1538478363-16255-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1538478363-16255-1-git-send-email-will.deacon@arm.com> References: <1538478363-16255-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The recently merged API for ensuring break-before-make on page-table entries when installing huge mappings in the vmalloc/ioremap region is fairly counter-intuitive, resulting in the arch freeing functions (e.g. pmd_free_pte_page()) being called even on entries that aren't present. This resulted in a minor bug in the arm64 implementation, giving rise to spurious VM_WARN messages. This patch moves the pXd_present() checks out into the core code, refactoring the callsites at the same time so that we avoid the complex conjunctions when determining whether or not we can put down a huge mapping. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Suggested-by: Linus Torvalds Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- lib/ioremap.c | 56 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 517f5853ffed..6c72764af19c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -76,6 +76,25 @@ static int ioremap_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pmd_enabled()) + return 0; + + if ((end - addr) != PMD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PMD_SIZE)) + return 0; + + if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + return 0; + + return pmd_set_huge(pmd, phys_addr, prot); +} + static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -89,13 +108,8 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, do { next = pmd_addr_end(addr, end); - if (ioremap_pmd_enabled() && - ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd, addr)) { - if (pmd_set_huge(pmd, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) return -ENOMEM; @@ -103,6 +117,25 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, return 0; } +static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pud_enabled()) + return 0; + + if ((end - addr) != PUD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PUD_SIZE)) + return 0; + + if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + return 0; + + return pud_set_huge(pud, phys_addr, prot); +} + static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -116,13 +149,8 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, do { next = pud_addr_end(addr, end); - if (ioremap_pud_enabled() && - ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud, addr)) { - if (pud_set_huge(pud, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) return -ENOMEM; From patchwork Tue Oct 2 11:06:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 147963 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp5007104lji; Tue, 2 Oct 2018 04:05:42 -0700 (PDT) X-Google-Smtp-Source: ACcGV62VQdNzFEVxA6pkfNEYE54QYzEZpbWdFZz5a56p8D9v8cU7IcruGZph4ANLL+o9MA98+5g2 X-Received: by 2002:a63:e749:: with SMTP id j9-v6mr8247656pgk.246.1538478342734; Tue, 02 Oct 2018 04:05:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538478342; cv=none; d=google.com; s=arc-20160816; b=eAnM3ajoR5OB7xQ35AMyKIDxsA0WZDAGoaFwDz5Amd6zBRo1YXN1KBQgsjDk/Q522E 7obFPvzIxwV7c05ZkVr4KGhsa/cr13fXJ2DTm+gI3+4EZitqkCjx+T0olww8GktYvt1b SMZySPRCsWOs8Uh/Xx0IrhNCmzjZP684lBzANRAe5hZYrl1sy7b4Fsja+E9R3RcDS1Pa ssYqMUrQNT845Y4GHoqJhvzDM/vjXtZpiVxU6A11yDyobKjw+8PsDGQ28+WbTmg2FEBX 47Ui/P6aNwvyQkWURK3zOHtU0HrCe/RGgRsDBD73yaZ4hcRS/02PrhtbvXxrDKGyJ4I2 QfkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=5Z6J2RQTqMWhpJ0pzFttI0SwIzpZ6lRPEY1BeHTRt8o=; b=Wgea0N0LLGhIlBNI0OWMNasTGki6SFVsta19E0Lzn83M2UQme+UcJiEqYMjSm672c/ BKkKXbEdem0vAsrYxdlA/Z20RS7Llm9zrXY0WchzGK4J2MFKU3mxEnLjmZ4SXycfekER y3fNZbq+KBBKWem+wz4FE3FzEIw1sGpZJpLTFsmjUR0IJzajTiBoXbLlo9c2MT7cHSaA poGyvNNyQRAHh8h/GoDroWngzZXD9sF9YfGLmdeNASy9pzXysB8rKgw0Jf7WuHSVOLda lR7njcSqX1R2OmqWb9O2+xsQSYP59UpgTDY/a8w0IBOZ8aSsT8qQLVfLrhdrgI4+57HM tJ+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m125-v6si15192738pgm.541.2018.10.02.04.05.42; Tue, 02 Oct 2018 04:05:42 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727661AbeJBRsY (ORCPT + 32 others); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:34686 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727546AbeJBRsY (ORCPT ); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9ABFF15BF; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6BBD53F7F4; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 5D9771AE3E55; Tue, 2 Oct 2018 12:06:04 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v2 2/5] arm64: mmu: Drop pXd_present() checks from pXd_free_pYd_table() Date: Tue, 2 Oct 2018 12:06:00 +0100 Message-Id: <1538478363-16255-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1538478363-16255-1-git-send-email-will.deacon@arm.com> References: <1538478363-16255-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the core code checks this for us, we don't need to do it in the backend. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8080c9f489c3..0dcb3354d6dd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -985,10 +985,8 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - if (!pmd_present(pmd)) - return 1; if (!pmd_table(pmd)) { - VM_WARN_ON(!pmd_table(pmd)); + VM_WARN_ON(1); return 1; } @@ -1008,10 +1006,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - if (!pud_present(pud)) - return 1; if (!pud_table(pud)) { - VM_WARN_ON(!pud_table(pud)); + VM_WARN_ON(1); return 1; } From patchwork Tue Oct 2 11:06:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 147968 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp5007546lji; Tue, 2 Oct 2018 04:06:05 -0700 (PDT) X-Google-Smtp-Source: ACcGV63XJj5dd5M7GzIgrq1c2MTMTe3Tu6kXcVz0Q9cEzvORA2NW41dcpSLvWz502uNOYQ/Tl5fU X-Received: by 2002:a62:5882:: with SMTP id m124-v6mr15795683pfb.249.1538478365615; Tue, 02 Oct 2018 04:06:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538478365; cv=none; d=google.com; s=arc-20160816; b=L6qJZqTAeUdvTZ58nRypj09anLwxFlbosCKq9tIEGcTV6V5jfjTji4VIIL0V9p4H2m FR7nXAJ0CxY1vdQnfc36n3u4VCSh3oF4dX3du6o8HLP1T0i+0tZgKNFFjlxqyCNR6gik T09/9IXBxCMtDVmi5heF63yV0wg2GLGSTzb2lVqWBf/Qp1PluvSrWQpz2khWyWNFsRyM Qye0kyMVw6yewxNegxzYvg8yMxHBIu93O0VetgLfw9GhbtGZjX89UiaVktUU4YBapBRN fk4erU8wiUhSUnXhmi0Ciw7U3NwWkuZOId/9CiRDprBrmYUNeHJ9R2evA6cgb0VtIje8 0WNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=qqOjGbQQBgw+VVbEnWZUHYnvbPx0vTCUDG5+OY61Bac=; b=e6gFyjF+XXjiz381IiZcd83eRbuOrMwB1P0izGLnKDwLXCJnCrG63WDj3irFpo1kbC m+nis4YpAl4Qnf9FVWB62CSTGhFvsuIhI5nQPGzc+BnLnlgV8iCFpiymlLg6/tIjpYw+ y7g5cQPnzIDjiNRcT69Blb5ebSAaNap6xdNxSZgY8vOCWyd2jpW6s6gGF4SC0/9fwKvG tIxMvtA8L2iUp0AbZ8cdF1eMBvItKeIYwin7mUKlRI7yxsZ13k3GS61DJyo/dFq+vhgh Y825j3yAtv1kn6J5PlK420sXFt6pp5E+jvC16/k20UIpQDfw2tUonjM3jQMWw1yKrxYm o/Eg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 5-v6si16064721pfz.160.2018.10.02.04.06.05; Tue, 02 Oct 2018 04:06:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727734AbeJBRsq (ORCPT + 32 others); Tue, 2 Oct 2018 13:48:46 -0400 Received: from foss.arm.com ([217.140.101.70]:34720 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727573AbeJBRsY (ORCPT ); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AA5EB1682; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7B40A3F5B7; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 6DDCF1AE3E59; Tue, 2 Oct 2018 12:06:04 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v2 3/5] x86: pgtable: Drop pXd_none() checks from pXd_free_pYd_table() Date: Tue, 2 Oct 2018 12:06:01 +0100 Message-Id: <1538478363-16255-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1538478363-16255-1-git-send-email-will.deacon@arm.com> References: <1538478363-16255-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the core code checks this for us, we don't need to do it in the backend. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/x86/mm/pgtable.c | 6 ------ 1 file changed, 6 deletions(-) -- 2.1.4 diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index ae394552fb94..b4919c44a194 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -796,9 +796,6 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) pte_t *pte; int i; - if (pud_none(*pud)) - return 1; - pmd = (pmd_t *)pud_page_vaddr(*pud); pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); if (!pmd_sv) @@ -840,9 +837,6 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; - if (pmd_none(*pmd)) - return 1; - pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); From patchwork Tue Oct 2 11:06:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 147967 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp5007397lji; Tue, 2 Oct 2018 04:05:58 -0700 (PDT) X-Google-Smtp-Source: ACcGV63HvVosGh644EBMHjH5FUUOwF7zq5ZhbF5+OtLlSnm8w6I9OMBIGBPLR4vD7A5/8ZVkDOkh X-Received: by 2002:a63:5605:: with SMTP id k5-v6mr14283151pgb.189.1538478357983; Tue, 02 Oct 2018 04:05:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538478357; cv=none; d=google.com; s=arc-20160816; b=NoB96xItl858lYh56N5pDzqDvJTFVi3qsQKz06I0+V+YE/rpzM5S2NluFM6FS3gF6v tnro8vAnJO5BU/VsrMO0h6o9WEMmPXJB335UznXf0sg1vOpIj+LTtln+oh6XcDuu7erY O8MyrLrfjvSymOXBL26bb0PiY6+sQ7Oy8MtWIFXVEguNmgw4oIDuSimEMSCpiakiZ5EF U23BiOT5m+KAT37rULz2/j208clK1B48vjzzUF6z/eL3WadIHwHhPvKPaPJRdHMdxqRg 3vUPkQzns/UvoKIF5iplHM0eWvX/F7UoNqUeol6Okwr/lwLGMyR1c09KFH9M062xGRKq swUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=gyaouhO6M09bNRRp3jEmM8H6PKnPVZFp5tgRSOGTTCE=; b=JRhwFHABL//1Xtj0u1yVOFwhxFVKviy6QXDul9XNDc0Dj639drl/d21SqEUx2mmjXy KjVCuk58S2jCako2yUi0vpszXO9mRkFVKqzDu12h/zr0uC4q30eMa3FC2Q1PmC01Su7m 7L46Sxn4d/x4CUb1MTJuKnvJ6jvAM+XmcDKFvXkbZfiA5oUo9zQM5jy2IbZE3Y4MGwpG 3FMO5fn2bTGs9vnH2xR++2KMkWK++UVV7ssE07IQ2DS23FmKaCTwHps81ZNPHDgBkLli eLUN5sdyIXlcH2FihpHzX0VBoGTGY4lpfhRyzeHDMPpSyKW/NK0c8HpamUvGMCBMrBoM 2nug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n15-v6si6144718pgk.103.2018.10.02.04.05.57; Tue, 02 Oct 2018 04:05:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727711AbeJBRsj (ORCPT + 32 others); Tue, 2 Oct 2018 13:48:39 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:34736 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727576AbeJBRsY (ORCPT ); Tue, 2 Oct 2018 13:48:24 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C2DCA7A9; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 93D573F774; Tue, 2 Oct 2018 04:05:40 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 7E2771AE402D; Tue, 2 Oct 2018 12:06:04 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v2 4/5] lib/ioremap: Ensure phys_addr actually corresponds to a physical address Date: Tue, 2 Oct 2018 12:06:02 +0100 Message-Id: <1538478363-16255-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1538478363-16255-1-git-send-email-will.deacon@arm.com> References: <1538478363-16255-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current ioremap() code uses a phys_addr variable at each level of page table, which is confusingly offset by subtracting the base virtual address being mapped so that adding the current virtual address back on when iterating through the page table entries gives back the corresponding physical address. This is fairly confusing and results in all users of phys_addr having to add the current virtual address back on. Instead, this patch just updates phys_addr when iterating over the page table entries, ensuring that it's always up-to-date and doesn't require explicit offsetting. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Cc: Sean Christopherson Signed-off-by: Will Deacon --- lib/ioremap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 6c72764af19c..10d7c5485c39 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -101,19 +101,18 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, pmd_t *pmd; unsigned long next; - phys_addr -= addr; pmd = pmd_alloc(&init_mm, pud, addr); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) continue; - if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_pte_range(pmd, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pmd++, addr = next, addr != end); + } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -142,19 +141,18 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, pud_t *pud; unsigned long next; - phys_addr -= addr; pud = pud_alloc(&init_mm, p4d, addr); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pud(pud, addr, next, phys_addr, prot)) continue; - if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_pmd_range(pud, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pud++, addr = next, addr != end); + } while (pud++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -164,7 +162,6 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, p4d_t *p4d; unsigned long next; - phys_addr -= addr; p4d = p4d_alloc(&init_mm, pgd, addr); if (!p4d) return -ENOMEM; @@ -173,14 +170,14 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, if (ioremap_p4d_enabled() && ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr + addr, prot)) + IS_ALIGNED(phys_addr, P4D_SIZE)) { + if (p4d_set_huge(p4d, phys_addr, prot)) continue; } - if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) + if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM; - } while (p4d++, addr = next, addr != end); + } while (p4d++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -196,14 +193,13 @@ int ioremap_page_range(unsigned long addr, BUG_ON(addr >= end); start = addr; - phys_addr -= addr; pgd = pgd_offset_k(addr); do { next = pgd_addr_end(addr, end); - err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); + err = ioremap_p4d_range(pgd, addr, next, phys_addr, prot); if (err) break; - } while (pgd++, addr = next, addr != end); + } while (pgd++, phys_addr += (next - addr), addr = next, addr != end); flush_cache_vmap(start, end); From patchwork Tue Oct 2 11:06:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 147964 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp5007170lji; Tue, 2 Oct 2018 04:05:45 -0700 (PDT) X-Google-Smtp-Source: ACcGV63bfyzyiQKCHVV1ci6CIRYJcLV5YbmE5+KulrY1hMeMJHZ/mSIkjh/kK4Y2uT0bu4KTw/kN X-Received: by 2002:a17:902:758f:: with SMTP id j15-v6mr16354968pll.160.1538478345409; Tue, 02 Oct 2018 04:05:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538478345; cv=none; d=google.com; s=arc-20160816; b=CnogkBngIdc2YwqVZm/7tSVvH77PTfPNG2VHulurI66I/MGpDv/D9/gTzv6+oy607N vszp6hmzf5QyL3AY0pSmIsXHar+BH9V2mWdK8LyXqdlyrUg6vgyHUMEf5zYrU41TTDtd q6qc3w6NE47MXsEHiWX2601DQxmRK6H14Xq87EudchIjcRtCAoClkemByN6aE2kd8Afx fGzASLIH2LCtJjUFFfr9z0BXYS8Eq5NDIzvdFszDilX8tjOT/0OHM2+er7l/z5bpMsyS w+Wrbmfj2ydFvjjYrLHbCEi3uaG9x3GQktHBXVsEa6H0ou/3vU2KenYxHNxyMF4AwTR3 2Rng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=AUz4K2RTcj/O0JOlaWId1Il0sNteqiutU7pUtx2yB/c=; b=ZgdzNYI4fvV53wHNUdefIjFhGcDzeuKYLrnM0NFB2uSWG8gC5cKWZQQEd1o18PV9xq NXjiEahGvlKujQCxFs+mQyeYkyb1kNdPgJ7M2z/E48luft3/UHdXaywaEORfUTfqghB3 krr5ZgEaK3Qy6exQPm8OgL+VYOhxzLiq+gmGN/EhoTeGeV3mR8w0ysS2Z6aaD7+0etRD 1MDBznUJR044Ioj1yMYTU2WoyF3mLSzq+7AeLeSTSM6sCMPZsQklJ5vNLeeuF41Z6sGi 4RDeTBNPARtfYvjn2398yk/5j/xgttX7+xCQYPp/Z7ZgMRYFGMzT9JLKr6q6+hmtU4Ib Y6hw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m7-v6si2909674pfi.286.2018.10.02.04.05.45; Tue, 02 Oct 2018 04:05:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727691AbeJBRs0 (ORCPT + 32 others); Tue, 2 Oct 2018 13:48:26 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:34750 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727546AbeJBRsZ (ORCPT ); Tue, 2 Oct 2018 13:48:25 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6ADCD1684; Tue, 2 Oct 2018 04:05:41 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3BADA3F5B7; Tue, 2 Oct 2018 04:05:41 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 9696E1AE4464; Tue, 2 Oct 2018 12:06:04 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v2 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Tue, 2 Oct 2018 12:06:03 +0100 Message-Id: <1538478363-16255-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1538478363-16255-1-git-send-email-will.deacon@arm.com> References: <1538478363-16255-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0dcb3354d6dd..58776b90dd2a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1024,3 +1024,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index b4919c44a194..c6094997d060 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -779,6 +779,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 88ebc6102c7c..4297a2519ebf 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index 10d7c5485c39..063213685563 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;