From patchwork Mon Nov 26 17:07:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 152053 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp91933ljp; Mon, 26 Nov 2018 09:08:12 -0800 (PST) X-Google-Smtp-Source: AFSGD/W1tzxVgmq9gOyzV2WZsRx5qmjBfr4I43TeRgRdkToTzRXTdW0086X/Eqn9DtJCOVwD9/Yu X-Received: by 2002:a63:a611:: with SMTP id t17mr25443278pge.338.1543252092210; Mon, 26 Nov 2018 09:08:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543252092; cv=none; d=google.com; s=arc-20160816; b=oWS2XJ8D2sk+1JDLS0gl0LJh9Fy9CDytixhDMaBba0YK/rMOQo6fMWUKXAigpch1WQ 7hiOo4klIg21ABgM62uYpHFFra0RiKeQ4waRDu1k51n+V2ztlykNPVSNErS857hI6c1R AEhflUKZQbmaOpTFnWhy+WDu+MeKa7J2PlJOqYb7gqeVkaxYD4vdqCCT96exRXWRDVb1 GyO87XzBTeX/2jh6VKKvCDzaWKh+D8SH4Febv1FWvaGwu28p7xWC1xXevpunVAUK6WYe BncrdwYtTB7Z7xbHp/RkX+rMTmLT4QUQNCV2zvcTJWm8LfpD5DwMtAfjYGZXoHvsezht meHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Ka6oXZu8DJRXSCZkb2TermywvttDv1cohIKM9nr4o2s=; b=UEJ1VZrABCaEM9vKBSRHpurpjuackGwO9OMzpw4DNElqsolI9VQlVE0orsgFbTJ5eH JIH8suiPzFMN/Aio20wl9cAWunA+TDNpuf9lOVq9mn6dZmNeSCE7MnyRciJ+yRLoFjx0 pVnYIAl3SjHl4kcm8i1Xx6F0c0YyyW5qrhqcceSs3uNU3rlHO22Iqbl+zzVZXh2Fc0yp C0Z8cED4AxKrQ4YL/wy4tDtfGyYslePFidBpUs9gu0mz8F8dva8FjokaNE6I7SLadVmV elYPiShNgaS8swqqsJoGkLpNcJ7twSZPhtkwYEix3UAkzvqOvpynmihd3MegshDW+Evm XJ0A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y25si809125pgl.226.2018.11.26.09.08.00; Mon, 26 Nov 2018 09:08:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726895AbeK0EC2 (ORCPT + 32 others); Mon, 26 Nov 2018 23:02:28 -0500 Received: from foss.arm.com ([217.140.101.70]:42918 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726253AbeK0ECO (ORCPT ); Mon, 26 Nov 2018 23:02:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA76115AD; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9A9353F71D; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 178121AE0776; Mon, 26 Nov 2018 17:07:48 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v4 1/5] ioremap: Rework pXd_free_pYd_page() API Date: Mon, 26 Nov 2018 17:07:43 +0000 Message-Id: <1543252067-30831-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1543252067-30831-1-git-send-email-will.deacon@arm.com> References: <1543252067-30831-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The recently merged API for ensuring break-before-make on page-table entries when installing huge mappings in the vmalloc/ioremap region is fairly counter-intuitive, resulting in the arch freeing functions (e.g. pmd_free_pte_page()) being called even on entries that aren't present. This resulted in a minor bug in the arm64 implementation, giving rise to spurious VM_WARN messages. This patch moves the pXd_present() checks out into the core code, refactoring the callsites at the same time so that we avoid the complex conjunctions when determining whether or not we can put down a huge mapping. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Suggested-by: Linus Torvalds Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- lib/ioremap.c | 56 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 517f5853ffed..6c72764af19c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -76,6 +76,25 @@ static int ioremap_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pmd_enabled()) + return 0; + + if ((end - addr) != PMD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PMD_SIZE)) + return 0; + + if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + return 0; + + return pmd_set_huge(pmd, phys_addr, prot); +} + static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -89,13 +108,8 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, do { next = pmd_addr_end(addr, end); - if (ioremap_pmd_enabled() && - ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd, addr)) { - if (pmd_set_huge(pmd, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) return -ENOMEM; @@ -103,6 +117,25 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, return 0; } +static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pud_enabled()) + return 0; + + if ((end - addr) != PUD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PUD_SIZE)) + return 0; + + if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + return 0; + + return pud_set_huge(pud, phys_addr, prot); +} + static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -116,13 +149,8 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, do { next = pud_addr_end(addr, end); - if (ioremap_pud_enabled() && - ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud, addr)) { - if (pud_set_huge(pud, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) return -ENOMEM; From patchwork Mon Nov 26 17:07:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 152050 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp91353ljp; Mon, 26 Nov 2018 09:07:46 -0800 (PST) X-Google-Smtp-Source: AFSGD/VUDB2AoOXR/NW5J13tph4g9CV5DiPwholsXnbtjO+mRo3TYyp1nrT4nHjKXj7kuJH+lWg9 X-Received: by 2002:a17:902:8ec9:: with SMTP id x9mr29144155plo.27.1543252065864; Mon, 26 Nov 2018 09:07:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543252065; cv=none; d=google.com; s=arc-20160816; b=V5J+4Ga5gHJwYEPoE/uGw30tM3f1CJrwOQ6Cxmstbr66AV/2eISBhUUWvxVx2WCqoN imS/HhDIvFUgly8kT6+0CXWhNuEwA74R3h4QtZ6RUTUcoBbXnw7hp/ZoDNPZwoFIxUbA 5k3cC67n/aAqQd8/dm4LjkL9bSlAERL+9Lg33XI2V0EBrU1L+vkwP18MpndnSmNuswd5 IVFKpbTNozYX/Lm8d4e8X7p6geG4k6Cgu1NlyVw8yUNnM7gbVL1sf0olTLTJ0WL6ySmQ uwyrms5+taemQO9ZghYZtwsDV8K2GXjxmS+cICsJebEcKzeNUr/4CUaiWoO2gi/vqoL0 YMpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=/V6Ii5Ay/gyCu4FDeRHUvTmr/uoACgypNN/miYDcTTU=; b=HhMnlruln440CsUwa1UAAzS5vMD+aM+USEQDoldxUTumkNr3gD6FLKimf6CJzyiARR +ey5gxVmPsiT2PyUXoVrUmLt0Keb1fm8+XsYqfgHLqdYJDAQ/aFnC2kc/GZZFSG/wwpT aTSns5I7JR4iE+Z3yphmUtNdqa4Qs58HneqKPztr8SWmpea9G+NQ91zWVpGbOaiRGABO BF2H9jSR7w3Hg4sq5moFiwvkfLdY2UJeCLiElDJZ9AtJdGeGvTT88msiedBIqBWnZjcf KnCewpl7KOc4qBomKiUiK/XvuRd3EkfRWLPQqHmkFAybQNIrsiwPZxjQh2mz0KyREIyu Hpyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y25si809125pgl.226.2018.11.26.09.07.33; Mon, 26 Nov 2018 09:07:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726588AbeK0ECO (ORCPT + 32 others); Mon, 26 Nov 2018 23:02:14 -0500 Received: from foss.arm.com ([217.140.101.70]:42926 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726255AbeK0ECN (ORCPT ); Mon, 26 Nov 2018 23:02:13 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D839E168F; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A8CC93F906; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 271771AE088D; Mon, 26 Nov 2018 17:07:48 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v4 2/5] arm64: mmu: Drop pXd_present() checks from pXd_free_pYd_table() Date: Mon, 26 Nov 2018 17:07:44 +0000 Message-Id: <1543252067-30831-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1543252067-30831-1-git-send-email-will.deacon@arm.com> References: <1543252067-30831-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d1d6601b385d..786cfa6355be 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -989,10 +989,8 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - if (!pmd_present(pmd)) - return 1; if (!pmd_table(pmd)) { - VM_WARN_ON(!pmd_table(pmd)); + VM_WARN_ON(1); return 1; } @@ -1012,10 +1010,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - if (!pud_present(pud)) - return 1; if (!pud_table(pud)) { - VM_WARN_ON(!pud_table(pud)); + VM_WARN_ON(1); return 1; } From patchwork Mon Nov 26 17:07:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 152052 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp91669ljp; Mon, 26 Nov 2018 09:08:00 -0800 (PST) X-Google-Smtp-Source: AFSGD/XFBZmx/9eWXhpJWUimwgJqMfoDh6eMlfLUHp3On7fgL0m9h9hQIdHYWTrDwSPjsKRqdQw/ X-Received: by 2002:a17:902:e085:: with SMTP id cb5mr28355372plb.24.1543252080204; Mon, 26 Nov 2018 09:08:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543252080; cv=none; d=google.com; s=arc-20160816; b=pO1buyBqWfpSCpk2nle/SzzspNEW6F9+RqLVP+OBVC1QOaMghP75uZz/ESbiloBvHm bkxAbz3yKkYsScoD2IZTewJDtnfsTt/GY34853J1/r1dM1cjRsnW0RVHL8jgNMGjSjNn ABsDmwiDThH/SIkeT+HHjOSXk5F60Dr75CqKmJnjCnfBQl5YWo8iPKFx4BACkj+hT7z2 h+rtmy/tLIl2UJN+dPAKfkyv4D/WkbwhVoURLXKXw5t/hIg8TwTzljC70opvv3J/GI+D iUnXhg0d08YNAkQhlsLxbW99xxnmFpLtMiarUger2g0sbBdvK2zF/uJdgrxJPWlrBuk6 trxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=S3Pe7jCrFGE7JxF6m5RucqgSGzB7M8lCaWDcweULJ1Y=; b=yEIQMap+LewgYiM5086pVCjwXcjwQWT1OgmwqLIT7zLO7D8ESm6IiBT4MWwbJPmTIG Xps1Se5x7wsuRfjNMGKnxt3tEb/JJ8OkPYusAhODe0eHdr5UWzMGai7OAQQrtCI/F+AW HSS7u98UpAcvmemiSu+HEtKoKEf3MJqB1lr4iB0Eo3bZC/E602ToDCI5t97EJVXY7SBU mDE2vOKOVXOhsNkqwZ7nbbv1DD5g2cNn/4wwc9EfyiG6Ft24lcUlpexFVv5/P2u2MCXz 2bUk+JOAjwa7rTVge7HyP1sEmd+0WAQD1qyScW07JTQfMAm0WUgh6a+12fezQut4qQpY IPww== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y25si809125pgl.226.2018.11.26.09.07.46; Mon, 26 Nov 2018 09:08:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726418AbeK0ECO (ORCPT + 32 others); Mon, 26 Nov 2018 23:02:14 -0500 Received: from foss.arm.com ([217.140.101.70]:42942 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726260AbeK0ECN (ORCPT ); Mon, 26 Nov 2018 23:02:13 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EAB75169E; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BA90A3F911; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 3555C1AE0B58; Mon, 26 Nov 2018 17:07:48 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v4 3/5] x86/pgtable: Drop pXd_none() checks from pXd_free_pYd_table() Date: Mon, 26 Nov 2018 17:07:45 +0000 Message-Id: <1543252067-30831-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1543252067-30831-1-git-send-email-will.deacon@arm.com> References: <1543252067-30831-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Michal Hocko Cc: Andrew Morton Acked-by: Thomas Gleixner Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/x86/mm/pgtable.c | 6 ------ 1 file changed, 6 deletions(-) -- 2.1.4 diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 59274e2c1ac4..e95a7d6ac8f8 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -811,9 +811,6 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) pte_t *pte; int i; - if (pud_none(*pud)) - return 1; - pmd = (pmd_t *)pud_page_vaddr(*pud); pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); if (!pmd_sv) @@ -855,9 +852,6 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; - if (pmd_none(*pmd)) - return 1; - pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); From patchwork Mon Nov 26 17:07:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 152054 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp92144ljp; Mon, 26 Nov 2018 09:08:23 -0800 (PST) X-Google-Smtp-Source: AFSGD/UL2hI6UI9U+pT/0IiH2iUaCGQh3sc4fWuw2t5CQZDvO5FeyLx4BoqqEzkpcRk39C39bcKS X-Received: by 2002:a17:902:8346:: with SMTP id z6mr28512931pln.340.1543252102915; Mon, 26 Nov 2018 09:08:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543252102; cv=none; d=google.com; s=arc-20160816; b=lXOFHyMSjz44hEh+aiIEhNBl8AR29ML+W2AlnXinEhvB1UhtzPwx5XRAGsnWO20ek9 QOBthSlpJIUuGMnleZAHpwKFGrT+7zK2vbiWcdhIifCHFmHqqczUVBf4VGQbHjgsPdvd Tw75IkvhW1Plo2o1JEc5rEWrQ4z5OaYzcInlv2O1FmPRSxUzei7oM88ENDOn/G9KznHX wI8wBt/pOV7JBkRiDvQDUJbh+AGYcfmhDixw6lYsVKc0W2VkYpzVTEkNDkBPx4dfdDBO u+xSe8U9WBj+rHcg0hQW1phKhKtyW3mg79SbmFK/n1UOluEJRLkGIBakw2hjVWlJN2I9 Wzuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=gyaouhO6M09bNRRp3jEmM8H6PKnPVZFp5tgRSOGTTCE=; b=HnGJ2a4WP77PvpF1/Vr7aO3M5scXY9GZnjC2FFh1GyiiHb5YYVXHsD0bu6rVhSR/qm T8kZ1w4wSFx8ZOpzqursAJTesxlfDeULuJep5BZdedKzp9X59OhkTYVjVRQMkZtWBmn4 pyzMQo+4GuuUxPSxQ/SZ1mc9AsqQ03e0HKP3YwpJQhBqA3z9+F4ADoQDKQNAWn81c74u rqJbtKEWbmz0yLhky1jt0yGd+34H3RDDrrgiHS7tQfU2e3i/2nkzI6qvhd2C1MFHOENk pQwhWoM4o/8pQwDxjCQ+yVYsvR0WkzUb5k0P7ZU35jHZT3XW/cBZzWUMT5O4OH6/zxIq MeYg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y25si809125pgl.226.2018.11.26.09.08.12; Mon, 26 Nov 2018 09:08:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726956AbeK0ECf (ORCPT + 32 others); Mon, 26 Nov 2018 23:02:35 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:42960 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726210AbeK0ECN (ORCPT ); Mon, 26 Nov 2018 23:02:13 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 090881993; Mon, 26 Nov 2018 09:07:31 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CD0B03F59C; Mon, 26 Nov 2018 09:07:30 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 47AA11AE0B68; Mon, 26 Nov 2018 17:07:48 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v4 4/5] lib/ioremap: Ensure phys_addr actually corresponds to a physical address Date: Mon, 26 Nov 2018 17:07:46 +0000 Message-Id: <1543252067-30831-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1543252067-30831-1-git-send-email-will.deacon@arm.com> References: <1543252067-30831-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current ioremap() code uses a phys_addr variable at each level of page table, which is confusingly offset by subtracting the base virtual address being mapped so that adding the current virtual address back on when iterating through the page table entries gives back the corresponding physical address. This is fairly confusing and results in all users of phys_addr having to add the current virtual address back on. Instead, this patch just updates phys_addr when iterating over the page table entries, ensuring that it's always up-to-date and doesn't require explicit offsetting. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Cc: Sean Christopherson Signed-off-by: Will Deacon --- lib/ioremap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) -- 2.1.4 Tested-by: Sean Christopherson Reviewed-by: Sean Christopherson diff --git a/lib/ioremap.c b/lib/ioremap.c index 6c72764af19c..10d7c5485c39 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -101,19 +101,18 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, pmd_t *pmd; unsigned long next; - phys_addr -= addr; pmd = pmd_alloc(&init_mm, pud, addr); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) continue; - if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_pte_range(pmd, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pmd++, addr = next, addr != end); + } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -142,19 +141,18 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, pud_t *pud; unsigned long next; - phys_addr -= addr; pud = pud_alloc(&init_mm, p4d, addr); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pud(pud, addr, next, phys_addr, prot)) continue; - if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_pmd_range(pud, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pud++, addr = next, addr != end); + } while (pud++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -164,7 +162,6 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, p4d_t *p4d; unsigned long next; - phys_addr -= addr; p4d = p4d_alloc(&init_mm, pgd, addr); if (!p4d) return -ENOMEM; @@ -173,14 +170,14 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, if (ioremap_p4d_enabled() && ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr + addr, prot)) + IS_ALIGNED(phys_addr, P4D_SIZE)) { + if (p4d_set_huge(p4d, phys_addr, prot)) continue; } - if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) + if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM; - } while (p4d++, addr = next, addr != end); + } while (p4d++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -196,14 +193,13 @@ int ioremap_page_range(unsigned long addr, BUG_ON(addr >= end); start = addr; - phys_addr -= addr; pgd = pgd_offset_k(addr); do { next = pgd_addr_end(addr, end); - err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); + err = ioremap_p4d_range(pgd, addr, next, phys_addr, prot); if (err) break; - } while (pgd++, addr = next, addr != end); + } while (pgd++, phys_addr += (next - addr), addr = next, addr != end); flush_cache_vmap(start, end); From patchwork Mon Nov 26 17:07:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 152051 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp91381ljp; Mon, 26 Nov 2018 09:07:46 -0800 (PST) X-Google-Smtp-Source: AJdET5cd1IqGaw2gaLygeEZ+Rs2Dys36eF4+zM8KiJRG60+AEQlLZhnykGbePL0dFZspfXoiVNiZ X-Received: by 2002:a62:3687:: with SMTP id d129-v6mr28991303pfa.56.1543252066824; Mon, 26 Nov 2018 09:07:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543252066; cv=none; d=google.com; s=arc-20160816; b=n3EFNWXpW7QOvaEI0MfeQYWlZwfm6BbNYo5iCbUECpcAdH6EslIYo8ZsoSVGVJ58Sl PoKWjKOvgYbFG6L+11xLzzOkWH8VOLOrXe300AoNSv+DuB0huM7SGz38ymkGPIX+a37B lN9510FbZGSKv4uQFMxTgR7LNPkBKpFQSZwSExLTTg2ttljlNT/GilN8Srg89Vbk3mHb La6IDq7iNuyGLmRT0mBQxBro838v4nNVD4Gb46aCftf3iuELj7CLDvthQNGoh34f5FSO gGJ2gnqzFktYWXeejwA4gJGFSxg2KGXEv9B5QIK/Iyi7bn4P6RRto9UUBOS9GxDN4Rzc Bncg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=l44sI+xexd99+rlEiWTnTtPsv1VhQI3OmU75tH27288=; b=zLaqEkgxDaFyJCD2A3TIUVDfz1S8MPuupbzbLwU21xW87b7FQXhEK3B5lWzR1U3IOD kC9IC0ma1J7SXDde8hcdTmo2hL/Ay20T9zonQ2+cLV9pr8I4V+VH7as7uk/hzmvFF5ig H/W2jb+XTZX94dUcFAQA3EEtXHiIqpt1DA99XWIdfOva+CYurJChAo2OMDgYVJQ7w012 x2aocPRgcQIybxhwIFk9PYVzRuTJ1sg5GPWTnNFo1oymGfLYEriHzbi6r37AFXbpFegp aKFCzzUMyJmE0L8zXyiT/+hFw6DCThpJz2HKV9iAu3eS66E3XDbjwVur7qzdaZC8EMuW 8wgQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y25si809125pgl.226.2018.11.26.09.07.45; Mon, 26 Nov 2018 09:07:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726781AbeK0ECO (ORCPT + 32 others); Mon, 26 Nov 2018 23:02:14 -0500 Received: from foss.arm.com ([217.140.101.70]:42984 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726260AbeK0ECO (ORCPT ); Mon, 26 Nov 2018 23:02:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AB6F019BF; Mon, 26 Nov 2018 09:07:31 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7C2B43F59C; Mon, 26 Nov 2018 09:07:31 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 59FE01AE0C3D; Mon, 26 Nov 2018 17:07:48 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v4 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Mon, 26 Nov 2018 17:07:47 +0000 Message-Id: <1543252067-30831-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1543252067-30831-1-git-send-email-will.deacon@arm.com> References: <1543252067-30831-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 786cfa6355be..cf9a26d3d7f5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1028,3 +1028,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index e95a7d6ac8f8..b0284eab14dc 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -794,6 +794,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 359fb935ded6..e0381a4ce7d4 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index 10d7c5485c39..063213685563 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;