From patchwork Wed Sep 12 10:26:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 146547 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp4647131ljw; Wed, 12 Sep 2018 03:26:19 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbY7BTOYVlFkW6EGEolA9b6KQ+J+3DCVN64kh3jWdpTIrUN6ciMxdGIWoGlX+gfa5w+Iq2n X-Received: by 2002:a17:902:8e81:: with SMTP id bg1-v6mr1416703plb.129.1536747979838; Wed, 12 Sep 2018 03:26:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747979; cv=none; d=google.com; s=arc-20160816; b=kVlThctpnaOEdWTwzBbeN5o1d8WUAuVIH0tgHSrijReM5cGIcAkSmN6SkUUHd5iPUI 0zsYlkAKaM51M87Gs/7xTb6d+IDEq6FBWm36dMg7TG88kzvkce/L3CcfiWqZ32yGbjuG 28snw5dbmupdEjbfUwoXxsGHKTIVjm2TD+c7RLf8xsHaEhhk/2NxsDm7/WWhuyDYL2yA YZzFCZpzIJywtH9GjsV8BOTTutx0WOlx9enTakl27PfpdefdZgP2KZFNgNuZdYHMs+HZ nEyBbMB7HqrWamcFL2Fhye36uPIO7eFuurrmhLaSBNup8IVj9CNn82aeRSuAQp9ovMe+ 0g8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=9KAgIv8nfsqMOg72uzaVHfLrGe37Gw1OAqB4rZKj4mk=; b=EYA3JuMskTvCAMnUsMJjYCvh+AtHKFGGJVA5w/2v8efAmb7DHzDieDpp02cyyorvag Wqv5/0Q4ROYqOzUhrASxI+snFrHiKVMjMkwFWurIAJL2565/VsqSx89ub5chGf58Jnvx Q2ZiWuidUUpNlaIvQV++o0wG1qFQtODY/3nxO6LEO7Zz+jKApmmBQ1PAPglrRhwzsbL4 216LcVhwDdCjZDqL+17W9iH/Ss4sTyIeQ6CiP7G1MUBy5RXP3sCz+P41DBRv+cmNVGY5 Qx7CCsOIXey56nHOnVzPTWjVjwWCN6VQ+bsiM5SUKwCFEbJBNIrk9K1owx7ow5/mYEz8 2xmw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b12-v6si669661pgh.264.2018.09.12.03.26.19; Wed, 12 Sep 2018 03:26:19 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727992AbeILP3z (ORCPT + 32 others); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:56964 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726672AbeILP3z (ORCPT ); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E54361596; Wed, 12 Sep 2018 03:26:01 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B73E03F71F; Wed, 12 Sep 2018 03:26:01 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 841A41AE3018; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 1/5] ioremap: Rework pXd_free_pYd_page() API Date: Wed, 12 Sep 2018 11:26:10 +0100 Message-Id: <1536747974-25875-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The recently merged API for ensuring break-before-make on page-table entries when installing huge mappings in the vmalloc/ioremap region is fairly counter-intuitive, resulting in the arch freeing functions (e.g. pmd_free_pte_page()) being called even on entries that aren't present. This resulted in a minor bug in the arm64 implementation, giving rise to spurious VM_WARN messages. This patch moves the pXd_present() checks out into the core code, refactoring the callsites at the same time so that we avoid the complex conjunctions when determining whether or not we can put down a huge mapping. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Suggested-by: Linus Torvalds Signed-off-by: Will Deacon --- lib/ioremap.c | 56 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) -- 2.1.4 Reviewed-by: Toshi Kani Reviewed-by: Toshi Kani diff --git a/lib/ioremap.c b/lib/ioremap.c index 517f5853ffed..6c72764af19c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -76,6 +76,25 @@ static int ioremap_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pmd_enabled()) + return 0; + + if ((end - addr) != PMD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PMD_SIZE)) + return 0; + + if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + return 0; + + return pmd_set_huge(pmd, phys_addr, prot); +} + static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -89,13 +108,8 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, do { next = pmd_addr_end(addr, end); - if (ioremap_pmd_enabled() && - ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd, addr)) { - if (pmd_set_huge(pmd, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) return -ENOMEM; @@ -103,6 +117,25 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, return 0; } +static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pud_enabled()) + return 0; + + if ((end - addr) != PUD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PUD_SIZE)) + return 0; + + if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + return 0; + + return pud_set_huge(pud, phys_addr, prot); +} + static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -116,13 +149,8 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, do { next = pud_addr_end(addr, end); - if (ioremap_pud_enabled() && - ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud, addr)) { - if (pud_set_huge(pud, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) return -ENOMEM; From patchwork Wed Sep 12 10:26:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 146548 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp4647214ljw; Wed, 12 Sep 2018 03:26:25 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYGSUBSa5c6k99ccu1nJKnrbGEm8pyAOGbBzTpLh75ZhRJv1hXHos9klpo392DPHIQ90QZv X-Received: by 2002:a63:352:: with SMTP id 79-v6mr1454463pgd.112.1536747984807; Wed, 12 Sep 2018 03:26:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747984; cv=none; d=google.com; s=arc-20160816; b=kHeD0HAPKr8cB5pk2jMIq9v2qLidEB7vH2lzRutCLen/ntkUcR5B5g43Y+JKWrDec9 Kk/hTjd1DGy+MgzC8+vcfkAI9hpbSMCf1gHTVbWTFSe6A4L3cM0cnHJ1+JKTQmnUhwiM C7S2/yMEslEfcNhWoGGIUyOZRs62a+vbVX714JZuS/iIwDRNvPTnj/p+joXCbWh/+3fp PI2rOvMsbXX99J04NcFXYhuJkKZoa2/7Lx+PJh6Ya+bQ5Xl52EBN0IictIO0j5rG1VVJ IiaLCRpg5B0d7FQBoW4JSzTV931SqHTdg27c9kUibp1K7iSbpuMEXrijqjBDJdkmdKVc hlRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=5Z6J2RQTqMWhpJ0pzFttI0SwIzpZ6lRPEY1BeHTRt8o=; b=mloKecCYlYQgPxO+MmYczi9r5B+BWiL47+TqDeCZ/Wh30K4Mshk++LNcVjg1UeQ99T zDQUrCTnXzo5YY1yDSdeTTgS37oYxipYHzLdKVfR9KcsAxhj8LC6Thg7qovDDYnndqQu 321KzpqbNefX6G6AygshzYjv1OqhaB3K4iJXv/imhAHCcLhh7y9sm/+mclPH54gSG3UC qs19Rp4d8vTKYbJNEhsdZz9Z8VLij84qpVrV/fzTCE4OBQFL7TFsSnJYb4JAQjE3aZuo Fl1XW/ZnLgbHc9X1Ku1HXBqPpjJS4puoTA5rY10OiU/GRS5GlVZe0zcCGMMH1fJBznHe V+qg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b12-v6si669661pgh.264.2018.09.12.03.26.24; Wed, 12 Sep 2018 03:26:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727875AbeILP3z (ORCPT + 32 others); Wed, 12 Sep 2018 11:29:55 -0400 Received: from foss.arm.com ([217.140.101.70]:56980 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727161AbeILP3z (ORCPT ); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 01D3B15BF; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C76BD3F7DB; Wed, 12 Sep 2018 03:26:01 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 939EC1AE3318; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 2/5] arm64: mmu: Drop pXd_present() checks from pXd_free_pYd_table() Date: Wed, 12 Sep 2018 11:26:11 +0100 Message-Id: <1536747974-25875-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the core code checks this for us, we don't need to do it in the backend. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8080c9f489c3..0dcb3354d6dd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -985,10 +985,8 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - if (!pmd_present(pmd)) - return 1; if (!pmd_table(pmd)) { - VM_WARN_ON(!pmd_table(pmd)); + VM_WARN_ON(1); return 1; } @@ -1008,10 +1006,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - if (!pud_present(pud)) - return 1; if (!pud_table(pud)) { - VM_WARN_ON(!pud_table(pud)); + VM_WARN_ON(1); return 1; } From patchwork Wed Sep 12 10:26:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 146546 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp4647066ljw; Wed, 12 Sep 2018 03:26:14 -0700 (PDT) X-Google-Smtp-Source: ANB0VdanaJ/FTPZIaTAsDfz7SNtpL7njnWTTBPoCyv8FF6HC203L1YfFlDB0WiHeU6AJUfInLf5L X-Received: by 2002:a65:448c:: with SMTP id l12-v6mr1432855pgq.277.1536747974765; Wed, 12 Sep 2018 03:26:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747974; cv=none; d=google.com; s=arc-20160816; b=EKsvI4c84vfwtCNH7b2gEchBT3/0KyI08/3FLkBC0PeqDmNaB7rgvHJdtvJ+4BAKbt A6BuiukS5vtBrLgRCpRhA3+ntf7I7jo6AF9fcOYHmeA10UdRco+M30QW+so8a3WC21Ze P4zahDIoKj9QSvdY4MRV63pEFD87XH4psGWieDQVPJoOjfWLZUna5uDwpwQNIuEVixRx SPPODgNd9lYYmh1U7TmbhYNFxO/nVzP4Yb+OYgFIg2DjScAhXhk6a85n6tPHBVMqKqtu 8DfGB5vVXYS7R2E+HDFB3BPuNHZC+As7RuBaQwZ9Ks7J25prDSjq8UvgV33hmOY1yk96 Argg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=1JT3gBGB2fQ1eze1dftN8gxIxDA2nlm9MPmGKsks1qM=; b=zpcpdWBtm/LTdWtxrKJQ8hgpDBta2GqXqwfWwySgWIuuTLJt/uufl59zW10SbCg526 zCPQcsWfbh4Xi52o7vwYXy4Xb8Q8MABSxcqj3Y034d+wDr6y+42flSnqZiliypZ7SCJl MPNRDLnNIns0ip7I4vGIps+HYxl3YH8r3tidQMjn+JwAOuns6sFtjmRGXz3orT1d2Gai ROZWJ0N8FwUu9AKU0okp6B/SEqvOMTcxY51PA8NNn11GUpMZ4B6IzJ4wo60+Nwq4E0u9 /yq769iE/+8OyRTbcVx7TYZBdvHTDrTkwnmwwTgFoaIztNTJi1y8SIViMJZWE8I8PM1p jqcQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y3-v6si648528pgg.266.2018.09.12.03.26.14; Wed, 12 Sep 2018 03:26:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728014AbeILP3z (ORCPT + 32 others); Wed, 12 Sep 2018 11:29:55 -0400 Received: from foss.arm.com ([217.140.101.70]:56992 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727278AbeILP3z (ORCPT ); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 111FC1682; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D766F3F907; Wed, 12 Sep 2018 03:26:01 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id A3E011AE33B8; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 3/5] x86: pgtable: Drop pXd_none() checks from pXd_free_pYd_table() Date: Wed, 12 Sep 2018 11:26:12 +0100 Message-Id: <1536747974-25875-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now that the core code checks this for us, we don't need to do it in the backend. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/x86/mm/pgtable.c | 6 ------ 1 file changed, 6 deletions(-) -- 2.1.4 Reviewed-by: Toshi Kani diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index ae394552fb94..b4919c44a194 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -796,9 +796,6 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) pte_t *pte; int i; - if (pud_none(*pud)) - return 1; - pmd = (pmd_t *)pud_page_vaddr(*pud); pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); if (!pmd_sv) @@ -840,9 +837,6 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; - if (pmd_none(*pmd)) - return 1; - pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); From patchwork Wed Sep 12 10:26:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 146544 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp4646928ljw; Wed, 12 Sep 2018 03:26:05 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbkUqaXRW/lpVHeE7yiAD41PDJsVCR0ns4Al+ZvDS7wN4ZDjHACPq9OE3dcpLqOlQQOng/F X-Received: by 2002:a63:8543:: with SMTP id u64-v6mr1475018pgd.248.1536747965506; Wed, 12 Sep 2018 03:26:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747965; cv=none; d=google.com; s=arc-20160816; b=vZBfKHLkJakPFRFTEwK5/59L4QVXlnZRwfIESvmPZ/zXrMmINlwWB+/EsyokuwrMdG q5XQxc2ZRJrFkEAgMy5WhnxNzY+NmTTlPzjZnMY71DbW65EQCgIYJ7j7okbW6WDvwRFL W026s13Yj9JDQ58M/dRCu48FLAXluuX/4xhm5G0l5svKgSD2b4GmC0zC4jU70DaaIavZ xih9+RPRlr/AiHRGzFcrkic7q5+uN6+WPCn6SIrHsibpVZRiijMeveuyaRI1j3Kfga5P zgF3IY9LMqfMifP6tjgw/5zxm9gkoPfJdZ3pNmQrzrOv1UBMzyLPCQJGSVWV/wY/kbsg ggDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=mCQhMX470BDGByPnZUeAM4vjCfUfWAlr0VZ7SYs45do=; b=pFBvknnq+mp/QtnSJ1I/7BXzYEqElLVn8SoZ5VVJh1nEQhc1uDw/KLRLwjMkkOO6Jp m6QxEFclKYIQr+F8xuDkLWMBCSoIaoowo5txe8gbbwXiG8+JgxGuTaKT+g/tiHZR10Mm XIQykhaK6w1NKUtgzzLu+R0iTE3YULkfrqYWCMAT/yki8WkNbWS/aBmrTJWYZ7MIxJuE DqK5ZW932lJcL9SdWV45XDtp+qBu89hyOY5k8TUA4hRQx6GSJm+qPpcpbN1IBXnfqCix 7b1VAYHPIg2vYwRo6e2orbzN+Lj0ysO6xa7MYZOQUvutYGAf9eJB3sgMwIYa1t2j7Jqc Dj2Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r84-v6si573114pfj.355.2018.09.12.03.26.05; Wed, 12 Sep 2018 03:26:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728050AbeILP3z (ORCPT + 32 others); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57002 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727611AbeILP3z (ORCPT ); Wed, 12 Sep 2018 11:29:55 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1F1481684; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E55DB3F614; Wed, 12 Sep 2018 03:26:01 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id B42871AE3411; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 4/5] lib/ioremap: Ensure phys_addr actually corresponds to a physical address Date: Wed, 12 Sep 2018 11:26:13 +0100 Message-Id: <1536747974-25875-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current ioremap() code uses a phys_addr variable at each level of page table, which is confusingly offset by subtracting the base virtual address being mapped so that adding the current virtual address back on when iterating through the page table entries gives back the corresponding physical address. This is fairly confusing and results in all users of phys_addr having to add the current virtual address back on. Instead, this patch just updates phys_addr when iterating over the page table entries, ensuring that it's always up-to-date and doesn't require explicit offsetting. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- lib/ioremap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 6c72764af19c..fc834a59c90c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -101,19 +101,18 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, pmd_t *pmd; unsigned long next; - phys_addr -= addr; pmd = pmd_alloc(&init_mm, pud, addr); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) continue; - if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_pte_range(pmd, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pmd++, addr = next, addr != end); + } while (pmd++, addr = next, phys_addr += PMD_SIZE, addr != end); return 0; } @@ -142,19 +141,18 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, pud_t *pud; unsigned long next; - phys_addr -= addr; pud = pud_alloc(&init_mm, p4d, addr); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pud(pud, addr, next, phys_addr, prot)) continue; - if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_pmd_range(pud, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pud++, addr = next, addr != end); + } while (pud++, addr = next, phys_addr += PUD_SIZE, addr != end); return 0; } @@ -164,7 +162,6 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, p4d_t *p4d; unsigned long next; - phys_addr -= addr; p4d = p4d_alloc(&init_mm, pgd, addr); if (!p4d) return -ENOMEM; @@ -173,14 +170,14 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, if (ioremap_p4d_enabled() && ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr + addr, prot)) + IS_ALIGNED(phys_addr, P4D_SIZE)) { + if (p4d_set_huge(p4d, phys_addr, prot)) continue; } - if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) + if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM; - } while (p4d++, addr = next, addr != end); + } while (p4d++, addr = next, phys_addr += P4D_SIZE, addr != end); return 0; } @@ -196,14 +193,13 @@ int ioremap_page_range(unsigned long addr, BUG_ON(addr >= end); start = addr; - phys_addr -= addr; pgd = pgd_offset_k(addr); do { next = pgd_addr_end(addr, end); - err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); + err = ioremap_p4d_range(pgd, addr, next, phys_addr, prot); if (err) break; - } while (pgd++, addr = next, addr != end); + } while (pgd++, addr = next, phys_addr += PGDIR_SIZE, addr != end); flush_cache_vmap(start, end); From patchwork Wed Sep 12 10:26:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 146545 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp4646934ljw; Wed, 12 Sep 2018 03:26:06 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYZjYBUqzhG5FUo6n3SSz3BzdzggJaNDP5fTQCiDul/iyqY61T/VZ9Aqh6gn7d1B8k5jDjd X-Received: by 2002:a62:9645:: with SMTP id c66-v6mr1503191pfe.56.1536747965868; Wed, 12 Sep 2018 03:26:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747965; cv=none; d=google.com; s=arc-20160816; b=dC44dd1frohwqbwipDKiO3zhV55+/UFsy56QyXjowdburS5xJ5BxDpEkVBhZ3tQWA8 OY0slsDwX6+Go5GFk4Fj9c/v3Qdhoo3N7qFdNotHXm3On/yNZTLzxoA1YbZMUtQtg0Vr OChJmxSaGmXxBqzkOSImSvf+Q5x6Av/HGvmYeWGX0kOc8JuTmhkb7xjJrmS9k64YvALW Vq4hemv7DcZCRU+GWGgSyledqT3wz6i/HHwvwXkXy1K5j0N1VyA4ZpIdGVLzBKTTP74Z TFwOl0+vhh7UMtlbP59s0YEx/8BFYjVJ6OOR1vCyw8cbErzUh3IU0nSfwiZVwRlrMeDI aI8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=UQhbSo2CjhJBbAoBN3seYvduCvHh3YrpIQUGvyxA6kg=; b=kLDuU9HJAG7v68VWlL0LNCmwLzHeMadb4/pXLHUtlRi7sJj82kOjeJNpt3l94ruP5m 4fNzwOq4IWDo5jGaJqRhxvEHhgv59YGvVnAiApZ91ArSCzk3wN2bGV0mbeXxtGfQYJrq 97my6qhUlB/n35q7xkvGFVZj8KJL5C9C2CYuLhuOpQRgi+IMtpgogGDbAluGhzLlDjmc n9AhTzEsfmP4ZiuI6PUCqm5f9DZAP0+JjRK4v5pYuWi6nDfUV4syokUIV48hHZ7g8ETH nnv68WmJABqsunkuBiR8a6i/bHrY6vyw+SqA8GHfIWXcHOERdrnLtManqfm29rb8oNmI 3bAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r84-v6si573114pfj.355.2018.09.12.03.26.05; Wed, 12 Sep 2018 03:26:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728086AbeILP34 (ORCPT + 32 others); Wed, 12 Sep 2018 11:29:56 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57040 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728001AbeILP34 (ORCPT ); Wed, 12 Sep 2018 11:29:56 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C15401688; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 936BE3F614; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id C26BA1AE357F; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Wed, 12 Sep 2018 11:26:14 +0100 Message-Id: <1536747974-25875-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) -- 2.1.4 Reviewed-by: Toshi Kani diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0dcb3354d6dd..58776b90dd2a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1024,3 +1024,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index b4919c44a194..c6094997d060 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -779,6 +779,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 88ebc6102c7c..4297a2519ebf 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index fc834a59c90c..49d2e23dad2e 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;