From patchwork Wed Oct 10 16:23:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 148567 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1062075lji; Wed, 10 Oct 2018 09:23:17 -0700 (PDT) X-Google-Smtp-Source: ACcGV616ZN8Scrx/PM3q18MII4yAYf2Bk+0i37acdEFV5g6XFdyM0+58aciv5Yuw+AV2Jr5PqaEB X-Received: by 2002:a62:8708:: with SMTP id i8-v6mr35320495pfe.150.1539188597346; Wed, 10 Oct 2018 09:23:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539188597; cv=none; d=google.com; s=arc-20160816; b=d6NIrrhFp6KKAmAZUyg+/5YccmFr8Ya4LjxZJLK6qq0R2uHScNq6Nmbd1+1U533NcE 10SWszi9FWM4VZvrQhOdR8wVDnbySuh/m4EleEviLpvQidkPsnmpAlj+IqREl+s+7owj Hgq4bHiN2X5u4ZtyVMQhpp15mdi9uHz2Nhbqq0Lj6wNgIkWDD35MSCvQSusR2xRuTBNq fnqMJZa/UEaB18YF484o9X/5hfCrLbNKvRjPd0IBHfzRXvgxP9Cy2/lzxPYAsmy31Tca 3XSxWF5q8/ObrnwPsoMxDyDUOdJC+3ZUNwJ5wmtTAMEUczJKuzzpwswdn0QmAA4G3eW1 X9zQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Ka6oXZu8DJRXSCZkb2TermywvttDv1cohIKM9nr4o2s=; b=A2Tn3+ejaf0AURjx7s9SSfPnkfEdz+XWVn66e6+/wVkOjX3zqLF3PAdE2Sf6N83g99 Q+h0oH5BrPs+ezWikqYuuL3Dwf4c9mcqDSZgFVoy916+lkU5b2OQrxB+vXu/ra2bWoph llF77GzB7TvXArJZvPq9xFVTusKUgs9m/mOvG56QcFK/fLi2LHDKC6KxCVhDRHRyUqdV yPV7+1k2bu8Eo6ZupWIZLJmWmBTht6ZEuFzRqVfz3F9NkZVd44I3V6ziD0LGRElX9jWO IG5h4NZYbjmkP367DW9GSmE/Gw/aGJBZFVMEfYWUQhuD8r98IPv6Q3QUPbPM9d0azy66 SDYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x18-v6si69516plr.146.2018.10.10.09.23.17; Wed, 10 Oct 2018 09:23:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726871AbeJJXqH (ORCPT + 32 others); Wed, 10 Oct 2018 19:46:07 -0400 Received: from foss.arm.com ([217.140.101.70]:54800 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726479AbeJJXqG (ORCPT ); Wed, 10 Oct 2018 19:46:06 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 59B261596; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2BBA93F740; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id D75B21AE07A7; Wed, 10 Oct 2018 17:23:12 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v3 1/5] ioremap: Rework pXd_free_pYd_page() API Date: Wed, 10 Oct 2018 17:23:00 +0100 Message-Id: <1539188584-15819-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1539188584-15819-1-git-send-email-will.deacon@arm.com> References: <1539188584-15819-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The recently merged API for ensuring break-before-make on page-table entries when installing huge mappings in the vmalloc/ioremap region is fairly counter-intuitive, resulting in the arch freeing functions (e.g. pmd_free_pte_page()) being called even on entries that aren't present. This resulted in a minor bug in the arm64 implementation, giving rise to spurious VM_WARN messages. This patch moves the pXd_present() checks out into the core code, refactoring the callsites at the same time so that we avoid the complex conjunctions when determining whether or not we can put down a huge mapping. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Suggested-by: Linus Torvalds Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- lib/ioremap.c | 56 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 14 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 517f5853ffed..6c72764af19c 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -76,6 +76,25 @@ static int ioremap_pte_range(pmd_t *pmd, unsigned long addr, return 0; } +static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pmd_enabled()) + return 0; + + if ((end - addr) != PMD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PMD_SIZE)) + return 0; + + if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + return 0; + + return pmd_set_huge(pmd, phys_addr, prot); +} + static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -89,13 +108,8 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, do { next = pmd_addr_end(addr, end); - if (ioremap_pmd_enabled() && - ((next - addr) == PMD_SIZE) && - IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd, addr)) { - if (pmd_set_huge(pmd, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) return -ENOMEM; @@ -103,6 +117,25 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, return 0; } +static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_pud_enabled()) + return 0; + + if ((end - addr) != PUD_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, PUD_SIZE)) + return 0; + + if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + return 0; + + return pud_set_huge(pud, phys_addr, prot); +} + static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -116,13 +149,8 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, do { next = pud_addr_end(addr, end); - if (ioremap_pud_enabled() && - ((next - addr) == PUD_SIZE) && - IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud, addr)) { - if (pud_set_huge(pud, phys_addr + addr, prot)) - continue; - } + if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + continue; if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) return -ENOMEM; From patchwork Wed Oct 10 16:23:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 148569 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1062338lji; Wed, 10 Oct 2018 09:23:31 -0700 (PDT) X-Google-Smtp-Source: ACcGV60jC5mCCoesokkFE7pmhl4OI6VXR2oMapInKSx/Atl9MxZCWbh2FgMUBn5XcpOC4iDq0ZGx X-Received: by 2002:a63:fa09:: with SMTP id y9-v6mr30320520pgh.177.1539188611234; Wed, 10 Oct 2018 09:23:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539188611; cv=none; d=google.com; s=arc-20160816; b=Fp17vQWJJYE4VLtgYSmkljAJ34+L4mItYxq59S2wY1eZO/MyHvEfLKncj6eFx18Eyh /DXvPqOuqlWCZ1vo7sexni0ubTKsKI3uUm7jPacMAXvpaP1A6qE/zygKorG7PNOynF8n i9UxyUTuI6YpZc+kgeFO8tFIoqX+Gxe8sj+mPzqL66HAwnHCnvAVJmz6QDjtA2vFNjin ThrD0tNsucR76HK0E1SGLWsp0UD4QmY8zm5TOxxuXbLZvP6hxsqSvjkyHQh6aDp+naHC Ai2oFRthZz++ygVPjNS+Qthwb7fT7aEiFjM1sDusUgWz1NgpmEjDm7smDj+IJfWUtcJO 0FoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=N4Cv0QE3PH+lPyq9AU+EbNnOQ1f2Mu7HygBKhQjY+BQ=; b=t/VQ9RHFAVFGI4X1uVPeDOl9IUdmoijIX9+UaisUxC35EXPuHcjFe4Cf94gAh9bDJU OdA2rsMP7b4q+LezxCFf5An6ba/JRicKs+VMH6uf4Y0C86P1ocgfPNbT67ILmhR022fQ OimV9wokpicNGuVITn7GkPCjQExUfijB1sEZvj5bMHC+xP0hb16hyYIy0ov7zvUsrQzZ PK1Z+NjU/bFYJDHFPGDHlS/I/jeUomOvUwhXPuaAZYPo7X6Xj1hkh1rmf6n/LtgjqOyq HG/R3GEISSTefcI5XNKP5c1UQtYyMz6Gkcq28ILVYpCvhO9KtQyIC0dkcJXlvWeHqB9L /OIA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s16-v6si23552017pgg.19.2018.10.10.09.23.30; Wed, 10 Oct 2018 09:23:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726728AbeJJXqG (ORCPT + 32 others); Wed, 10 Oct 2018 19:46:06 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:54812 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726515AbeJJXqG (ORCPT ); Wed, 10 Oct 2018 19:46:06 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C08C15B2; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3E11F3F7C3; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id E4FA01AE0964; Wed, 10 Oct 2018 17:23:12 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v3 2/5] arm64: mmu: Drop pXd_present() checks from pXd_free_pYd_table() Date: Wed, 10 Oct 2018 17:23:01 +0100 Message-Id: <1539188584-15819-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1539188584-15819-1-git-send-email-will.deacon@arm.com> References: <1539188584-15819-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 8080c9f489c3..0dcb3354d6dd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -985,10 +985,8 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - if (!pmd_present(pmd)) - return 1; if (!pmd_table(pmd)) { - VM_WARN_ON(!pmd_table(pmd)); + VM_WARN_ON(1); return 1; } @@ -1008,10 +1006,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - if (!pud_present(pud)) - return 1; if (!pud_table(pud)) { - VM_WARN_ON(!pud_table(pud)); + VM_WARN_ON(1); return 1; } From patchwork Wed Oct 10 16:23:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 148571 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1062505lji; Wed, 10 Oct 2018 09:23:40 -0700 (PDT) X-Google-Smtp-Source: ACcGV63/yFaBAUT1/EvJjluNtuIP0+Y3XLIqT/XOTBgSuXWCd/hYWLuFPAGbeFUQoOQ5VjGWDTi+ X-Received: by 2002:a17:902:728e:: with SMTP id d14-v6mr117826pll.51.1539188620074; Wed, 10 Oct 2018 09:23:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539188620; cv=none; d=google.com; s=arc-20160816; b=DZFxAKEYN90A/m6HuyJ7bE+/cxyy2C6+3AMnaJ1VQXuF61KUw+HOhuugH8zcyVjaKm LvXN6DU7HOjwDo9BqGaApG7msKv0KSTgJUCSuMUrxq7fhDBVfXoI6OVk1+Ui4AccXkMd q1ndb8xEy5R4VBKvQMKfoRXQhHSWEG9yNRYBmCuIfgVhvDonRdC6GoF235izZvTk1OSP sYc5o33Q22O5vw6Ae73tbqH7tslFXHc7yj0c1Ej2DiQ1R8a/LkctsTo2g5fd1+gpJ1UC p7cpDaRFGb5gjw6LaYoTLd64tOCd4D8XxJgeaVBT9bTAmSEsziE92crPngmkVerFMNZ+ ptEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Knz78Us/CKYc8fbiR3DNa1DlNQlZ1CX9pImVCVrcNmY=; b=sg3guJgJVfS7QSZ9wVaXG4QVOi9qs+6rY0Q2cRrofN+G+E1vmaewGk0piI3N57ZPZ+ R1U6P8AdBHX8MqpqOB0x/thz6mHAk8y2x8EJyQZq9lNxplcc69kwq9dEwqxCB3+fOO4T 9Ii4PbAVsjBnmIGCHsKfIx6Bdl2x+4+CblZKpRvFDY6Ue0imHKQYpRmx1HY9QBTKKCo/ 6dUNA3/RXCfELKm+5qVDz3mi6F9APCG93FEc22ejT2yror3muPseOFykNqvFVfDLLxkW d2/+iJ6w2XG+iMvJHLjuDLXFFwOMf9w5ljMN4G3cCTFIaDBgw2cCXZWHNKlhpGVhS8kS rpQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y30-v6si24306378pgk.13.2018.10.10.09.23.39; Wed, 10 Oct 2018 09:23:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727007AbeJJXqb (ORCPT + 32 others); Wed, 10 Oct 2018 19:46:31 -0400 Received: from foss.arm.com ([217.140.101.70]:54828 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726562AbeJJXqG (ORCPT ); Wed, 10 Oct 2018 19:46:06 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7BE6315BE; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4DDEC3F909; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 034541AE189D; Wed, 10 Oct 2018 17:23:12 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v3 3/5] x86/pgtable: Drop pXd_none() checks from pXd_free_pYd_table() Date: Wed, 10 Oct 2018 17:23:02 +0100 Message-Id: <1539188584-15819-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1539188584-15819-1-git-send-email-will.deacon@arm.com> References: <1539188584-15819-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The core code already has a check for pXd_none(), so remove it from the architecture implementation. Cc: Chintan Pandya Cc: Toshi Kani Cc: Michal Hocko Cc: Andrew Morton Acked-by: Thomas Gleixner Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/x86/mm/pgtable.c | 6 ------ 1 file changed, 6 deletions(-) -- 2.1.4 diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index ae394552fb94..b4919c44a194 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -796,9 +796,6 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) pte_t *pte; int i; - if (pud_none(*pud)) - return 1; - pmd = (pmd_t *)pud_page_vaddr(*pud); pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); if (!pmd_sv) @@ -840,9 +837,6 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; - if (pmd_none(*pmd)) - return 1; - pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); From patchwork Wed Oct 10 16:23:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 148570 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1062365lji; Wed, 10 Oct 2018 09:23:32 -0700 (PDT) X-Google-Smtp-Source: ACcGV63yz0OgULLD7sLo1uQi3fmOCMta5tqdropY8GbKZj4GTdk1rdmNRE0IQ3c5pK+lwxaNt561 X-Received: by 2002:a65:4cc3:: with SMTP id n3-v6mr30927553pgt.257.1539188612405; Wed, 10 Oct 2018 09:23:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539188612; cv=none; d=google.com; s=arc-20160816; b=QVtaU6MOtlSmrH14K55HXQIjORA2ERzOi3BrZHrLn32+5fNK+GiR05WZBf0BOBzNis nvJ9q9XPSuC1y3wGskTWtgRghHWZYq7JjsUZJ0G9E8/cdslAs4pAZ984r5xWupPrTHX5 akVUf6dePtrn7utf8TR9J6y8xruptaLkCueXDsaP1PWKH600+1CWJ47eNh5NVM+53ecq pQ10KFCtMQ0qO03wJkdqC5RYEAxj0ksRa/N5w9L89tE8z55VGkyvT9RacbGqX6pC/xv8 vgNRjaHFLdPe9pkr2IE14gwPWP4a3ioLkuEqwQpbVR6m96jt0qyNSmRFcGYAkYiFWjq6 X4zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=gyaouhO6M09bNRRp3jEmM8H6PKnPVZFp5tgRSOGTTCE=; b=JR/oepbqFFbJIkonMiKXxBvM98qLXdLZ/64B37ddtI2sK0OdVxG1uckF2W17w0b1UN adE85qYn+xmEB/zCI3RFQ0bMAdU2bWpZzfy6fv2sHe8u4qlpTQ0Z15q7G2FtYwZT6GMl XArxd0OXlILG0wzx/eSwvHPXmkZDqg+HO95nwR4omkPZW1xt5wzTrmcC7NxuvnJklgyi vLuLEO5ujZL3UqnmXbSa8v79DKzYhrlFrACJgIvyyn4Os6Ei2PjRoXaxrUev97pvRFkA y7uQtfZe1TtTYJLoYesgZNsLrZr/tecK+sTrPDR/CVgWQUnuofVhq7wlO7y1EN13U6Pj aqRw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s16-v6si23552017pgg.19.2018.10.10.09.23.32; Wed, 10 Oct 2018 09:23:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726946AbeJJXqW (ORCPT + 32 others); Wed, 10 Oct 2018 19:46:22 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:54858 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726649AbeJJXqG (ORCPT ); Wed, 10 Oct 2018 19:46:06 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8EEF31682; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 60D913F5B3; Wed, 10 Oct 2018 09:23:13 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 137311AE2D6C; Wed, 10 Oct 2018 17:23:13 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v3 4/5] lib/ioremap: Ensure phys_addr actually corresponds to a physical address Date: Wed, 10 Oct 2018 17:23:03 +0100 Message-Id: <1539188584-15819-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1539188584-15819-1-git-send-email-will.deacon@arm.com> References: <1539188584-15819-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The current ioremap() code uses a phys_addr variable at each level of page table, which is confusingly offset by subtracting the base virtual address being mapped so that adding the current virtual address back on when iterating through the page table entries gives back the corresponding physical address. This is fairly confusing and results in all users of phys_addr having to add the current virtual address back on. Instead, this patch just updates phys_addr when iterating over the page table entries, ensuring that it's always up-to-date and doesn't require explicit offsetting. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Cc: Sean Christopherson Signed-off-by: Will Deacon --- lib/ioremap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) -- 2.1.4 diff --git a/lib/ioremap.c b/lib/ioremap.c index 6c72764af19c..10d7c5485c39 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -101,19 +101,18 @@ static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, pmd_t *pmd; unsigned long next; - phys_addr -= addr; pmd = pmd_alloc(&init_mm, pud, addr); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pmd(pmd, addr, next, phys_addr, prot)) continue; - if (ioremap_pte_range(pmd, addr, next, phys_addr + addr, prot)) + if (ioremap_pte_range(pmd, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pmd++, addr = next, addr != end); + } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -142,19 +141,18 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, pud_t *pud; unsigned long next; - phys_addr -= addr; pud = pud_alloc(&init_mm, p4d, addr); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (ioremap_try_huge_pud(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_try_huge_pud(pud, addr, next, phys_addr, prot)) continue; - if (ioremap_pmd_range(pud, addr, next, phys_addr + addr, prot)) + if (ioremap_pmd_range(pud, addr, next, phys_addr, prot)) return -ENOMEM; - } while (pud++, addr = next, addr != end); + } while (pud++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -164,7 +162,6 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, p4d_t *p4d; unsigned long next; - phys_addr -= addr; p4d = p4d_alloc(&init_mm, pgd, addr); if (!p4d) return -ENOMEM; @@ -173,14 +170,14 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, if (ioremap_p4d_enabled() && ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr + addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr + addr, prot)) + IS_ALIGNED(phys_addr, P4D_SIZE)) { + if (p4d_set_huge(p4d, phys_addr, prot)) continue; } - if (ioremap_pud_range(p4d, addr, next, phys_addr + addr, prot)) + if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM; - } while (p4d++, addr = next, addr != end); + } while (p4d++, phys_addr += (next - addr), addr = next, addr != end); return 0; } @@ -196,14 +193,13 @@ int ioremap_page_range(unsigned long addr, BUG_ON(addr >= end); start = addr; - phys_addr -= addr; pgd = pgd_offset_k(addr); do { next = pgd_addr_end(addr, end); - err = ioremap_p4d_range(pgd, addr, next, phys_addr+addr, prot); + err = ioremap_p4d_range(pgd, addr, next, phys_addr, prot); if (err) break; - } while (pgd++, addr = next, addr != end); + } while (pgd++, phys_addr += (next - addr), addr = next, addr != end); flush_cache_vmap(start, end); From patchwork Wed Oct 10 16:23:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 148568 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp1062176lji; Wed, 10 Oct 2018 09:23:23 -0700 (PDT) X-Google-Smtp-Source: ACcGV61FXu5g0K38kC49MKfSO52wu5G5xr+3Pq6BjCKWRe155QYHaS42stxYiM/83pSSZJtNHKJh X-Received: by 2002:a63:6b05:: with SMTP id g5-v6mr30648612pgc.344.1539188603062; Wed, 10 Oct 2018 09:23:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539188603; cv=none; d=google.com; s=arc-20160816; b=BxzA+PKdrX1S0hORqQSIE5J2eZKRrpLRAQxgcDygMK4YCnjc0nA5qgsg+gJvQnMhB2 TayPlFqbJNT3oxK4a2i1zlZZ2qIF/M2lNQtSKxMEgCdqoM15aX9vNH7pbmP54MYwqJQo TiYqO004rrS6E+0ZLvsupy8MbUHWUO179vk8HFrqkeCdVyQzyLUX6DipskwgHd8T3xoG 7d9csp3Vh8F/NmEMillHfjiN7QziptSu7yFSbrWzAiwDexxUYtnAYUObd+hcQQPWOhQe SjNg/5lxSF04TDRO6H0Ei7cJYG0f31hR7O1wV/t8B4SFf4cEbmPMvEI4jNd1JjiKWChz 4u+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=AUz4K2RTcj/O0JOlaWId1Il0sNteqiutU7pUtx2yB/c=; b=qFn7iZmVDHe0/H9taWoKAB77Cty20CIDCFNeQPrPuu1zv8yK2Ah3pukafVQEYPGFB6 Pa6souAYYnH9iyWjOMrrNZewJllosc1wR46Aj4TV25SeXjp/aId3mgTnlWbcm3IhQDHn eB2UM8t1oMwVZTt2cp1hF/3llMtqWaomJoiftKUbDXjK+VTLfp5bo4oJVVvcQqHssQaX rU+yWviwslbf0kvGAwbedWn6CjBSwZsQt0Rb0CwgYcAHXnRuU41RXb+ZwefZQ9ILf9yn N0tOBkkmzKB3gn/iWxuaCE2iQ5O7M48BGWb7hHvg1KYlRVI6WB6HX3CEElsI8BaY6zSq rlzA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x18-v6si69516plr.146.2018.10.10.09.23.22; Wed, 10 Oct 2018 09:23:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726921AbeJJXqO (ORCPT + 32 others); Wed, 10 Oct 2018 19:46:14 -0400 Received: from foss.arm.com ([217.140.101.70]:54898 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726503AbeJJXqH (ORCPT ); Wed, 10 Oct 2018 19:46:07 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3B2B51688; Wed, 10 Oct 2018 09:23:14 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0D7553F5B3; Wed, 10 Oct 2018 09:23:14 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 265EB1AE2D8E; Wed, 10 Oct 2018 17:23:13 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, sean.j.christopherson@intel.com, Will Deacon Subject: [PATCH v3 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Wed, 10 Oct 2018 17:23:04 +0100 Message-Id: <1539188584-15819-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1539188584-15819-1-git-send-email-will.deacon@arm.com> References: <1539188584-15819-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Reviewed-by: Toshi Kani Signed-off-by: Will Deacon --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0dcb3354d6dd..58776b90dd2a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1024,3 +1024,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index b4919c44a194..c6094997d060 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -779,6 +779,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 88ebc6102c7c..4297a2519ebf 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index 10d7c5485c39..063213685563 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;