From patchwork Thu Oct 29 16:18:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 311023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8ED9C2D0A3 for ; Thu, 29 Oct 2020 16:19:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 92AA7214F1 for ; Thu, 29 Oct 2020 16:19:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1603988385; bh=vBwurL4/Q9Hcp/7WaGQxkN1fGezdnq9akegXwjuyoXo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=IhpdzvXJHRIiyIfJmPKM5gWFZrjQTS9L53RcSLLiie7PEAPlKuvQy3xFoznOOJ63Z jCbc/del3k5GBP7xXUbT1mJP5ocTO4Z5Td82EFhst0RB+TQRIG9CxJ3LYl/v97SHgD DiDDsrD2c3xUgXyoY6wUDQaySf6ykF6YWpxJtVZI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726654AbgJ2QTl (ORCPT ); Thu, 29 Oct 2020 12:19:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:42294 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725764AbgJ2QTk (ORCPT ); Thu, 29 Oct 2020 12:19:40 -0400 Received: from aquarius.haifa.ibm.com (nesher1.haifa.il.ibm.com [195.110.40.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E196F20838; Thu, 29 Oct 2020 16:19:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1603988379; bh=vBwurL4/Q9Hcp/7WaGQxkN1fGezdnq9akegXwjuyoXo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iu1hGk5C+9KA0JuAZ9aELW2NgTQCTT7LskLXiHWtTRA4uwcYlzt218pXb7cTVqj74 KFZPwNQijSQWol2FX9/4R0GtrYmSusx0ENyOCULN4uaa8YmjU2rVFqr972RKvBVtH4 Js2pTF9s8Furil7Hm6afgBC8x7fq0sZqHqhgpjgc= From: Mike Rapoport To: Andrew Morton Cc: Albert Ou , Andy Lutomirski , Benjamin Herrenschmidt , Borislav Petkov , Catalin Marinas , Christian Borntraeger , Christoph Lameter , "David S. Miller" , Dave Hansen , David Hildenbrand , David Rientjes , "Edgecombe, Rick P" , "H. Peter Anvin" , Heiko Carstens , Ingo Molnar , Joonsoo Kim , "Kirill A. Shutemov" , Len Brown , Michael Ellerman , Mike Rapoport , Mike Rapoport , Palmer Dabbelt , Paul Mackerras , Paul Walmsley , Pavel Machek , Pekka Enberg , Peter Zijlstra , "Rafael J. Wysocki" , Thomas Gleixner , Vasily Gorbik , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pm@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v2 1/4] mm: introduce debug_pagealloc_map_pages() helper Date: Thu, 29 Oct 2020 18:18:59 +0200 Message-Id: <20201029161902.19272-2-rppt@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201029161902.19272-1-rppt@kernel.org> References: <20201029161902.19272-1-rppt@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Mike Rapoport When CONFIG_DEBUG_PAGEALLOC is enabled, it unmaps pages from the kernel direct mapping after free_pages(). The pages than need to be mapped back before they could be used. Theese mapping operations use __kernel_map_pages() guarded with with debug_pagealloc_enabled(). The only place that calls __kernel_map_pages() without checking whether DEBUG_PAGEALLOC is enabled is the hibernation code that presumes availability of this function when ARCH_HAS_SET_DIRECT_MAP is set. Still, on arm64, __kernel_map_pages() will bail out when DEBUG_PAGEALLOC is not enabled but set_direct_map_invalid_noflush() may render some pages not present in the direct map and hibernation code won't be able to save such pages. To make page allocation debugging and hibernation interaction more robust, the dependency on DEBUG_PAGEALLOC or ARCH_HAS_SET_DIRECT_MAP has to be made more explicit. Start with combining the guard condition and the call to __kernel_map_pages() into a single debug_pagealloc_map_pages() function to emphasize that __kernel_map_pages() should not be called without DEBUG_PAGEALLOC and use this new function to map/unmap pages when page allocation debug is enabled. Signed-off-by: Mike Rapoport Reviewed-by: David Hildenbrand --- include/linux/mm.h | 10 ++++++++++ mm/memory_hotplug.c | 3 +-- mm/page_alloc.c | 6 ++---- mm/slab.c | 8 +++----- 4 files changed, 16 insertions(+), 11 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ef360fe70aaf..1fc0609056dc 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2936,12 +2936,22 @@ kernel_map_pages(struct page *page, int numpages, int enable) { __kernel_map_pages(page, numpages, enable); } + +static inline void debug_pagealloc_map_pages(struct page *page, + int numpages, int enable) +{ + if (debug_pagealloc_enabled_static()) + __kernel_map_pages(page, numpages, enable); +} + #ifdef CONFIG_HIBERNATION extern bool kernel_page_present(struct page *page); #endif /* CONFIG_HIBERNATION */ #else /* CONFIG_DEBUG_PAGEALLOC || CONFIG_ARCH_HAS_SET_DIRECT_MAP */ static inline void kernel_map_pages(struct page *page, int numpages, int enable) {} +static inline void debug_pagealloc_map_pages(struct page *page, + int numpages, int enable) {} #ifdef CONFIG_HIBERNATION static inline bool kernel_page_present(struct page *page) { return true; } #endif /* CONFIG_HIBERNATION */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index b44d4c7ba73b..e2b6043a4428 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -614,8 +614,7 @@ void generic_online_page(struct page *page, unsigned int order) * so we should map it first. This is better than introducing a special * case in page freeing fast path. */ - if (debug_pagealloc_enabled_static()) - kernel_map_pages(page, 1 << order, 1); + debug_pagealloc_map_pages(page, 1 << order, 1); __free_pages_core(page, order); totalram_pages_add(1UL << order); #ifdef CONFIG_HIGHMEM diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 23f5066bd4a5..9a66a1ff9193 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1272,8 +1272,7 @@ static __always_inline bool free_pages_prepare(struct page *page, */ arch_free_page(page, order); - if (debug_pagealloc_enabled_static()) - kernel_map_pages(page, 1 << order, 0); + debug_pagealloc_map_pages(page, 1 << order, 0); kasan_free_nondeferred_pages(page, order); @@ -2270,8 +2269,7 @@ inline void post_alloc_hook(struct page *page, unsigned int order, set_page_refcounted(page); arch_alloc_page(page, order); - if (debug_pagealloc_enabled_static()) - kernel_map_pages(page, 1 << order, 1); + debug_pagealloc_map_pages(page, 1 << order, 1); kasan_alloc_pages(page, order); kernel_poison_pages(page, 1 << order, 1); set_page_owner(page, order, gfp_flags); diff --git a/mm/slab.c b/mm/slab.c index b1113561b98b..340db0ce74c4 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1431,10 +1431,8 @@ static bool is_debug_pagealloc_cache(struct kmem_cache *cachep) #ifdef CONFIG_DEBUG_PAGEALLOC static void slab_kernel_map(struct kmem_cache *cachep, void *objp, int map) { - if (!is_debug_pagealloc_cache(cachep)) - return; - - kernel_map_pages(virt_to_page(objp), cachep->size / PAGE_SIZE, map); + debug_pagealloc_map_pages(virt_to_page(objp), + cachep->size / PAGE_SIZE, map); } #else @@ -2062,7 +2060,7 @@ int __kmem_cache_create(struct kmem_cache *cachep, slab_flags_t flags) #if DEBUG /* - * If we're going to use the generic kernel_map_pages() + * If we're going to use the generic debug_pagealloc_map_pages() * poisoning, then it's going to smash the contents of * the redzone and userword anyhow, so switch them off. */