From patchwork Sat Sep 19 04:20:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 309517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79358C43464 for ; Sat, 19 Sep 2020 04:20:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 44BCE2344C for ; Sat, 19 Sep 2020 04:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489205; bh=/967+0nd3QfFE67e6Yxm69sBtWBBFlo25Sam1ALVV4I=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=X9eb5sspj9Bvsm28P2QN1Ma7/RV/jvUdVR409WjyhaU9bKM1sp1lFz0FUb94MBvJn ohs7lrhlfNgO/XI6QSBfdUnOMRBwIacQqPfmuKuZj2hVyTF13c3U29ycJVuofxeN5H qoY3/RL/W0amkSQfRVy1qYssKKTvZ5suhSkvD4NM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726152AbgISEUE (ORCPT ); Sat, 19 Sep 2020 00:20:04 -0400 Received: from mail.kernel.org ([198.145.29.99]:54360 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEUE (ORCPT ); Sat, 19 Sep 2020 00:20:04 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B7902235FA; Sat, 19 Sep 2020 04:20:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489204; bh=/967+0nd3QfFE67e6Yxm69sBtWBBFlo25Sam1ALVV4I=; h=Date:From:To:Subject:In-Reply-To:From; b=KoADUO6ASU2e5+r6O8uV1RfC5GKpAEjYkJLRtzfT0Pj3rBq9vosSnCJZrhaDYBFXJ +bVsqMVNLoHRpvKGfCkfflrNoGX8Uz/sDC2AEjb/JOwhDsVgksTi3peN0YCVPBEQpj hDv4sy6d0/z3GrZ2FGf1bAEzRh3HvY530wM4TPz0= Date: Fri, 18 Sep 2020 21:20:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, cai@lca.pw, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, shakeelb@google.com, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 02/15] ksm: reinstate memcg charge on copied pages Message-ID: <20200919042003.LTbtjVRr7%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins Subject: ksm: reinstate memcg charge on copied pages Patch series "mm: fixes to past from future testing". Here's a set of independent fixes against 5.9-rc2: prompted by testing Alex Shi's "warning on !memcg" and lru_lock series, but I think fit for 5.9 - though maybe only the first for stable. This patch (of 5): In 5.8 some instances of memcg charging in do_swap_page() and unuse_pte() were removed, on the understanding that swap cache is now already charged at those points; but a case was missed, when ksm_might_need_to_copy() has decided it must allocate a substitute page: such pages were never charged. Fix it inside ksm_might_need_to_copy(). This was discovered by Alex Shi's prospective commit "mm/memcg: warning on !memcg after readahead page charged". But there is a another surprise: this also fixes some rarer uncharged PageAnon cases, when KSM is configured in, but has never been activated. ksm_might_need_to_copy()'s anon_vma->root and linear_page_index() check sometimes catches a case which would need to have been copied if KSM were turned on. Or that's my optimistic interpretation (of my own old code), but it leaves some doubt as to whether everything is working as intended there - might it hint at rare anon ptes which rmap cannot find? A question not easily answered: put in the fix for missed memcg charges. Link: http://lkml.kernel.org/r/alpine.LSU.2.11.2008301343270.5954@eggly.anvils Link: http://lkml.kernel.org/r/alpine.LSU.2.11.2008301358020.5954@eggly.anvils Fixes: 4c6355b25e8b ("mm: memcontrol: charge swapin pages on instantiation") Signed-off-by: Hugh Dickins Reviewed-by: Shakeel Butt Acked-by: Johannes Weiner Cc: Alex Shi Cc: Michal Hocko Cc: Mike Kravetz Cc; Matthew Wilcox Cc: Qian Cai Cc: [5.8] Signed-off-by: Andrew Morton --- mm/ksm.c | 4 ++++ 1 file changed, 4 insertions(+) --- a/mm/ksm.c~ksm-reinstate-memcg-charge-on-copied-pages +++ a/mm/ksm.c @@ -2586,6 +2586,10 @@ struct page *ksm_might_need_to_copy(stru return page; /* let do_swap_page report the error */ new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address); + if (new_page && mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL)) { + put_page(new_page); + new_page = NULL; + } if (new_page) { copy_user_highpage(new_page, page, address, vma); From patchwork Sat Sep 19 04:20:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 263779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB92FC43463 for ; Sat, 19 Sep 2020 04:20:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E59922211 for ; Sat, 19 Sep 2020 04:20:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489211; bh=41YxocmAsZxH5HSeesZDPmKWXspsdtu9QFhZ4cINIiI=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=Wt9SMuI9lLJHWbleePxNwoaXLYd2k17UmOcaTa8KV/wFUMNeFuiS94H45YGsxvQmN KUXlJpKZqBVHbyPZ5gK5n2b365OyLq7QzJcuUJj2Xf3/vthEEkGB0WR3mX7lvk4Q/b 2nBOdXPVvhsrzIannknaszAZo0Pf/BAkmdeFnhq4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726170AbgISEUL (ORCPT ); Sat, 19 Sep 2020 00:20:11 -0400 Received: from mail.kernel.org ([198.145.29.99]:54500 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEUK (ORCPT ); Sat, 19 Sep 2020 00:20:10 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D975520874; Sat, 19 Sep 2020 04:20:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489210; bh=41YxocmAsZxH5HSeesZDPmKWXspsdtu9QFhZ4cINIiI=; h=Date:From:To:Subject:In-Reply-To:From; b=f48y29IGXBGqYz4BRglywcvewToQmfymmgbkxC3jqTRpE/uxKcrWyo/piw/cW2m81 QPylw6B9hxJTYSrv5ctbn1WSaJQvbApzB7U/ELkhKn4My1aeKprRvH89aDSi8iaAAl Y9ryj4hmTE7sDcOJlmGGejKkWRwt5RIJ6Syr7veI= Date: Fri, 18 Sep 2020 21:20:09 -0700 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, cai@lca.pw, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, shakeelb@google.com, shy828301@gmail.com, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 04/15] shmem: shmem_writepage() split unlikely i915 THP Message-ID: <20200919042009.bomzxmrg7%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Hugh Dickins Subject: shmem: shmem_writepage() split unlikely i915 THP drivers/gpu/drm/i915/gem/i915_gem_shmem.c contains a shmem_writeback() which calls shmem_writepage() from a shrinker: that usually works well enough; but if /sys/kernel/mm/transparent_hugepage/shmem_enabled has been set to "force" (documented as "Force the huge option on for all - very useful for testing"), shmem_writepage() is surprised to be called with a huge page, and crashes on the VM_BUG_ON_PAGE(PageCompound) (I did not find out where the crash happens when CONFIG_DEBUG_VM is off). LRU page reclaim always splits the shmem huge page first: I'd prefer not to demand that of i915, so check and split compound in shmem_writepage(). Link: http://lkml.kernel.org/r/alpine.LSU.2.11.2008301401390.5954@eggly.anvils Fixes: 2d6692e642e7 ("drm/i915: Start writeback from the shrinker") Signed-off-by: Hugh Dickins Reviewed-by: Shakeel Butt Acked-by: Yang Shi Cc: Alex Shi Cc: Johannes Weiner Cc: Michal Hocko Cc: Mike Kravetz Cc: Qian Cai Cc: [5.3+] Signed-off-by: Andrew Morton --- mm/shmem.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) --- a/mm/shmem.c~shmem-shmem_writepage-split-unlikely-i915-thp +++ a/mm/shmem.c @@ -1362,7 +1362,15 @@ static int shmem_writepage(struct page * swp_entry_t swap; pgoff_t index; - VM_BUG_ON_PAGE(PageCompound(page), page); + /* + * If /sys/kernel/mm/transparent_hugepage/shmem_enabled is "force", + * then drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, + * and its shmem_writeback() needs them to be split when swapping. + */ + if (PageTransCompound(page)) + if (split_huge_page(page) < 0) + goto redirty; + BUG_ON(!PageLocked(page)); mapping = page->mapping; index = page->index; From patchwork Sat Sep 19 04:20:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 309516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 943B7C43464 for ; Sat, 19 Sep 2020 04:20:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 628D82344C for ; Sat, 19 Sep 2020 04:20:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489223; bh=T16eJfOJMoalDgg9KcTh1y7MMH+8BiJwclfD9iF64J8=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=cKj4vSSb5pxp7pQeQmRGD+S0B9CQ8hG+03F8ML4aOykuYaMxWUNW0XJZULeJrGTP5 6DKI4/GM4JBBk7DIAhV9V7+OKRVNFIXa9Yjzlmv4PXQkFoK4clZIcqsMv50fpcFukC P5wEMAMh28CBaxPMrt5S/g5oC5R5YaqwGPlf10C4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726201AbgISEUW (ORCPT ); Sat, 19 Sep 2020 00:20:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:54786 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEUW (ORCPT ); Sat, 19 Sep 2020 00:20:22 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0BA2220874; Sat, 19 Sep 2020 04:20:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489222; bh=T16eJfOJMoalDgg9KcTh1y7MMH+8BiJwclfD9iF64J8=; h=Date:From:To:Subject:In-Reply-To:From; b=EggG7EyVJfkMA1ZJBnajXUYRSiukAvNJrve9kosRdMd+ZkF2mkKUWSDMhgNj2VRf1 jedAosI1ey2E1AxpjYh2+p14mj4dFEtXC4dcIl9K34aEStz+AX6+Wm9MuSWxrCtc4Y FQz/GZbGplrzD2MnzP8XzMeTKti0eUqv49zlXEZ0= Date: Fri, 18 Sep 2020 21:20:21 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anil.s.keshavamurthy@intel.com, davem@davemloft.net, linux-mm@kvack.org, mhiramat@kernel.org, mm-commits@vger.kernel.org, naveen.n.rao@linux.ibm.com, rostedt@goodmis.org, songliubraving@fb.com, songmuchun@bytedance.com, stable@vger.kernel.org, torvalds@linux-foundation.org, zhouchengming@bytedance.com Subject: [patch 08/15] kprobes: fix kill kprobe which has been marked as gone Message-ID: <20200919042021.X10NZatlE%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Muchun Song Subject: kprobes: fix kill kprobe which has been marked as gone If a kprobe is marked as gone, we should not kill it again. Otherwise, we can disarm the kprobe more than once. In that case, the statistics of kprobe_ftrace_enabled can unbalance which can lead to that kprobe do not work. Link: https://lkml.kernel.org/r/20200822030055.32383-1-songmuchun@bytedance.com Fixes: e8386a0cb22f ("kprobes: support probing module __exit function") Signed-off-by: Muchun Song Co-developed-by: Chengming Zhou Signed-off-by: Chengming Zhou Acked-by: Masami Hiramatsu Cc: "Naveen N . Rao" Cc: Anil S Keshavamurthy Cc: David S. Miller Cc: Song Liu Cc: Steven Rostedt Cc: Signed-off-by: Andrew Morton --- kernel/kprobes.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) --- a/kernel/kprobes.c~kprobes-fix-kill-kprobe-which-has-been-marked-as-gone +++ a/kernel/kprobes.c @@ -2140,6 +2140,9 @@ static void kill_kprobe(struct kprobe *p lockdep_assert_held(&kprobe_mutex); + if (WARN_ON_ONCE(kprobe_gone(p))) + return; + p->flags |= KPROBE_FLAG_GONE; if (kprobe_aggrprobe(p)) { /* @@ -2419,7 +2422,10 @@ static int kprobes_module_callback(struc mutex_lock(&kprobe_mutex); for (i = 0; i < KPROBE_TABLE_SIZE; i++) { head = &kprobe_table[i]; - hlist_for_each_entry(p, head, hlist) + hlist_for_each_entry(p, head, hlist) { + if (kprobe_gone(p)) + continue; + if (within_module_init((unsigned long)p->addr, mod) || (checkcore && within_module_core((unsigned long)p->addr, mod))) { @@ -2436,6 +2442,7 @@ static int kprobes_module_callback(struc */ kill_kprobe(p); } + } } if (val == MODULE_STATE_GOING) remove_module_kprobe_blacklist(mod); From patchwork Sat Sep 19 04:20:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 263778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDBB1C43463 for ; Sat, 19 Sep 2020 04:20:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0951235FC for ; Sat, 19 Sep 2020 04:20:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489226; bh=bbhbkMAaiTaRX+eyq+UtfFxMb5sRulwDpnswRN9D/44=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=nwRcNMb1QPTEJKY/pX/1b9VaA8dllb/Pk+H7/cm3BcJwdmzKOlxcZjZraXLN+OdDQ vQSNULJ8jJCNEIjYO7HWYDnCD6l8m7uXudW+y2VpB7eseVATYAow0y7bedTgwveHvk +aVB+69jqXUGwSgiNOo0YFGLI8Q5hmZ9p2Lly8wg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726219AbgISEU0 (ORCPT ); Sat, 19 Sep 2020 00:20:26 -0400 Received: from mail.kernel.org ([198.145.29.99]:54878 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEU0 (ORCPT ); Sat, 19 Sep 2020 00:20:26 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4312721D43; Sat, 19 Sep 2020 04:20:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489225; bh=bbhbkMAaiTaRX+eyq+UtfFxMb5sRulwDpnswRN9D/44=; h=Date:From:To:Subject:In-Reply-To:From; b=0ipoy1tNSP6JHZdfQaQy0SvIOVEfcwLyfY5DY75mGeBhSDZ/CAF2uV+xs7deMPEGU w2cwvhfrAiTCDBH8wUlgYsF6J6rVbqwzGlWywJ2t61SngP4InoEPoaR0re1jPgLqqH bGDWJ2yt0ioU65l5zJfEYZFAOLFa7+byo3etaL8s= Date: Fri, 18 Sep 2020 21:20:24 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bharata@linux.ibm.com, bskeggs@redhat.com, hch@lst.de, jgg@nvidia.com, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, shy828301@gmail.com, stable@vger.kernel.org, torvalds@linux-foundation.org, ziy@nvidia.com Subject: [patch 09/15] mm/thp: fix __split_huge_pmd_locked() for migration PMD Message-ID: <20200919042024.huPVYFOJa%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Ralph Campbell Subject: mm/thp: fix __split_huge_pmd_locked() for migration PMD A migrating transparent huge page has to already be unmapped. Otherwise, the page could be modified while it is being copied to a new page and data could be lost. The function __split_huge_pmd() checks for a PMD migration entry before calling __split_huge_pmd_locked() leading one to think that __split_huge_pmd_locked() can handle splitting a migrating PMD. However, the code always increments the page->_mapcount and adjusts the memory control group accounting assuming the page is mapped. Also, if the PMD entry is a migration PMD entry, the call to is_huge_zero_pmd(*pmd) is incorrect because it calls pmd_pfn(pmd) instead of migration_entry_to_pfn(pmd_to_swp_entry(pmd)). Fix these problems by checking for a PMD migration entry. Link: https://lkml.kernel.org/r/20200903183140.19055-1-rcampbell@nvidia.com Fixes: 84c3fc4e9c56 ("mm: thp: check pmd migration entry in common path") Signed-off-by: Ralph Campbell Reviewed-by: Yang Shi Reviewed-by: Zi Yan Cc: Jerome Glisse Cc: John Hubbard Cc: Alistair Popple Cc: Christoph Hellwig Cc: Jason Gunthorpe Cc: Bharata B Rao Cc: Ben Skeggs Cc: Shuah Khan Cc: [4.14+] Signed-off-by: Andrew Morton --- mm/huge_memory.c | 42 +++++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 19 deletions(-) --- a/mm/huge_memory.c~mm-thp-fix-__split_huge_pmd_locked-for-migration-pmd +++ a/mm/huge_memory.c @@ -2022,7 +2022,7 @@ static void __split_huge_pmd_locked(stru put_page(page); add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); return; - } else if (is_huge_zero_pmd(*pmd)) { + } else if (pmd_trans_huge(*pmd) && is_huge_zero_pmd(*pmd)) { /* * FIXME: Do we want to invalidate secondary mmu by calling * mmu_notifier_invalidate_range() see comments below inside @@ -2116,30 +2116,34 @@ static void __split_huge_pmd_locked(stru pte = pte_offset_map(&_pmd, addr); BUG_ON(!pte_none(*pte)); set_pte_at(mm, addr, pte, entry); - atomic_inc(&page[i]._mapcount); - pte_unmap(pte); - } - - /* - * Set PG_double_map before dropping compound_mapcount to avoid - * false-negative page_mapped(). - */ - if (compound_mapcount(page) > 1 && !TestSetPageDoubleMap(page)) { - for (i = 0; i < HPAGE_PMD_NR; i++) + if (!pmd_migration) atomic_inc(&page[i]._mapcount); + pte_unmap(pte); } - lock_page_memcg(page); - if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { - /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); - if (TestClearPageDoubleMap(page)) { - /* No need in mapcount reference anymore */ + if (!pmd_migration) { + /* + * Set PG_double_map before dropping compound_mapcount to avoid + * false-negative page_mapped(). + */ + if (compound_mapcount(page) > 1 && + !TestSetPageDoubleMap(page)) { for (i = 0; i < HPAGE_PMD_NR; i++) - atomic_dec(&page[i]._mapcount); + atomic_inc(&page[i]._mapcount); + } + + lock_page_memcg(page); + if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { + /* Last compound_mapcount is gone. */ + __dec_lruvec_page_state(page, NR_ANON_THPS); + if (TestClearPageDoubleMap(page)) { + /* No need in mapcount reference anymore */ + for (i = 0; i < HPAGE_PMD_NR; i++) + atomic_dec(&page[i]._mapcount); + } } + unlock_page_memcg(page); } - unlock_page_memcg(page); smp_wmb(); /* make pte visible before pmd */ pmd_populate(mm, pmd, pgtable); From patchwork Sat Sep 19 04:20:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 309515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0DB3C43469 for ; Sat, 19 Sep 2020 04:20:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60725235FA for ; Sat, 19 Sep 2020 04:20:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489229; bh=uD6xAztbVsrxrxGE5nhe7BWP5qXoKvPMqGnZIlslD3o=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=tDB7Y7MZ6KVKdgHGKyFvZyJ16tQjz0BSnnbvHlA9qOcfutIJ8nrfqlEaDcXs3CEmz Wtp+fU5r9DhCX3BSmvd6z/2IphBEYbeR8o3lmTJuXPW2b4y1uEe1vuvd2zpxjac7Ah alk5oaN55tTI+/Y14MkElgnII58ixqkOGVyvx9FQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726222AbgISEU2 (ORCPT ); Sat, 19 Sep 2020 00:20:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:54966 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEU2 (ORCPT ); Sat, 19 Sep 2020 00:20:28 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5F48122208; Sat, 19 Sep 2020 04:20:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489228; bh=uD6xAztbVsrxrxGE5nhe7BWP5qXoKvPMqGnZIlslD3o=; h=Date:From:To:Subject:In-Reply-To:From; b=fTF4twVG61mzrdWwiLBMPDRNeGh4VU0UXQ7oQzZHdTnYYixVY2H3p4+byNy3+9XbV 92LAv96Ku38amK4nQiXjghYd/jqnvG0PMHx0w4bLS116tmX8oXh5mVMB3UR1ZwM+cv E/lM0HqwoUp2RB217XcVaRgox/YWM4/OLPebJK38= Date: Fri, 18 Sep 2020 21:20:28 -0700 From: Andrew Morton To: akpm@linux-foundation.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mm-commits@vger.kernel.org, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 10/15] selftests/vm: fix display of page size in map_hugetlb Message-ID: <20200919042028.rpQMuOsNE%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Christophe Leroy Subject: selftests/vm: fix display of page size in map_hugetlb The displayed size is in bytes while the text says it is in kB. Shift it by 10 to really display kBytes. Link: https://lkml.kernel.org/r/e27481224564a93d14106e750de31189deaa8bc8.1598861977.git.christophe.leroy@csgroup.eu Fixes: fa7b9a805c79 ("tools/selftest/vm: allow choosing mem size and page size in map_hugetlb") Signed-off-by: Christophe Leroy Cc: Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/map_hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/tools/testing/selftests/vm/map_hugetlb.c~selftests-vm-fix-display-of-page-size-in-map_hugetlb +++ a/tools/testing/selftests/vm/map_hugetlb.c @@ -83,7 +83,7 @@ int main(int argc, char **argv) } if (shift) - printf("%u kB hugepages\n", 1 << shift); + printf("%u kB hugepages\n", 1 << (shift - 10)); else printf("Default size hugepages\n"); printf("Mapping %lu Mbytes\n", (unsigned long)length >> 20); From patchwork Sat Sep 19 04:20:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 263777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EB21C43467 for ; Sat, 19 Sep 2020 04:20:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B02D235F8 for ; Sat, 19 Sep 2020 04:20:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489233; bh=IsciPojv2+hUVaVLTM3iG2V5aNsBJRVuhcx6O3Zlqb4=; h=Date:From:To:Subject:In-Reply-To:List-ID:From; b=ZqqFB3TK1GwUS9q8IV5JsBLp0QFtRJASrujGPCyfoWv22tPJYq2DGLX7aN0YvE7QZ TqmvfUkz4jww8TnKsIB/v9g01JAWfnLQVp+gkGq1EXP5GeQZ9KCn10/GacEZ8saJ+K oa9pUfMNNOKGD5CN2/jZWDiiuIB1p8pSGH63n1bw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726249AbgISEUc (ORCPT ); Sat, 19 Sep 2020 00:20:32 -0400 Received: from mail.kernel.org ([198.145.29.99]:55052 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726009AbgISEUc (ORCPT ); Sat, 19 Sep 2020 00:20:32 -0400 Received: from localhost.localdomain (c-71-198-47-131.hsd1.ca.comcast.net [71.198.47.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 60A0E21D43; Sat, 19 Sep 2020 04:20:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600489231; bh=IsciPojv2+hUVaVLTM3iG2V5aNsBJRVuhcx6O3Zlqb4=; h=Date:From:To:Subject:In-Reply-To:From; b=MS7RHm7OEKbcYwFHjMHeJNRK/fMtJkcTsTZb1qTpZMX/Bej7Kqt0xAsNEfJnGqHfr 7MOrFC7eN1N3J3ou/2+CNApAvrqkARe/90F5/mD9yXNO2ROvByO1AA2Xf8koVdlflb cPV9iTUFwxqtGlt97mSJ4pGf+hzgRc+Iemfi4p8c= Date: Fri, 18 Sep 2020 21:20:31 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, richard.weiyang@gmail.com, rientjes@google.com, stable@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 11/15] mm/memory_hotplug: drain per-cpu pages again during memory offline Message-ID: <20200919042031.EJlTFXy4s%akpm@linux-foundation.org> In-Reply-To: <20200918211925.7e97f0ef63d92f5cfe5ccbc5@linux-foundation.org> User-Agent: s-nail v14.8.16 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Pavel Tatashin Subject: mm/memory_hotplug: drain per-cpu pages again during memory offline There is a race during page offline that can lead to infinite loop: a page never ends up on a buddy list and __offline_pages() keeps retrying infinitely or until a termination signal is received. Thread#1 - a new process: load_elf_binary begin_new_exec exec_mmap mmput exit_mmap tlb_finish_mmu tlb_flush_mmu release_pages free_unref_page_list free_unref_page_prepare set_pcppage_migratetype(page, migratetype); // Set page->index migration type below MIGRATE_PCPTYPES Thread#2 - hot-removes memory __offline_pages start_isolate_page_range set_migratetype_isolate set_pageblock_migratetype(page, MIGRATE_ISOLATE); Set migration type to MIGRATE_ISOLATE-> set drain_all_pages(zone); // drain per-cpu page lists to buddy allocator. Thread#1 - continue free_unref_page_commit migratetype = get_pcppage_migratetype(page); // get old migration type list_add(&page->lru, &pcp->lists[migratetype]); // add new page to already drained pcp list Thread#2 Never drains pcp again, and therefore gets stuck in the loop. The fix is to try to drain per-cpu lists again after check_pages_isolated_cb() fails. Link: https://lkml.kernel.org/r/20200903140032.380431-1-pasha.tatashin@soleen.com Link: https://lkml.kernel.org/r/20200904151448.100489-2-pasha.tatashin@soleen.com Link: http://lkml.kernel.org/r/20200904070235.GA15277@dhcp22.suse.cz Fixes: c52e75935f8d ("mm: remove extra drain pages on pcp list") Signed-off-by: Pavel Tatashin Acked-by: David Rientjes Acked-by: Vlastimil Babka Acked-by: Michal Hocko Acked-by: David Hildenbrand Cc: Oscar Salvador Cc: Wei Yang Cc: Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 14 ++++++++++++++ mm/page_isolation.c | 8 ++++++++ 2 files changed, 22 insertions(+) --- a/mm/memory_hotplug.c~mm-memory_hotplug-drain-per-cpu-pages-again-during-memory-offline +++ a/mm/memory_hotplug.c @@ -1575,6 +1575,20 @@ static int __ref __offline_pages(unsigne /* check again */ ret = walk_system_ram_range(start_pfn, end_pfn - start_pfn, NULL, check_pages_isolated_cb); + /* + * per-cpu pages are drained in start_isolate_page_range, but if + * there are still pages that are not free, make sure that we + * drain again, because when we isolated range we might + * have raced with another thread that was adding pages to pcp + * list. + * + * Forward progress should be still guaranteed because + * pages on the pcp list can only belong to MOVABLE_ZONE + * because has_unmovable_pages explicitly checks for + * PageBuddy on freed pages on other zones. + */ + if (ret) + drain_all_pages(zone); } while (ret); /* Ok, all of our target is isolated. --- a/mm/page_isolation.c~mm-memory_hotplug-drain-per-cpu-pages-again-during-memory-offline +++ a/mm/page_isolation.c @@ -170,6 +170,14 @@ __first_valid_page(unsigned long pfn, un * pageblocks we may have modified and return -EBUSY to caller. This * prevents two threads from simultaneously working on overlapping ranges. * + * Please note that there is no strong synchronization with the page allocator + * either. Pages might be freed while their page blocks are marked ISOLATED. + * In some cases pages might still end up on pcp lists and that would allow + * for their allocation even when they are in fact isolated already. Depending + * on how strong of a guarantee the caller needs drain_all_pages might be needed + * (e.g. __offline_pages will need to call it after check for isolated range for + * a next retry). + * * Return: the number of isolated pageblocks on success and -EBUSY if any part * of range cannot be isolated. */