From patchwork Mon Feb 1 10:01:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374219 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1118861jah; Mon, 1 Feb 2021 02:02:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJxjvK0dfUMO8zl8BjQfGwql/v0ujFRJVkjjA2JiADYvwbufLDSY5uNScI9VGNsNJkOzCKWy X-Received: by 2002:a17:906:d189:: with SMTP id c9mr17414842ejz.36.1612173773528; Mon, 01 Feb 2021 02:02:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173773; cv=none; d=google.com; s=arc-20160816; b=ZWsO2ZkkGLcAdrPHLhcYuBEDqHpQFRojEwXUbTatEE6BTDA8I9iiZpvkTv0R123tE7 jIR4e8jkDFl50bYh53lXXzMi4rwgy0Uavm0o1SzpSip8DiD1ZGikC2ubNKsW3RvXUsk9 XxPKilif0ortaCnjGZJyL2e5l6bBHIU1DBpQnsFFGvpSgmNLCTzrcD7Xbn4ukbHEdQ7E eSA3m4LOiqAiZzpotDzuXoWJN7md6Dx48KbxAfVJJ0X/B8t8NxBfhX1FRaY4P416YdCw yEre/KofaGASUXEpd54LNYYyakGe33aN+GHzL1MHkd7WI7XlT0UyGc9QuLNQOY67BrXG 284Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=z8ARMO4rc5T6cHiYqoD+hfUOUvVqAYAZAdQL3TYDR3A=; b=KjNj2V0NEbnwg59vJcF0Kt2ZMqrjvgpRmJJnSBCHHaI2JAUqP1K7+gib6hXA0wfham mbynSE3dwyuUhMdhpsixEJYj+PpdirnH4CXt6D64uk/R1YTbS3ml1J8VRaq2bumjQXG/ SjkDqAbktGnjtuDgAs+dEK/wmSITciCPV1BAozV/tn8suQrbrGzbV8SCpqQRHlKvxuyb js8WMI28kWt/yXjAe7eJ2cPFddd7X3kXHfpRKk5wOJuCUVTU+WfA+uG7KJsHTgvUVFtW 7mJy9FRs4sN0CeSYWe/mERtZqFGL9o3g4EPYmOnsfcP6nDYPurvh9prR2Ftz2UPlBCPl qxIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SSl7ez0L; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y23si201010edv.312.2021.02.01.02.02.53; Mon, 01 Feb 2021 02:02:53 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SSl7ez0L; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232443AbhBAKCw (ORCPT + 13 others); Mon, 1 Feb 2021 05:02:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229558AbhBAKCw (ORCPT ); Mon, 1 Feb 2021 05:02:52 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9F13C061574 for ; Mon, 1 Feb 2021 02:02:11 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id f16so12064645wmq.5 for ; Mon, 01 Feb 2021 02:02:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z8ARMO4rc5T6cHiYqoD+hfUOUvVqAYAZAdQL3TYDR3A=; b=SSl7ez0L8qQDzHhF5mrJTf+NLIZl+atJJqJSg/oHrw+RCMgkKGQNyoYOi2Jy/tq1R8 KydFuKgGDPYLiWHBS5lFLgvr0BVybM55C0+3lWkk/5fSCoSoCOeK6UFCcLDKOC0SFx1k FUiNzQIr7nFeUL0Xc7vo5Vg0gVKt2Q+yS2mtAM35JdhT8YAuUXsGwY7925d/FCPtXRtl 6YbBUSsOXmVhhBJefdalPX9HgLLkSlpAS0VswOOvvRKZjxBnUZ5QFCbWW5n10GC0yc6s e3NPeOX7QL2oPtR9Sc8Z37joSpXgcKfnRPG0WonB6dUudRWjspnyz97se18S/EP0rc3q 2bsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z8ARMO4rc5T6cHiYqoD+hfUOUvVqAYAZAdQL3TYDR3A=; b=SQrPw5rDyZBD9cEuo8i/fOF3mnUr5yPI84k+I5VU1HsREOpNjPqFNRLxbwSqzrY7Mj 2ovgtxuBx4KGzR6FyGDEwWm/wK33wZFnD/0blv0+aNtuqQHVSRZKGm8uE6XWU6xs2m17 Q+pwARqaGRiH3D/sgYuwMfr4hDuK0vD/n/hT2AHfHWTeWhvlSiPaolIqjUHKyp5nYH/r q2DMDmnqqSw15VqTb1/i4L2q2+NsmHkgCidBF8UIwHdwrQVDzDDxIBONxlCf6oU6dn+2 9joldFUkh0h+PwN8/Jpi4StHExflgNcQ/+DmiupfC9MH2nP5RIkJO1mG27xvmH6tAQGD xOEA== X-Gm-Message-State: AOAM533UFkx4TuN7qJvwHEr9N66XDeLgQkQHP1Bk1Z4oKyFosldhvKLR de+TdGvQQjitHdavFqt97+OQ3we+iRZHUp0T X-Received: by 2002:a1c:4483:: with SMTP id r125mr13773702wma.80.1612173729685; Mon, 01 Feb 2021 02:02:09 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:08 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Arnd Bergmann , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 01/12] y2038: futex: Move compat implementation into futex.c Date: Mon, 1 Feb 2021 10:01:32 +0000 Message-Id: <20210201100143.2028618-2-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Arnd Bergmann commit 04e7712f4460585e5eed5b853fd8b82a9943958f upstream. We are going to share the compat_sys_futex() handler between 64-bit architectures and 32-bit architectures that need to deal with both 32-bit and 64-bit time_t, and this is easier if both entry points are in the same file. In fact, most other system call handlers do the same thing these days, so let's follow the trend here and merge all of futex_compat.c into futex.c. In the process, a few minor changes have to be done to make sure everything still makes sense: handle_futex_death() and futex_cmpxchg_enabled() become local symbol, and the compat version of the fetch_robust_entry() function gets renamed to compat_fetch_robust_entry() to avoid a symbol clash. This is intended as a purely cosmetic patch, no behavior should change. Signed-off-by: Arnd Bergmann Signed-off-by: Greg Kroah-Hartman [Lee: Back-ported to satisfy a build dependency] Signed-off-by: Lee Jones --- include/linux/futex.h | 8 -- kernel/Makefile | 3 - kernel/futex.c | 195 +++++++++++++++++++++++++++++++++++++++- kernel/futex_compat.c | 201 ------------------------------------------ 4 files changed, 192 insertions(+), 215 deletions(-) delete mode 100644 kernel/futex_compat.c -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index c015fa91e7cce..fb4e12cbe887e 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -11,9 +11,6 @@ union ktime; long do_futex(u32 __user *uaddr, int op, u32 val, union ktime *timeout, u32 __user *uaddr2, u32 val2, u32 val3); -extern int -handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi); - /* * Futexes are matched on equal values of this key. * The key type depends on whether it's a shared or private mapping. @@ -58,11 +55,6 @@ union futex_key { #ifdef CONFIG_FUTEX extern void exit_robust_list(struct task_struct *curr); extern void exit_pi_state_list(struct task_struct *curr); -#ifdef CONFIG_HAVE_FUTEX_CMPXCHG -#define futex_cmpxchg_enabled 1 -#else -extern int futex_cmpxchg_enabled; -#endif #else static inline void exit_robust_list(struct task_struct *curr) { diff --git a/kernel/Makefile b/kernel/Makefile index 184fa9aa58027..92488cf6ad913 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -47,9 +47,6 @@ obj-$(CONFIG_PROFILING) += profile.o obj-$(CONFIG_STACKTRACE) += stacktrace.o obj-y += time/ obj-$(CONFIG_FUTEX) += futex.o -ifeq ($(CONFIG_COMPAT),y) -obj-$(CONFIG_FUTEX) += futex_compat.o -endif obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o obj-$(CONFIG_SMP) += smp.o ifneq ($(CONFIG_SMP),y) diff --git a/kernel/futex.c b/kernel/futex.c index 7123d9cab4568..220e3924869b0 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -44,6 +44,7 @@ * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +#include #include #include #include @@ -171,8 +172,10 @@ * double_lock_hb() and double_unlock_hb(), respectively. */ -#ifndef CONFIG_HAVE_FUTEX_CMPXCHG -int __read_mostly futex_cmpxchg_enabled; +#ifdef CONFIG_HAVE_FUTEX_CMPXCHG +#define futex_cmpxchg_enabled 1 +#else +static int __read_mostly futex_cmpxchg_enabled; #endif /* @@ -3123,7 +3126,7 @@ SYSCALL_DEFINE3(get_robust_list, int, pid, * Process a futex-list entry, check whether it's owned by the * dying task, and do notification if so: */ -int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi) +static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi) { u32 uval, uninitialized_var(nval), mval; @@ -3354,6 +3357,192 @@ SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); } +#ifdef CONFIG_COMPAT +/* + * Fetch a robust-list pointer. Bit 0 signals PI futexes: + */ +static inline int +compat_fetch_robust_entry(compat_uptr_t *uentry, struct robust_list __user **entry, + compat_uptr_t __user *head, unsigned int *pi) +{ + if (get_user(*uentry, head)) + return -EFAULT; + + *entry = compat_ptr((*uentry) & ~1); + *pi = (unsigned int)(*uentry) & 1; + + return 0; +} + +static void __user *futex_uaddr(struct robust_list __user *entry, + compat_long_t futex_offset) +{ + compat_uptr_t base = ptr_to_compat(entry); + void __user *uaddr = compat_ptr(base + futex_offset); + + return uaddr; +} + +/* + * Walk curr->robust_list (very carefully, it's a userspace list!) + * and mark any locks found there dead, and notify any waiters. + * + * We silently return on any sign of list-walking problem. + */ +void compat_exit_robust_list(struct task_struct *curr) +{ + struct compat_robust_list_head __user *head = curr->compat_robust_list; + struct robust_list __user *entry, *next_entry, *pending; + unsigned int limit = ROBUST_LIST_LIMIT, pi, pip; + unsigned int uninitialized_var(next_pi); + compat_uptr_t uentry, next_uentry, upending; + compat_long_t futex_offset; + int rc; + + if (!futex_cmpxchg_enabled) + return; + + /* + * Fetch the list head (which was registered earlier, via + * sys_set_robust_list()): + */ + if (compat_fetch_robust_entry(&uentry, &entry, &head->list.next, &pi)) + return; + /* + * Fetch the relative futex offset: + */ + if (get_user(futex_offset, &head->futex_offset)) + return; + /* + * Fetch any possibly pending lock-add first, and handle it + * if it exists: + */ + if (compat_fetch_robust_entry(&upending, &pending, + &head->list_op_pending, &pip)) + return; + + next_entry = NULL; /* avoid warning with gcc */ + while (entry != (struct robust_list __user *) &head->list) { + /* + * Fetch the next entry in the list before calling + * handle_futex_death: + */ + rc = compat_fetch_robust_entry(&next_uentry, &next_entry, + (compat_uptr_t __user *)&entry->next, &next_pi); + /* + * A pending lock might already be on the list, so + * dont process it twice: + */ + if (entry != pending) { + void __user *uaddr = futex_uaddr(entry, futex_offset); + + if (handle_futex_death(uaddr, curr, pi)) + return; + } + if (rc) + return; + uentry = next_uentry; + entry = next_entry; + pi = next_pi; + /* + * Avoid excessively long or circular lists: + */ + if (!--limit) + break; + + cond_resched(); + } + if (pending) { + void __user *uaddr = futex_uaddr(pending, futex_offset); + + handle_futex_death(uaddr, curr, pip); + } +} + +COMPAT_SYSCALL_DEFINE2(set_robust_list, + struct compat_robust_list_head __user *, head, + compat_size_t, len) +{ + if (!futex_cmpxchg_enabled) + return -ENOSYS; + + if (unlikely(len != sizeof(*head))) + return -EINVAL; + + current->compat_robust_list = head; + + return 0; +} + +COMPAT_SYSCALL_DEFINE3(get_robust_list, int, pid, + compat_uptr_t __user *, head_ptr, + compat_size_t __user *, len_ptr) +{ + struct compat_robust_list_head __user *head; + unsigned long ret; + struct task_struct *p; + + if (!futex_cmpxchg_enabled) + return -ENOSYS; + + rcu_read_lock(); + + ret = -ESRCH; + if (!pid) + p = current; + else { + p = find_task_by_vpid(pid); + if (!p) + goto err_unlock; + } + + ret = -EPERM; + if (!ptrace_may_access(p, PTRACE_MODE_READ_REALCREDS)) + goto err_unlock; + + head = p->compat_robust_list; + rcu_read_unlock(); + + if (put_user(sizeof(*head), len_ptr)) + return -EFAULT; + return put_user(ptr_to_compat(head), head_ptr); + +err_unlock: + rcu_read_unlock(); + + return ret; +} + +COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, + struct compat_timespec __user *, utime, u32 __user *, uaddr2, + u32, val3) +{ + struct timespec ts; + ktime_t t, *tp = NULL; + int val2 = 0; + int cmd = op & FUTEX_CMD_MASK; + + if (utime && (cmd == FUTEX_WAIT || cmd == FUTEX_LOCK_PI || + cmd == FUTEX_WAIT_BITSET || + cmd == FUTEX_WAIT_REQUEUE_PI)) { + if (compat_get_timespec(&ts, utime)) + return -EFAULT; + if (!timespec_valid(&ts)) + return -EINVAL; + + t = timespec_to_ktime(ts); + if (cmd == FUTEX_WAIT) + t = ktime_add_safe(ktime_get(), t); + tp = &t; + } + if (cmd == FUTEX_REQUEUE || cmd == FUTEX_CMP_REQUEUE || + cmd == FUTEX_CMP_REQUEUE_PI || cmd == FUTEX_WAKE_OP) + val2 = (int) (unsigned long) utime; + + return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); +} +#endif /* CONFIG_COMPAT */ + static void __init futex_detect_cmpxchg(void) { #ifndef CONFIG_HAVE_FUTEX_CMPXCHG diff --git a/kernel/futex_compat.c b/kernel/futex_compat.c deleted file mode 100644 index 4ae3232e7a28a..0000000000000 --- a/kernel/futex_compat.c +++ /dev/null @@ -1,201 +0,0 @@ -/* - * linux/kernel/futex_compat.c - * - * Futex compatibililty routines. - * - * Copyright 2006, Red Hat, Inc., Ingo Molnar - */ - -#include -#include -#include -#include -#include -#include - -#include - - -/* - * Fetch a robust-list pointer. Bit 0 signals PI futexes: - */ -static inline int -fetch_robust_entry(compat_uptr_t *uentry, struct robust_list __user **entry, - compat_uptr_t __user *head, unsigned int *pi) -{ - if (get_user(*uentry, head)) - return -EFAULT; - - *entry = compat_ptr((*uentry) & ~1); - *pi = (unsigned int)(*uentry) & 1; - - return 0; -} - -static void __user *futex_uaddr(struct robust_list __user *entry, - compat_long_t futex_offset) -{ - compat_uptr_t base = ptr_to_compat(entry); - void __user *uaddr = compat_ptr(base + futex_offset); - - return uaddr; -} - -/* - * Walk curr->robust_list (very carefully, it's a userspace list!) - * and mark any locks found there dead, and notify any waiters. - * - * We silently return on any sign of list-walking problem. - */ -void compat_exit_robust_list(struct task_struct *curr) -{ - struct compat_robust_list_head __user *head = curr->compat_robust_list; - struct robust_list __user *entry, *next_entry, *pending; - unsigned int limit = ROBUST_LIST_LIMIT, pi, pip; - unsigned int uninitialized_var(next_pi); - compat_uptr_t uentry, next_uentry, upending; - compat_long_t futex_offset; - int rc; - - if (!futex_cmpxchg_enabled) - return; - - /* - * Fetch the list head (which was registered earlier, via - * sys_set_robust_list()): - */ - if (fetch_robust_entry(&uentry, &entry, &head->list.next, &pi)) - return; - /* - * Fetch the relative futex offset: - */ - if (get_user(futex_offset, &head->futex_offset)) - return; - /* - * Fetch any possibly pending lock-add first, and handle it - * if it exists: - */ - if (fetch_robust_entry(&upending, &pending, - &head->list_op_pending, &pip)) - return; - - next_entry = NULL; /* avoid warning with gcc */ - while (entry != (struct robust_list __user *) &head->list) { - /* - * Fetch the next entry in the list before calling - * handle_futex_death: - */ - rc = fetch_robust_entry(&next_uentry, &next_entry, - (compat_uptr_t __user *)&entry->next, &next_pi); - /* - * A pending lock might already be on the list, so - * dont process it twice: - */ - if (entry != pending) { - void __user *uaddr = futex_uaddr(entry, futex_offset); - - if (handle_futex_death(uaddr, curr, pi)) - return; - } - if (rc) - return; - uentry = next_uentry; - entry = next_entry; - pi = next_pi; - /* - * Avoid excessively long or circular lists: - */ - if (!--limit) - break; - - cond_resched(); - } - if (pending) { - void __user *uaddr = futex_uaddr(pending, futex_offset); - - handle_futex_death(uaddr, curr, pip); - } -} - -COMPAT_SYSCALL_DEFINE2(set_robust_list, - struct compat_robust_list_head __user *, head, - compat_size_t, len) -{ - if (!futex_cmpxchg_enabled) - return -ENOSYS; - - if (unlikely(len != sizeof(*head))) - return -EINVAL; - - current->compat_robust_list = head; - - return 0; -} - -COMPAT_SYSCALL_DEFINE3(get_robust_list, int, pid, - compat_uptr_t __user *, head_ptr, - compat_size_t __user *, len_ptr) -{ - struct compat_robust_list_head __user *head; - unsigned long ret; - struct task_struct *p; - - if (!futex_cmpxchg_enabled) - return -ENOSYS; - - rcu_read_lock(); - - ret = -ESRCH; - if (!pid) - p = current; - else { - p = find_task_by_vpid(pid); - if (!p) - goto err_unlock; - } - - ret = -EPERM; - if (!ptrace_may_access(p, PTRACE_MODE_READ_REALCREDS)) - goto err_unlock; - - head = p->compat_robust_list; - rcu_read_unlock(); - - if (put_user(sizeof(*head), len_ptr)) - return -EFAULT; - return put_user(ptr_to_compat(head), head_ptr); - -err_unlock: - rcu_read_unlock(); - - return ret; -} - -COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, - struct compat_timespec __user *, utime, u32 __user *, uaddr2, - u32, val3) -{ - struct timespec ts; - ktime_t t, *tp = NULL; - int val2 = 0; - int cmd = op & FUTEX_CMD_MASK; - - if (utime && (cmd == FUTEX_WAIT || cmd == FUTEX_LOCK_PI || - cmd == FUTEX_WAIT_BITSET || - cmd == FUTEX_WAIT_REQUEUE_PI)) { - if (compat_get_timespec(&ts, utime)) - return -EFAULT; - if (!timespec_valid(&ts)) - return -EINVAL; - - t = timespec_to_ktime(ts); - if (cmd == FUTEX_WAIT) - t = ktime_add_safe(ktime_get(), t); - tp = &t; - } - if (cmd == FUTEX_REQUEUE || cmd == FUTEX_CMP_REQUEUE || - cmd == FUTEX_CMP_REQUEUE_PI || cmd == FUTEX_WAKE_OP) - val2 = (int) (unsigned long) utime; - - return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); -} From patchwork Mon Feb 1 10:01:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374220 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1118882jah; Mon, 1 Feb 2021 02:02:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJwpS0C0WWTXlcOWI9001mvQo8b3KNkK3mbnbg9UZVFeE1Q7/4WvlNwNQEUvZkCKYAH3xCvV X-Received: by 2002:a17:906:4050:: with SMTP id y16mr16699015ejj.43.1612173775038; Mon, 01 Feb 2021 02:02:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173775; cv=none; d=google.com; s=arc-20160816; b=0ZK8ikFdqEQL5E4P6YodFOHUxn8yXPfin55vU2dO3rsIVHZIPEj0v09t8RdZK8PXyV Vbw03lxiynTovdRDlulO3EHYS87VnnAkVIv1ptHnaJhm8J3RedlG26qeHm8Gi06W6PWC 3JyeZsXJ4M8nbdjWBpc63fjZ+DOWgeHuIIDjK4MsqY30LSQ++2hIaFqDEbvkj/v6tGJK aNl8cWuwrcxCd4uP1+e8VPLhZvLYqMqii5MonSBmH7ktICSMVnh1RBJLMml5viEoJOJC 98zpnbKQj9DXF4nG50pP2UO1edJSge7q7lof4swku47uzJun6jMuqio+civyXhXlOIKU CNpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SbL/H5EXTKeLRGZKCiBhrlOPYKA29ODSWb0e+HweDuw=; b=tz+jonwlKKfJkxa48gqhuLpXZ854OGbfjgWEnXKcrs5Zi1K7cDKLZz3wXkDIRM3kB8 7R6H12f+eNJnyx6m+W1G2SG+h1TyRlh8RHnJKDwY42vVH96a5UUn+ISQ1aYV9KtQHdYg e1MK8NdJN6s5Bb5OEOhreV4878BUEzP8WwiNR/1ul7jt06xL5QK9tY9v9+LAy8bfM/p5 RVc6MKPaeoBXMggqM+xVcDb6rl8rtFqJYGlt1h3fer87HRPS0UWpBXKH0QbZfrH4j4nm GaYk6MlYDL9lQ+oL5KSp2QfikEgqXldeHtLy1qpUnu1Yr5cEgaUZm7/uJQhVFfk23fNl 26Vw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wh34svON; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y23si201010edv.312.2021.02.01.02.02.54; Mon, 01 Feb 2021 02:02:55 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wh34svON; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232651AbhBAKCx (ORCPT + 13 others); Mon, 1 Feb 2021 05:02:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229558AbhBAKCx (ORCPT ); Mon, 1 Feb 2021 05:02:53 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE9D5C06174A for ; Mon, 1 Feb 2021 02:02:12 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id s7so12881610wru.5 for ; Mon, 01 Feb 2021 02:02:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SbL/H5EXTKeLRGZKCiBhrlOPYKA29ODSWb0e+HweDuw=; b=Wh34svONTy1KOVuy3DGyqQKhkF9yQBmUQyyd8NH+XN7h8xzRup8KPn7yR74/YVVLnd F0neHaspZKOV8K1Rktc1fplGiRrLIKC9CW1FHm6l6+560oIKvyWJx9oV7dPbC3W/SbLy bDSwCBy8SCXD+5wzPEH5g1EBObavfs8sDy96ERu7KYjNgZy7a3OPJjQ0RNYMDl1wWxUt mamqus//u7jgzXgb+AVeR+BpZ+wTO3Va7TlJD1ACWTXBA4Y9A7Ra7qHYpaVAx4BvfF1b o2PI/b4yn4AydWWwiWzAjtdAudS0b2Vz92f+fpcVmNGgGOqFvwdte60Xg1KCOr6P9pTE VDeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SbL/H5EXTKeLRGZKCiBhrlOPYKA29ODSWb0e+HweDuw=; b=Bi1SMu2ZIhSYChH6IX/JiNVLeGF7dtjDyxoL343ghMBrFDBl3f6/2OK/vCFWtX1D+L t8ayF4Kx+DsHQliSGZlC0XGKUssvEIN+rNCkADb66l+FLo2waR3SRKdzaWNmhsbu1hC+ XP/YQOtiSr05oZBLZevovCCFJOx5sTg176zd+UjQ3ZGPsTnrJmL/UgPt0sBznyRwWux1 mjf3w+hZldkKlmsCjkakGIhrMYBsJxIjJNqBUev2Bvbjb42pYqpPtJPS2d8TTT9gNLCL Khgeuy4EGFgRPb3PW+87jZYaiEe9cdgP9PzRWyJ5SEGn1Vou+ICQ1s8nraywls7PQPVI UukA== X-Gm-Message-State: AOAM530MF8jOkziOCEztWr7IfKUDythcJQh8gSwb8f4IfbrrICW7FzfI p2Lrpad+bpLpHEHc4LUqWG2Bw+dP8z9SryhW X-Received: by 2002:adf:ce89:: with SMTP id r9mr17449541wrn.345.1612173731189; Mon, 01 Feb 2021 02:02:11 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:10 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 02/12] futex: Move futex exit handling into futex code Date: Mon, 1 Feb 2021 10:01:33 +0000 Message-Id: <20210201100143.2028618-3-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit ba31c1a48538992316cc71ce94fa9cd3e7b427c0 upstream. The futex exit handling is #ifdeffed into mm_release() which is not pretty to begin with. But upcoming changes to address futex exit races need to add more functionality to this exit code. Split it out into a function, move it into futex code and make the various futex exit functions static. Preparatory only and no functional change. Folded build fix from Borislav. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.049705556@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/compat.h | 2 -- include/linux/futex.h | 24 +++++++++++++++++------- kernel/fork.c | 25 +++---------------------- kernel/futex.c | 28 ++++++++++++++++++++++++++-- 4 files changed, 46 insertions(+), 33 deletions(-) -- 2.25.1 diff --git a/include/linux/compat.h b/include/linux/compat.h index fab35daf87596..6b9d38a7adcaf 100644 --- a/include/linux/compat.h +++ b/include/linux/compat.h @@ -311,8 +311,6 @@ struct compat_kexec_segment; struct compat_mq_attr; struct compat_msgbuf; -extern void compat_exit_robust_list(struct task_struct *curr); - asmlinkage long compat_sys_set_robust_list(struct compat_robust_list_head __user *head, compat_size_t len); diff --git a/include/linux/futex.h b/include/linux/futex.h index fb4e12cbe887e..63d353cedfcde 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -1,6 +1,8 @@ #ifndef _LINUX_FUTEX_H #define _LINUX_FUTEX_H +#include + #include struct inode; @@ -53,14 +55,22 @@ union futex_key { #define FUTEX_KEY_INIT (union futex_key) { .both = { .ptr = 0ULL } } #ifdef CONFIG_FUTEX -extern void exit_robust_list(struct task_struct *curr); -extern void exit_pi_state_list(struct task_struct *curr); -#else -static inline void exit_robust_list(struct task_struct *curr) -{ -} -static inline void exit_pi_state_list(struct task_struct *curr) +static inline void futex_init_task(struct task_struct *tsk) { + tsk->robust_list = NULL; +#ifdef CONFIG_COMPAT + tsk->compat_robust_list = NULL; +#endif + INIT_LIST_HEAD(&tsk->pi_state_list); + tsk->pi_state_cache = NULL; } + +void futex_mm_release(struct task_struct *tsk); + +long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, + u32 __user *uaddr2, u32 val2, u32 val3); +#else +static inline void futex_init_task(struct task_struct *tsk) { } +static inline void futex_mm_release(struct task_struct *tsk) { } #endif #endif diff --git a/kernel/fork.c b/kernel/fork.c index b64efec4a6e6e..000447bfcfde5 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1085,20 +1085,7 @@ static int wait_for_vfork_done(struct task_struct *child, void mm_release(struct task_struct *tsk, struct mm_struct *mm) { /* Get rid of any futexes when releasing the mm */ -#ifdef CONFIG_FUTEX - if (unlikely(tsk->robust_list)) { - exit_robust_list(tsk); - tsk->robust_list = NULL; - } -#ifdef CONFIG_COMPAT - if (unlikely(tsk->compat_robust_list)) { - compat_exit_robust_list(tsk); - tsk->compat_robust_list = NULL; - } -#endif - if (unlikely(!list_empty(&tsk->pi_state_list))) - exit_pi_state_list(tsk); -#endif + futex_mm_release(tsk); uprobe_free_utask(tsk); @@ -1706,14 +1693,8 @@ static __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_BLOCK p->plug = NULL; #endif -#ifdef CONFIG_FUTEX - p->robust_list = NULL; -#ifdef CONFIG_COMPAT - p->compat_robust_list = NULL; -#endif - INIT_LIST_HEAD(&p->pi_state_list); - p->pi_state_cache = NULL; -#endif + futex_init_task(p); + /* * sigaltstack should be cleared when sharing the same VM */ diff --git a/kernel/futex.c b/kernel/futex.c index 220e3924869b0..156b23f4b9aac 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -339,6 +339,12 @@ static inline bool should_fail_futex(bool fshared) } #endif /* CONFIG_FAIL_FUTEX */ +#ifdef CONFIG_COMPAT +static void compat_exit_robust_list(struct task_struct *curr); +#else +static inline void compat_exit_robust_list(struct task_struct *curr) { } +#endif + static inline void futex_get_mm(union futex_key *key) { atomic_inc(&key->private.mm->mm_count); @@ -894,7 +900,7 @@ static struct task_struct * futex_find_get_task(pid_t pid) * Kernel cleans up PI-state, but userspace is likely hosed. * (Robust-futex cleanup is separate and might save the day for userspace.) */ -void exit_pi_state_list(struct task_struct *curr) +static void exit_pi_state_list(struct task_struct *curr) { struct list_head *next, *head = &curr->pi_state_list; struct futex_pi_state *pi_state; @@ -3201,7 +3207,7 @@ static inline int fetch_robust_entry(struct robust_list __user **entry, * * We silently return on any sign of list-walking problem. */ -void exit_robust_list(struct task_struct *curr) +static void exit_robust_list(struct task_struct *curr) { struct robust_list_head __user *head = curr->robust_list; struct robust_list __user *entry, *next_entry, *pending; @@ -3264,6 +3270,24 @@ void exit_robust_list(struct task_struct *curr) curr, pip); } +void futex_mm_release(struct task_struct *tsk) +{ + if (unlikely(tsk->robust_list)) { + exit_robust_list(tsk); + tsk->robust_list = NULL; + } + +#ifdef CONFIG_COMPAT + if (unlikely(tsk->compat_robust_list)) { + compat_exit_robust_list(tsk); + tsk->compat_robust_list = NULL; + } +#endif + + if (unlikely(!list_empty(&tsk->pi_state_list))) + exit_pi_state_list(tsk); +} + long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3) { From patchwork Mon Feb 1 10:01:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374221 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1118903jah; Mon, 1 Feb 2021 02:02:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJx3tMMLAjN2MXXKibsZAWOz5ln1WR3GqZCWYm2OKOAhOLAiv5zLC+NMnphFq3puo7XbdWqi X-Received: by 2002:a05:6402:1682:: with SMTP id a2mr18322099edv.30.1612173777045; Mon, 01 Feb 2021 02:02:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173777; cv=none; d=google.com; s=arc-20160816; b=HmAjjsTKzPorUwdbAlL4XeAwsKLEGZOrOZ6tLxQTzbrGeEUvbutNS37bH3obhX+Znl jYv/rlETvTI6F3OF2RKyCcUvRoynkyG681ArXQ5RBQAQQaDGJ/rJPG3Aomgi3nA6Djsb 47KAPLz5XBVdf77WLaP4i8KlwA2+KM4Wh9Snhnc4iiMoqsYO3JGMCtlOJF6Zz46rXJC9 CLpgohBabQ6HDLL7lr069FhAS+D+WYkoI34k8PQvx5Kl1fxBillMJrOW50zUsJqI22cL DI3sFIqVzWheeFNJhNkRMZLU7XwYICcMh0nxXHIaYN2Qs4dhcW+Mjk89Qu/eY5CE0OUB QF0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zTLj8rZ658syZ+1sSn6qZ3dkC38f1lTVa0M8kGwEtcs=; b=EvJR4aoRQnhVpXM0kuiT7b2DNaZGkRdnk18vJ4N0kDugiQ2fVG4UkzPDt0HlPWC9F6 n3U9aHBlRmeARLV/TLnz+474JB3o552Wc9kHdvnt6N/z7qtmalRcxHAsj4aODOnrVh9E YlZlkYZECtwEtEOzu5fLOWWhkLV49z63+BwR3vQx5d8EG22K/nqgdcGeAhK249yG3gM/ Nou+HYy+DgKCZmFHa2Dw94ijdwZhzc7D2cOdVk6y1dPce3w5IkCHdRwgyV8ww2CmHPf/ JqVWqwKflGPuXV689kJJfeMeofSR5cDLVwStmzXoJbZ+IxeLFCsFT7rJzZp/pOQsX5hN Aqpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DNEty4bM; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y23si201010edv.312.2021.02.01.02.02.56; Mon, 01 Feb 2021 02:02:57 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DNEty4bM; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232862AbhBAKC4 (ORCPT + 13 others); Mon, 1 Feb 2021 05:02:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229558AbhBAKCz (ORCPT ); Mon, 1 Feb 2021 05:02:55 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DB55C061756 for ; Mon, 1 Feb 2021 02:02:14 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id m1so9335890wml.2 for ; Mon, 01 Feb 2021 02:02:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zTLj8rZ658syZ+1sSn6qZ3dkC38f1lTVa0M8kGwEtcs=; b=DNEty4bMcb7C8YuwlL3GTFizFq4+l1T4dFgKHwQCInBZLd/hZvZyxc9w+YuRKlmu2l ZTco1UGayiVHuQJIJMUMz8AC/bLyMvnnUk5tlQSkpeX+GFRiTeOBaJafIAQN8N12jdtG yn9kBRHv1+zSVyOC0P5D3N8BJkPKzddMC8jQYOFfZp+c0bxgxoI4CQQiuG3ctnnXS4yP NAv2UtCPZHN4UC2liRFOHhg6eIt0p83+WVi1JcNRmjTZjfW5Qwb06433540lknOKW5qk FPLWrPreeCYgBr0GRewBot4rKLI0yICvRJoN8IpvfCRNJUSOuMHwppG/dvkXJKHFoaac PQTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zTLj8rZ658syZ+1sSn6qZ3dkC38f1lTVa0M8kGwEtcs=; b=UOt0lAkDtIVvCvTPIkuqBpYdMCzq5ScwpgZ8Nm5oP8FiftsbP6jPJ5F7DRapBekBxH J+GXmGJ20ZiukhQ/HKbY91HGgGfKznXXkUtRzmjdIpc1W+oYhKhaGsSHeCQMhqlRZIur O7y9xH+RSmloT7VqObcvMi+aNsDveWIillnA1ykKjOkMTTyCq5rDvQ/3qXQfZlRgYPWs K9MOC5/2xVdRNmYRLIyWC2W9IvffYdcJXPm016WcaMQmAi+j87tSv8bpXPNPNycdid6g nkv6YlizNVVjHQsb8/nngOHQQwYWWQ1Xx6X1gmv7U+skcMXrKRyzZ79sd/6CtLjnpvFm 7mZg== X-Gm-Message-State: AOAM533SkJPDN9qXNTGbWz9Tv3M82LKSjOq9r4mRV8m3Fp2m8pY3YCbZ d7+W8eKmW97SVtiFzKNkX+7ihyZxOnbW5qO/ X-Received: by 2002:a1c:7206:: with SMTP id n6mr1626129wmc.33.1612173732622; Mon, 01 Feb 2021 02:02:12 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:11 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 03/12] futex: Replace PF_EXITPIDONE with a state Date: Mon, 1 Feb 2021 10:01:34 +0000 Message-Id: <20210201100143.2028618-4-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3d4775df0a89240f671861c6ab6e8d59af8e9e41 upstream. The futex exit handling relies on PF_ flags. That's suboptimal as it requires a smp_mb() and an ugly lock/unlock of the exiting tasks pi_lock in the middle of do_exit() to enforce the observability of PF_EXITING in the futex code. Add a futex_state member to task_struct and convert the PF_EXITPIDONE logic over to the new state. The PF_EXITING dependency will be cleaned up in a later step. This prepares for handling various futex exit issues later. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.149449274@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 34 ++++++++++++++++++++++++++++++++++ include/linux/sched.h | 2 +- kernel/exit.c | 18 ++---------------- kernel/futex.c | 17 ++++++++--------- 4 files changed, 45 insertions(+), 26 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 63d353cedfcde..a0de6fe28e00b 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -55,6 +55,11 @@ union futex_key { #define FUTEX_KEY_INIT (union futex_key) { .both = { .ptr = 0ULL } } #ifdef CONFIG_FUTEX +enum { + FUTEX_STATE_OK, + FUTEX_STATE_DEAD, +}; + static inline void futex_init_task(struct task_struct *tsk) { tsk->robust_list = NULL; @@ -63,6 +68,34 @@ static inline void futex_init_task(struct task_struct *tsk) #endif INIT_LIST_HEAD(&tsk->pi_state_list); tsk->pi_state_cache = NULL; + tsk->futex_state = FUTEX_STATE_OK; +} + +/** + * futex_exit_done - Sets the tasks futex state to FUTEX_STATE_DEAD + * @tsk: task to set the state on + * + * Set the futex exit state of the task lockless. The futex waiter code + * observes that state when a task is exiting and loops until the task has + * actually finished the futex cleanup. The worst case for this is that the + * waiter runs through the wait loop until the state becomes visible. + * + * This has two callers: + * + * - futex_mm_release() after the futex exit cleanup has been done + * + * - do_exit() from the recursive fault handling path. + * + * In case of a recursive fault this is best effort. Either the futex exit + * code has run already or not. If the OWNER_DIED bit has been set on the + * futex then the waiter can take it over. If not, the problem is pushed + * back to user space. If the futex exit code did not run yet, then an + * already queued waiter might block forever, but there is nothing which + * can be done about that. + */ +static inline void futex_exit_done(struct task_struct *tsk) +{ + tsk->futex_state = FUTEX_STATE_DEAD; } void futex_mm_release(struct task_struct *tsk); @@ -72,5 +105,6 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, #else static inline void futex_init_task(struct task_struct *tsk) { } static inline void futex_mm_release(struct task_struct *tsk) { } +static inline void futex_exit_done(struct task_struct *tsk) { } #endif #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index 1872d4e9acbe1..4de48b251447f 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1815,6 +1815,7 @@ struct task_struct { #endif struct list_head pi_state_list; struct futex_pi_state *pi_state_cache; + unsigned int futex_state; #endif #ifdef CONFIG_PERF_EVENTS struct perf_event_context *perf_event_ctxp[perf_nr_task_contexts]; @@ -2276,7 +2277,6 @@ extern void thread_group_cputime_adjusted(struct task_struct *p, cputime_t *ut, * Per process flags */ #define PF_EXITING 0x00000004 /* getting shut down */ -#define PF_EXITPIDONE 0x00000008 /* pi exit done on shut down */ #define PF_VCPU 0x00000010 /* I'm a virtual CPU */ #define PF_WQ_WORKER 0x00000020 /* I'm a workqueue worker */ #define PF_FORKNOEXEC 0x00000040 /* forked but didn't exec */ diff --git a/kernel/exit.c b/kernel/exit.c index f9943ef23fa82..969e1468f2538 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -785,16 +785,7 @@ void __noreturn do_exit(long code) */ if (unlikely(tsk->flags & PF_EXITING)) { pr_alert("Fixing recursive fault but reboot is needed!\n"); - /* - * We can do this unlocked here. The futex code uses - * this flag just to verify whether the pi state - * cleanup has been done or not. In the worst case it - * loops once more. We pretend that the cleanup was - * done as there is no way to return. Either the - * OWNER_DIED bit is set by now or we push the blocked - * task into the wait for ever nirwana as well. - */ - tsk->flags |= PF_EXITPIDONE; + futex_exit_done(tsk); set_current_state(TASK_UNINTERRUPTIBLE); schedule(); } @@ -876,12 +867,7 @@ void __noreturn do_exit(long code) * Make sure we are holding no locks: */ debug_check_no_locks_held(); - /* - * We can do this unlocked here. The futex code uses this flag - * just to verify whether the pi state cleanup has been done - * or not. In the worst case it loops once more. - */ - tsk->flags |= PF_EXITPIDONE; + futex_exit_done(tsk); if (tsk->io_context) exit_io_context(tsk); diff --git a/kernel/futex.c b/kernel/futex.c index 156b23f4b9aac..51bbe57bb14ac 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1099,19 +1099,18 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, } /* - * We need to look at the task state flags to figure out, - * whether the task is exiting. To protect against the do_exit - * change of the task flags, we do this protected by - * p->pi_lock: + * We need to look at the task state to figure out, whether the + * task is exiting. To protect against the change of the task state + * in futex_exit_release(), we do this protected by p->pi_lock: */ raw_spin_lock_irq(&p->pi_lock); - if (unlikely(p->flags & PF_EXITING)) { + if (unlikely(p->futex_state != FUTEX_STATE_OK)) { /* - * The task is on the way out. When PF_EXITPIDONE is - * set, we know that the task has finished the - * cleanup: + * The task is on the way out. When the futex state is + * FUTEX_STATE_DEAD, we know that the task has finished + * the cleanup: */ - int ret = (p->flags & PF_EXITPIDONE) ? -ESRCH : -EAGAIN; + int ret = (p->futex_state = FUTEX_STATE_DEAD) ? -ESRCH : -EAGAIN; raw_spin_unlock_irq(&p->pi_lock); put_task_struct(p); From patchwork Mon Feb 1 10:01:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374222 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1118910jah; Mon, 1 Feb 2021 02:02:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJzt1OcGJbfcWQ1K2Amjt+6KeHnWfUEBKzKCH6PO+6KA1/Nes3K/ou6tUekINSPgKtqR7s76 X-Received: by 2002:a17:906:5608:: with SMTP id f8mr16917565ejq.101.1612173777526; Mon, 01 Feb 2021 02:02:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173777; cv=none; d=google.com; s=arc-20160816; b=aEluaRcU+kvt6beLQ+nEFtx3ez2BVboSXQs96rPpeoHc0jPjFkuRZ9Kbmfjl0C8R3E NKxpvFo+PxzqIbs0UtGSiXxCfgDXM6Ps0rhzfP+VL44nagkneqyI2VmjGTHMJGVf4+cf 53M65Rglofyh+O37nyPY5TUkXBl8sNJYUk20dUQX23j+rgA0/IDxJUH/B9B85SfAp1/u yGMGzQLCTtRrH1FnZCuL/WwL534LzeFhA1N0RYxRkuTWvbh9e+K4Rb0Ze1K9JjPXD3FQ 01ZDk4IQH1o/2k0WK+Yx+0yVmHal0lB7XMmQccD43Aew8qTTYybqG+GeGtZFzfH+BsbF +gMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=j+pI7rP4alOtZeqOXZIiiPmT2Z8ZBGSq2k5HmQdHbwQ=; b=SObYvzf+7f/osTKscvNtIlpm0oXoO3VVpyZacvZEukmOqMe45e9421PCKyGpmIRB0g mYB8kWnjmVb2akHe9J86fsvQIDsuG2bLpkmt/EvlP6wF1PVZ4qi65+cKtYfYsKTxpRp9 HXDfRTkhVQKMI0Wb+C4LKvud+2xmL+2GSb90Q+QdcmoY4zRcfasovsZaKrzXQaR1mGUO WKmrg6IGkmm6XcC6xAWBaPb+oYQMTvLzNficbJGtVkkkq4DVqjLqOKNkx2XIusXHVFwe O4HVePjfGizycwcCpkRosJMoES42QlBocIwcv/mXss3+bvFWLmmXhMVrphd0U0pbtOG5 OrYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Rt/sFTvf"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y23si201010edv.312.2021.02.01.02.02.57; Mon, 01 Feb 2021 02:02:57 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Rt/sFTvf"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229558AbhBAKC4 (ORCPT + 13 others); Mon, 1 Feb 2021 05:02:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232790AbhBAKCz (ORCPT ); Mon, 1 Feb 2021 05:02:55 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 527E8C0613D6 for ; Mon, 1 Feb 2021 02:02:15 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id g10so15884497wrx.1 for ; Mon, 01 Feb 2021 02:02:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j+pI7rP4alOtZeqOXZIiiPmT2Z8ZBGSq2k5HmQdHbwQ=; b=Rt/sFTvfyALMfEuNJqHxRt42NAbvFlxfpyDxd4WvR4tzgOgT8BiTE1h3/cR067jYbB G2a9TidXAOiiL7JNtulcuB5DzozZAbe0NQ9vtoW33fBSWipvxwcROquNonsOizLe5R9w BhEcU4i+KOJ5rW7d1xfasOZC5XgUmZw7jBngWRH8icfp5Rq0M1c9RsYJgVnXRAgBKbkG 6Fyq4cgD0p8VBRqppRfBXaFkQL4tvGn1nouAMwcnhMsLpTSjvo/Rv7WmC+0Osz4hLZCZ WomNIKOwMK6dYnnhK8Ue58jHWuiIn6GY5PS2vzWMRl/yTxOOcGGpDpkQR7/hHOljzbmB 6Ysg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j+pI7rP4alOtZeqOXZIiiPmT2Z8ZBGSq2k5HmQdHbwQ=; b=OQ0VlINDYW+sFY36CqML+qFWMfIlVSpcHQ1vtXK+MJSB6h+ho4/ufxpavw3YH2pfNT 2NBwIf2w1/1+pwWSFHJt6z6HADf8rH2tmckt1SBXQyVMVY/9bIL/w3N/mP0BkzLP5xyO rNrAqCgc9RSJCpEVTP4lZ8mKQtoDJl9PvtCLVo87S2AJyBXak8TWmn2kBivVpFODGf2O IAgFeUe7k7W2EOfxx+0QR6Be2+mf5iBApw8dbS4gqilEh2ng67QIIzQ5qzz8SZy6KtNK oB625uGZp+7qTth+UtPiwg289UZ5hOMXtyv5HagM7J+OhAenS8Ozz2IoXJKYUMt8gX8E KK7A== X-Gm-Message-State: AOAM533kRrYDRlGUlXnO665dSp2xZDKRR5i6QNEEyI4l28QowUR0+T62 nKV8p8yFX2t17idtBLonOeq9YeHLZtFBUD69 X-Received: by 2002:adf:b60e:: with SMTP id f14mr4988231wre.99.1612173733625; Mon, 01 Feb 2021 02:02:13 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:13 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 04/12] exit/exec: Seperate mm_release() Date: Mon, 1 Feb 2021 10:01:35 +0000 Message-Id: <20210201100143.2028618-5-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 4610ba7ad877fafc0a25a30c6c82015304120426 upstream. mm_release() contains the futex exit handling. mm_release() is called from do_exit()->exit_mm() and from exec()->exec_mm(). In the exit_mm() case PF_EXITING and the futex state is updated. In the exec_mm() case these states are not touched. As the futex exit code needs further protections against exit races, this needs to be split into two functions. Preparatory only, no functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.240518241@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- fs/exec.c | 2 +- include/linux/sched.h | 6 ++++-- kernel/exit.c | 2 +- kernel/fork.c | 12 +++++++++++- 4 files changed, 17 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/fs/exec.c b/fs/exec.c index cd5da140f94cb..319a1f5732fa9 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -1021,7 +1021,7 @@ static int exec_mmap(struct mm_struct *mm) /* Notify parent that we're no longer interested in the old VM */ tsk = current; old_mm = current->mm; - mm_release(tsk, old_mm); + exec_mm_release(tsk, old_mm); if (old_mm) { sync_mm_rss(old_mm); diff --git a/include/linux/sched.h b/include/linux/sched.h index 4de48b251447f..fcbe5904cbd97 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2955,8 +2955,10 @@ extern struct mm_struct *get_task_mm(struct task_struct *task); * succeeds. */ extern struct mm_struct *mm_access(struct task_struct *task, unsigned int mode); -/* Remove the current tasks stale references to the old mm_struct */ -extern void mm_release(struct task_struct *, struct mm_struct *); +/* Remove the current tasks stale references to the old mm_struct on exit() */ +extern void exit_mm_release(struct task_struct *, struct mm_struct *); +/* Remove the current tasks stale references to the old mm_struct on exec() */ +extern void exec_mm_release(struct task_struct *, struct mm_struct *); #ifdef CONFIG_HAVE_COPY_THREAD_TLS extern int copy_thread_tls(unsigned long, unsigned long, unsigned long, diff --git a/kernel/exit.c b/kernel/exit.c index 969e1468f2538..b65285f5ee0c9 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -464,7 +464,7 @@ static void exit_mm(struct task_struct *tsk) struct mm_struct *mm = tsk->mm; struct core_state *core_state; - mm_release(tsk, mm); + exit_mm_release(tsk, mm); if (!mm) return; sync_mm_rss(mm); diff --git a/kernel/fork.c b/kernel/fork.c index 000447bfcfde5..ad9dbbf03d7bc 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1082,7 +1082,7 @@ static int wait_for_vfork_done(struct task_struct *child, * restoring the old one. . . * Eric Biederman 10 January 1998 */ -void mm_release(struct task_struct *tsk, struct mm_struct *mm) +static void mm_release(struct task_struct *tsk, struct mm_struct *mm) { /* Get rid of any futexes when releasing the mm */ futex_mm_release(tsk); @@ -1119,6 +1119,16 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm) complete_vfork_done(tsk); } +void exit_mm_release(struct task_struct *tsk, struct mm_struct *mm) +{ + mm_release(tsk, mm); +} + +void exec_mm_release(struct task_struct *tsk, struct mm_struct *mm) +{ + mm_release(tsk, mm); +} + /* * Allocate a new mm structure and copy contents from the * mm structure of the passed in task structure. From patchwork Mon Feb 1 10:01:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374229 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120356jah; Mon, 1 Feb 2021 02:05:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJwXnxebxPIoHX1Q9jW9au7p3/1orh7tiSEzwGz9wnqofmVB/C1R8/izS2supBbVsV8xg0aT X-Received: by 2002:a05:6402:100b:: with SMTP id c11mr17886303edu.193.1612173900526; Mon, 01 Feb 2021 02:05:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173900; cv=none; d=google.com; s=arc-20160816; b=QKVRyzO/+QWStQyhbDHzg8mPMfPNWN5rafm+QI4upxFTk196oCkFhQpQNYwUPCtHHv 72vBbbFmAArwRg2OwDSTi2yEjDtJWLI7ul3uVHAIA5MWvkWkA0FKgzvI8er4V099y5xx juFLSiPmkz24Ts/+eVzuYlgQpA5H0c5wi+2Dlqtw110gLGF0xp/Gn5MLbXtSg/h5Qrk1 bxbN91q3QwFmlKpOXHiMCd/87Z99mbwP4IFyyVGQkx5wX94Vx9qb0sNrqIEOd6oaCkej 9xt9ZOlnntZpQt2NwLMDUbU/jsY93+xH0fm8rPkCyaFadFirM7se4aivvuPWtGOgBMuV mh2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qqBoT1xpWEYMKezlYyKQ2zkhv0Ilyy8rHIYG+0h8Im8=; b=AI3VaNTGKD65a4Pv5U1SlfTSf1AdnUiDJoLVROzMiQlhkUWzqsL6roFPlVA5gFbc61 7myW34Kiptzr5Mh8RGZUrSHAcEqIE4jQNAUt1t3l7VJW0IBk44n8PxL1scMWTMwt6Zyu DR60elzwB76Mr2ffOPn/LNG0zfF9cq44zcC0NTtmEFqaBEEtAXqszQ15eCS0OO0KKcMC 32lqBqu+pMSRPVvPtDY/xizdwxOQHCveaQuYx7q3kizz9IoF1FUJqHak4eDhHR85+Tf4 FYF7LBhLfwF1WFAVVEgyz1WKodDQaz2Hrz1NA5RhY4Be+kThXJcnhrHlu6H3JxRXt0ck TYkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VZVSw6E8; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.05.00; Mon, 01 Feb 2021 02:05:00 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VZVSw6E8; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233009AbhBAKDl (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233011AbhBAKDi (ORCPT ); Mon, 1 Feb 2021 05:03:38 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C4B6C061788 for ; Mon, 1 Feb 2021 02:02:18 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id p15so15827322wrq.8 for ; Mon, 01 Feb 2021 02:02:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qqBoT1xpWEYMKezlYyKQ2zkhv0Ilyy8rHIYG+0h8Im8=; b=VZVSw6E8CrBMDpWIroh7AP3ijX6WSF/iNqU7a3OsFxWDE0C7NN8ad06UubkFhbSf/P lQ7oLzLCLKTTf9lN7OtY4wHQP7Mb+Fz5x1RjWlODXgfdFRBd80r42g9lJgS9BlNq68bu Fj0NWOAWoAZAli//1fi/8NttxABhPJcroBYktMSX07jPfzFzQvjF666VbzibrVmaZrOE lbosIeCcY5GXi1PrXa5hGUrE2YJEcGMPJogSmB1lf2eNI+BlaC14clUDpe8qfqtOZpdR 9BH6Bd14T2l6x8BcXaMfLaRXHikK9jwM5HSNK9rSf+AbYjxzj7lrXr2IbJVZFcQe/KJM gJBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qqBoT1xpWEYMKezlYyKQ2zkhv0Ilyy8rHIYG+0h8Im8=; b=eIeJXIQ6L+Fwf1aoY4BErFxkqp+wG6e7B8hTIqDItKE8XDAfmFeoJgsprIYmIiVvWE 9ucFBkq22b4RbdMBfA2L9LYDXtxpqTNpg7GWkE9xu8Y50o1yI+3E6SCRN4yBroJnnV7R fMjtE6xxW/iDAj1ltyOIkr60fb4qGpvRK3qeKEwL+ooq5O7w+tCtH+6Ma1sxN9EY4cTq fPwEB6/bJ0muc4WeAlt8Sv3fXxtivF1Zawkh2cR0PZO/HRTwPflCv4pJxnMiU8//Go26 xeO1cKL4s+Y2365TyOgl4q+qwFal8SVAL79p1co9QT/Br1q/bARF5MqoP9nt8wAnRme5 VzYg== X-Gm-Message-State: AOAM530VRrM17gzCLRFFXAH6+Iklg4RE+9F1OBbQt2pP+RZNrASMs5y5 EfixULX21BcAQwDzzeJZKy7zmHEchLHkCAxZ X-Received: by 2002:a5d:5283:: with SMTP id c3mr16682221wrv.319.1612173734636; Mon, 01 Feb 2021 02:02:14 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:14 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 05/12] futex: Split futex_mm_release() for exit/exec Date: Mon, 1 Feb 2021 10:01:36 +0000 Message-Id: <20210201100143.2028618-6-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 150d71584b12809144b8145b817e83b81158ae5f upstream. To allow separate handling of the futex exit state in the futex exit code for exit and exec, split futex_mm_release() into two functions and invoke them from the corresponding exit/exec_mm_release() callsites. Preparatory only, no functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.332094221@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 6 ++++-- kernel/fork.c | 5 ++--- kernel/futex.c | 7 ++++++- 3 files changed, 12 insertions(+), 6 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index a0de6fe28e00b..063a5cd00d770 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -98,13 +98,15 @@ static inline void futex_exit_done(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_DEAD; } -void futex_mm_release(struct task_struct *tsk); +void futex_exit_release(struct task_struct *tsk); +void futex_exec_release(struct task_struct *tsk); long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3); #else static inline void futex_init_task(struct task_struct *tsk) { } -static inline void futex_mm_release(struct task_struct *tsk) { } static inline void futex_exit_done(struct task_struct *tsk) { } +static inline void futex_exit_release(struct task_struct *tsk) { } +static inline void futex_exec_release(struct task_struct *tsk) { } #endif #endif diff --git a/kernel/fork.c b/kernel/fork.c index ad9dbbf03d7bc..91349fd3e162d 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1084,9 +1084,6 @@ static int wait_for_vfork_done(struct task_struct *child, */ static void mm_release(struct task_struct *tsk, struct mm_struct *mm) { - /* Get rid of any futexes when releasing the mm */ - futex_mm_release(tsk); - uprobe_free_utask(tsk); /* Get rid of any cached register state */ @@ -1121,11 +1118,13 @@ static void mm_release(struct task_struct *tsk, struct mm_struct *mm) void exit_mm_release(struct task_struct *tsk, struct mm_struct *mm) { + futex_exit_release(tsk); mm_release(tsk, mm); } void exec_mm_release(struct task_struct *tsk, struct mm_struct *mm) { + futex_exec_release(tsk); mm_release(tsk, mm); } diff --git a/kernel/futex.c b/kernel/futex.c index 51bbe57bb14ac..902ce420c4ba0 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3269,7 +3269,7 @@ static void exit_robust_list(struct task_struct *curr) curr, pip); } -void futex_mm_release(struct task_struct *tsk) +void futex_exec_release(struct task_struct *tsk) { if (unlikely(tsk->robust_list)) { exit_robust_list(tsk); @@ -3287,6 +3287,11 @@ void futex_mm_release(struct task_struct *tsk) exit_pi_state_list(tsk); } +void futex_exit_release(struct task_struct *tsk) +{ + futex_exec_release(tsk); +} + long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3) { From patchwork Mon Feb 1 10:01:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374223 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120291jah; Mon, 1 Feb 2021 02:04:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJzj3zYNmTp7VogW4nrKCq7LsIlRvzzMnvgtc2fgRUvRxzh0+gOJkpVy2Z0uM/oduLcM85Kk X-Received: by 2002:a17:906:82c9:: with SMTP id a9mr6267037ejy.547.1612173896134; Mon, 01 Feb 2021 02:04:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173896; cv=none; d=google.com; s=arc-20160816; b=HzToEqcMnxuB70jObnpy1RWOzMPUDbMd4SzLi8mSuhFIoZAorh9Fjc7MAliTwiNtze TB9+UUz4ZX18hkg2+RYXoKkalzrUEUMq51NS3TwWaRqjf5YQ42zvhNn6XF6zCFreBWLp ODooxxmE7jr0OZIXIYhXNTXJaowvAFb0wyR00233dO3YO6RqQIbhXTiCSY9A3+EGYpGu 1O2qXjy+m8msAuqVUvlGW/paHFJILkPAsQA/7dxkrSgJCuKRJ5sLicUrssRYLkMk3Hbl vbEcefwzkF8wSaNFy9O0O5t3YZ7gkEq+he0Ix1BzCEceE7vEZZZbdTHj2GVOAgDeGs1C 84lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=AWHPQeLileZutTX3lhjpX/znrmKRPfFsORf0TLQerQ4=; b=fbwSpENvRJOPRiHIARuZEK/pJSQCRlAawwr2VZHQZ3pwpnUg32XpHs8YJEhhIOpAgx 99CVidlyShSYLOdX0romedfbWyq2hVJzzKPGMhui825b8qpTJqWNU7tvqpg7BWqC9t9p /tKSeDO3hH25UePUIIEzaRV64fjQqV7UEfnxK0hru+jT2H/SAf25Z2arkz8VyYtWfaBd oEBOn90z+D7sv5NWa6LThbzPTx1JlEa9fRLQGUYXPOZeq5ALY8bK1Iuj4Kpk81VZ9Q8d 9/tMOhb4hUL2mrRqn8/woq0MPeDglqqOnWqo27OXCVgDsnCyhgF2gPRXgUEGZziseJaz TlgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JXsktWtI; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.04.55; Mon, 01 Feb 2021 02:04:56 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JXsktWtI; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232893AbhBAKDe (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232995AbhBAKDc (ORCPT ); Mon, 1 Feb 2021 05:03:32 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 366F8C06178A for ; Mon, 1 Feb 2021 02:02:19 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id 6so15866126wri.3 for ; Mon, 01 Feb 2021 02:02:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AWHPQeLileZutTX3lhjpX/znrmKRPfFsORf0TLQerQ4=; b=JXsktWtI4N+2CsLZCZ4Gdc9iJ0cpbUsmX60SKgfItYjkzRz0DkuW6MIl/Gz2r4H0+/ fMPJGiN3w7h1jzaY7NoZr9GH80QOY4BASXdGeJgw0Mi2ppH36QxZ5J0kRQsE7eU7Vd1k roQv1l3Q8INz1WUK8/RyzYyohB7kpqwv+nDkuc/hWCr11HRDVJT4Udf3Yq3ibjbV6UIT kRJ1eOCZxQKi0UDSMpvIDLaTqrWU/ZW7O0Va0utcLZ/+IS7JmImmKIG9KHCm/83q/VhO j88USs0Y3eMpbrUcOmNDeEWgtTa9csc6Vujr2d24UpJAYYdLaQhMHm5yLll4UfFlrXIB tSXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AWHPQeLileZutTX3lhjpX/znrmKRPfFsORf0TLQerQ4=; b=CXmrDBqxlw5VBp1/t9/Dt9EZrMOzDLmZOib2eqHUq9F7B6F3rAscjvmsQgvL833vb5 UEZLhdHSd/OYnMazQEy0FPoFm9PIzKm2vySCZbUGWfKFJXOVmv+I5d/V+CQdol0wqGwW js+iwJLdeWQ7bPkT9Lt1ojNNprgTvS7Ry+OV0KcOiK4AzuWuT5ljdgo+rDuMaruEJMqU o7QfOdNXnExj/1daHYfJ7xUowSsFqwsSnFMdIsXjJbe6rv3S1Oc5DakQW+VHmNMhzMaD O2o9z7Aswk0vhl9TB8KUL9NuhptllADgeV0DRNzK9VCxZE0gTokTRX+3WFnp5dF7pk9S V6rA== X-Gm-Message-State: AOAM5336tFxGa0ERpyKBRM1kn2NOT5cn1wwQuylrP1LIYwCBqbDZzLx9 Po1W04+CKLIZUtgeHoi7aO+AggoNCaaEvbLm X-Received: by 2002:adf:dfc7:: with SMTP id q7mr17239970wrn.153.1612173737649; Mon, 01 Feb 2021 02:02:17 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:16 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 06/12] futex: Set task::futex_state to DEAD right after handling futex exit Date: Mon, 1 Feb 2021 10:01:37 +0000 Message-Id: <20210201100143.2028618-7-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit f24f22435dcc11389acc87e5586239c1819d217c upstream. Setting task::futex_state in do_exit() is rather arbitrarily placed for no reason. Move it into the futex code. Note, this is only done for the exit cleanup as the exec cleanup cannot set the state to FUTEX_STATE_DEAD because the task struct is still in active use. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.439511191@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/exit.c | 1 - kernel/futex.c | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) -- 2.25.1 diff --git a/kernel/exit.c b/kernel/exit.c index b65285f5ee0c9..e87ab2ec654bc 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -867,7 +867,6 @@ void __noreturn do_exit(long code) * Make sure we are holding no locks: */ debug_check_no_locks_held(); - futex_exit_done(tsk); if (tsk->io_context) exit_io_context(tsk); diff --git a/kernel/futex.c b/kernel/futex.c index 902ce420c4ba0..e8322c3208a44 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3290,6 +3290,7 @@ void futex_exec_release(struct task_struct *tsk) void futex_exit_release(struct task_struct *tsk) { futex_exec_release(tsk); + futex_exit_done(tsk); } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 10:01:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374224 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120299jah; Mon, 1 Feb 2021 02:04:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJynFM2fuxbiBjp81xU+lXbnaEn6AUDjdxvlGzxfU4gbr/tTKY1uZ5UGPE1oyK4jcAwLGc/N X-Received: by 2002:a50:fd98:: with SMTP id o24mr5835875edt.304.1612173896550; Mon, 01 Feb 2021 02:04:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173896; cv=none; d=google.com; s=arc-20160816; b=ApyuczCO97gfufRm6xih6mWfVw0i1wLt6eImX3U5t8d/jrpLazvkmrIrfZZXT1Bl3x nkOA8nJdHb84269NXaouxNXDU4S/Syo5sIizgtGsS7X9OywgjElSjX8p8IMrEpO8OKOa n9D2uwonjj9xgVgYOmPwElH/XL5C7OPP8/0l1q7D2dmFDIMwY6HigPYxulvcFWL0qfsf Kr/kdWW1CdRz7MCfMDRz3Sf9gN7sB7FGwNoR6Gm4AtXiv5HNbKnIXv5Q03y1NZtAp4Vd 92KunV2UQ3wD94d9S/qMajXyNNOuh7G4TdXHV4gJMs7au/b6XKsEM4hxDVMDXtOXBaEq JqmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=H8SGnzV5I3ZFHFe8hRMUykXFpl/MX53xmNixUdziUtA=; b=b7UQfmvCDA2oExHxXToGDqGznoKkDt4KRS5mBYGv/Uj3b0xphYb5acrsS7n6S8nOGk d0ZKc7IkSMUiiJtANxBDljtxP1mrIpeoMokeGrcLmuCez+39+H4j4w8aqS0uD3su5Ph8 IgElW6NCmX5w/lS05tFy/sdRmT1k8Qu5xhlr03ZLBZrMtMw0o3NZ+EpEpIMdqTL6CXHW UoUpKnWBWCsL16G8knsoS8X3Hnzisznvy6c5a9GMPrqH5+QIsMWLi7oQz7RrPW7JbORM 1iNHozCq9znGnIuDemDeyKAzT9J0FN/X1O7xuRM/Z6djaK7hhuW0XfZp+16kkaUZVx7e xtXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wq8MoyoH; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.04.56; Mon, 01 Feb 2021 02:04:56 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wq8MoyoH; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233010AbhBAKDe (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232893AbhBAKDd (ORCPT ); Mon, 1 Feb 2021 05:03:33 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4FB9C06178B for ; Mon, 1 Feb 2021 02:02:21 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id y187so12623279wmd.3 for ; Mon, 01 Feb 2021 02:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H8SGnzV5I3ZFHFe8hRMUykXFpl/MX53xmNixUdziUtA=; b=Wq8MoyoHM/kx+Ta2dRX15xYcAFJvW0RSFkP/xCs1VUuI1u4tI21qsaJxKYQ+xld60s Q27H7xMWNt8uF6+VwtBDc/osF9mWasBknWc8Lz3WiE9b2SeY8fyGP+eIH4ZcC/GAeX25 pZw2h7VWZHPo5Y1H9q6iPaWOds6KIKK5WZ2Awm/2TcWeWaanAsoRa4YoTLnrm786wxiq fiV56cdA+Om8sAzghF9zMBbN20mV3IG4ycd/lqWpXSdb3SU5QrHnzb4XbQ9g+kGokvcF udQy9tu5UrCD/WPbXUDNAxX6lKe/+oUzzXgnpgmyyvVxiJ88zicZ2lFgYMygRvPauudE KcYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H8SGnzV5I3ZFHFe8hRMUykXFpl/MX53xmNixUdziUtA=; b=jRVUV4KEPb/JMn9DzoJZNKlP8+xCRq23+hRTLDK/Ve8mpOaMu+rmYYZd+rfZPkSdWE 4AHgO7iSuJnd/1FvbS4Drh5v7sRhR3nhclARPeP5rpxo0FP6pJH+gaHFX7eDYeHmiMDp bnoaPxZVh8VXeDs5ULRyjs2ow9vhwRW55f2XcO1pBU0dW/dzXxuSjyD7x+HpR//tDf8L fBJ4tsq+oMwJFHVXvrHU5QwSDxNHsKenzDCoq/bjrr3zsa8j5/D0PZeNo75QM88UOTUa /Lpem2sXAOgrY4FLJ1N4PTcWaPokYMxXsrLQcigDuudcg465h72mCpF2ftjFZQNYoaNx GMPw== X-Gm-Message-State: AOAM531vIlHZElbNkjp8yIOvur5HQKP69eCnSvu6UrJBRWSpLco0F+sv MH79cBD52gG4Dwk76PuD6gm9iRLVRaJd1fg9 X-Received: by 2002:a05:600c:4f50:: with SMTP id m16mr14036164wmq.175.1612173738875; Mon, 01 Feb 2021 02:02:18 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:18 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 07/12] futex: Mark the begin of futex exit explicitly Date: Mon, 1 Feb 2021 10:01:38 +0000 Message-Id: <20210201100143.2028618-8-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 18f694385c4fd77a09851fd301236746ca83f3cb upstream. Instead of relying on PF_EXITING use an explicit state for the futex exit and set it in the futex exit function. This moves the smp barrier and the lock/unlock serialization into the futex code. As with the DEAD state this is restricted to the exit path as exec continues to use the same task struct. This allows to simplify that logic in a next step. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.539409004@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 31 +++---------------------------- kernel/exit.c | 12 +----------- kernel/futex.c | 37 ++++++++++++++++++++++++++++++++++++- 3 files changed, 40 insertions(+), 40 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 063a5cd00d770..805508373fcea 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -57,6 +57,7 @@ union futex_key { #ifdef CONFIG_FUTEX enum { FUTEX_STATE_OK, + FUTEX_STATE_EXITING, FUTEX_STATE_DEAD, }; @@ -71,33 +72,7 @@ static inline void futex_init_task(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_OK; } -/** - * futex_exit_done - Sets the tasks futex state to FUTEX_STATE_DEAD - * @tsk: task to set the state on - * - * Set the futex exit state of the task lockless. The futex waiter code - * observes that state when a task is exiting and loops until the task has - * actually finished the futex cleanup. The worst case for this is that the - * waiter runs through the wait loop until the state becomes visible. - * - * This has two callers: - * - * - futex_mm_release() after the futex exit cleanup has been done - * - * - do_exit() from the recursive fault handling path. - * - * In case of a recursive fault this is best effort. Either the futex exit - * code has run already or not. If the OWNER_DIED bit has been set on the - * futex then the waiter can take it over. If not, the problem is pushed - * back to user space. If the futex exit code did not run yet, then an - * already queued waiter might block forever, but there is nothing which - * can be done about that. - */ -static inline void futex_exit_done(struct task_struct *tsk) -{ - tsk->futex_state = FUTEX_STATE_DEAD; -} - +void futex_exit_recursive(struct task_struct *tsk); void futex_exit_release(struct task_struct *tsk); void futex_exec_release(struct task_struct *tsk); @@ -105,7 +80,7 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3); #else static inline void futex_init_task(struct task_struct *tsk) { } -static inline void futex_exit_done(struct task_struct *tsk) { } +static inline void futex_exit_recursive(struct task_struct *tsk) { } static inline void futex_exit_release(struct task_struct *tsk) { } static inline void futex_exec_release(struct task_struct *tsk) { } #endif diff --git a/kernel/exit.c b/kernel/exit.c index e87ab2ec654bc..8716f0780fe3d 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -785,22 +785,12 @@ void __noreturn do_exit(long code) */ if (unlikely(tsk->flags & PF_EXITING)) { pr_alert("Fixing recursive fault but reboot is needed!\n"); - futex_exit_done(tsk); + futex_exit_recursive(tsk); set_current_state(TASK_UNINTERRUPTIBLE); schedule(); } exit_signals(tsk); /* sets PF_EXITING */ - /* - * Ensure that all new tsk->pi_lock acquisitions must observe - * PF_EXITING. Serializes against futex.c:attach_to_pi_owner(). - */ - smp_mb(); - /* - * Ensure that we must observe the pi_state in exit_mm() -> - * mm_release() -> exit_pi_state_list(). - */ - raw_spin_unlock_wait(&tsk->pi_lock); /* sync mm's RSS info before statistics gathering */ if (tsk->mm) diff --git a/kernel/futex.c b/kernel/futex.c index e8322c3208a44..482000996b983 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3287,10 +3287,45 @@ void futex_exec_release(struct task_struct *tsk) exit_pi_state_list(tsk); } +/** + * futex_exit_recursive - Set the tasks futex state to FUTEX_STATE_DEAD + * @tsk: task to set the state on + * + * Set the futex exit state of the task lockless. The futex waiter code + * observes that state when a task is exiting and loops until the task has + * actually finished the futex cleanup. The worst case for this is that the + * waiter runs through the wait loop until the state becomes visible. + * + * This is called from the recursive fault handling path in do_exit(). + * + * This is best effort. Either the futex exit code has run already or + * not. If the OWNER_DIED bit has been set on the futex then the waiter can + * take it over. If not, the problem is pushed back to user space. If the + * futex exit code did not run yet, then an already queued waiter might + * block forever, but there is nothing which can be done about that. + */ +void futex_exit_recursive(struct task_struct *tsk) +{ + tsk->futex_state = FUTEX_STATE_DEAD; +} + void futex_exit_release(struct task_struct *tsk) { + tsk->futex_state = FUTEX_STATE_EXITING; + /* + * Ensure that all new tsk->pi_lock acquisitions must observe + * FUTEX_STATE_EXITING. Serializes against attach_to_pi_owner(). + */ + smp_mb(); + /* + * Ensure that we must observe the pi_state in exit_pi_state_list(). + */ + raw_spin_lock_irq(&tsk->pi_lock); + raw_spin_unlock_irq(&tsk->pi_lock); + futex_exec_release(tsk); - futex_exit_done(tsk); + + tsk->futex_state = FUTEX_STATE_DEAD; } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 10:01:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374225 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120307jah; Mon, 1 Feb 2021 02:04:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJy+6FmEPQdN4KrVzbkhG4l2/nlGtTOz6n8QsFdfHD1gfBMOMtfv4nKsGBh4c51EBhbkkOOw X-Received: by 2002:a05:6402:13c8:: with SMTP id a8mr17757784edx.191.1612173896957; Mon, 01 Feb 2021 02:04:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173896; cv=none; d=google.com; s=arc-20160816; b=atNrWkGXF1w531PvcM8sGohD8kHlV5PWmPGplm1Qpon6o+OZuRh8zSzBfADYtTh3Ps QatazgTbaz0BL/9fjUoJ0gAiaA4ixdsvjDnRErl+vSQC4AMTqZSv8rlDBKjMJSLBJAAV MQhrBqrGTdyEX9lfjVOt0Ppl2KwrRohSdN5l5uxRcQuveSVDbj0ZnsgJQnYpIy1QB7Qt TVu7wBoXSP/CEA+/X6ClV4zQ6JXE60pstFQflqFUL80dK8a6zG0A146EAyC1Q67CHyqQ 0FEK5OmAF8xxv50hK/L6jLGBzJvJejvfIucnV6HnZLfHk2KtwLhJYQ7HRwaS3wShOiAx ADLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xnIHaUcaGnvu4d6/FdlgyP+cy+Y5o5wrHg5QIdWEM68=; b=yuQHemY8MC0T+pPqUX3qb1kg+CEYOJOyPgiWCLrw/mjS5JVfR5Lh+NcAFkwS4a+I9i 7WXCoAH8PgH+bT11uZT4pK1zUC+50LJXl/ztzi8L0cD2dQdjqLquhVzdJV8EeDH1h+Ra 0cf+iDuAHRMpNVjD0cQ2Lzyj+3hqJ8cQCmSEfkvTya6ZzuobeQQG78fb/XbLp0paXeYZ barxT8w5kUmbsQFtRgF/nPhzZH+LFRvOQOwXuLgV5peoRadWnsPwKKHnQAk/RXNjy8Yu +kBrawyFo/wxr73n/+CXSujmO3s1gOTbwn8tdfB0wVYW2AYpgMaXBbIXPpwGrDzpcSg5 pbog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YjLwUqCD; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.04.56; Mon, 01 Feb 2021 02:04:56 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YjLwUqCD; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232995AbhBAKDf (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233009AbhBAKDd (ORCPT ); Mon, 1 Feb 2021 05:03:33 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA727C06178C for ; Mon, 1 Feb 2021 02:02:21 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id g10so15884928wrx.1 for ; Mon, 01 Feb 2021 02:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xnIHaUcaGnvu4d6/FdlgyP+cy+Y5o5wrHg5QIdWEM68=; b=YjLwUqCDzZLfeAElCV+0/ULjcstLNC/MoLRRObgnQFsRBftrxwI1ybQ0vUegpLMn6u w6JnJ4zL2gd9Fn9zHKcL9TH5FdxwM70HSRuaL+9qmERWmViUeIEHpF8Z2CnJrDFO6cLm O45+a4z0atnQyjafTZBc5LzGVXAawwy5LJeJR4P+y/3t4hC+dxkchxrnumivMAudYoYm N8tp3w+PzNrAJRexiYn5Hcw0eFN+UVW1mtkdg9AE4mxTYkzcvHwrABvwG7sscdKaZ9fy oS0Na9+/fKfRtW4fXEPbXdUptsqCyeIsuUM+v+VobKm7sKQCRJugxYGL1+rGR2QmqcD6 PG8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xnIHaUcaGnvu4d6/FdlgyP+cy+Y5o5wrHg5QIdWEM68=; b=n99Vm6FUV4s1YLP8Wy7+KAn7Q5fJ6dsfBBG2jr7AmNoXD+Ku+iYqd3DtT45+k+Ukk+ xmPy2iT4izFBByUAE9hmHb1rkMcO7Lh+Yke895HyWXiUcrf2T1yVww76k7HG1W/vUEP/ 8jsboc33mrR+BMTzUgBQVqi33b3fn6nNc6Ho1YBC+IB28mgG1sQBzXG0vPZ/it3NptNE t09Z/0zCXzDzZsYxfdUaTGzsCflsyNq3Nb99IWLYJaMMXmKJkKgfQNdj0nwvbrXVWFCb C66GoUyQY0eTP8VjCnWSfmkj4U3LUaqtnNFy5+Vuw+WjfjohpB4p9LPY6HJ+m4S0WPOh GWqA== X-Gm-Message-State: AOAM533ekzc+LEXICDSvDqE5UdyrUzaofPcWZSeYS7xymtWWAkzNfR9m 2VmtFfsT0s4c6N5UDN2Ih1nLz6CCl2AARuqx X-Received: by 2002:a05:6000:1202:: with SMTP id e2mr5596646wrx.328.1612173740097; Mon, 01 Feb 2021 02:02:20 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:19 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 08/12] futex: Sanitize exit state handling Date: Mon, 1 Feb 2021 10:01:39 +0000 Message-Id: <20210201100143.2028618-9-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 4a8e991b91aca9e20705d434677ac013974e0e30 upstream. Instead of having a smp_mb() and an empty lock/unlock of task::pi_lock move the state setting into to the lock section. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.645603214@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 482000996b983..f1e8ba64fe8ae 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3311,16 +3311,19 @@ void futex_exit_recursive(struct task_struct *tsk) void futex_exit_release(struct task_struct *tsk) { - tsk->futex_state = FUTEX_STATE_EXITING; - /* - * Ensure that all new tsk->pi_lock acquisitions must observe - * FUTEX_STATE_EXITING. Serializes against attach_to_pi_owner(). - */ - smp_mb(); /* - * Ensure that we must observe the pi_state in exit_pi_state_list(). + * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. + * + * This ensures that all subsequent checks of tsk->futex_state in + * attach_to_pi_owner() must observe FUTEX_STATE_EXITING with + * tsk->pi_lock held. + * + * It guarantees also that a pi_state which was queued right before + * the state change under tsk->pi_lock by a concurrent waiter must + * be observed in exit_pi_state_list(). */ raw_spin_lock_irq(&tsk->pi_lock); + tsk->futex_state = FUTEX_STATE_EXITING; raw_spin_unlock_irq(&tsk->pi_lock); futex_exec_release(tsk); From patchwork Mon Feb 1 10:01:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374226 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120316jah; Mon, 1 Feb 2021 02:04:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJxMV8N7vcNoXYvU6jS05fskzceP/BcswbKvCpqA+qEXRK9st+yYL55GGeiINTPruVLuabQG X-Received: by 2002:adf:8b41:: with SMTP id v1mr16930935wra.282.1612173897359; Mon, 01 Feb 2021 02:04:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173897; cv=none; d=google.com; s=arc-20160816; b=eAD2tJltnw6bI03469NmqnKlxMA/jdB7Bj7n7YK87YIMKDWfxVu94zVMhbNMXZcuE8 elwJjVmMBL30S2Y6w5DVnVerBv2V2Z3JA5L0n0cfcXjBL93haIImbegZWinB0x2F3Anh NqTYQ6zFd+hv6HeICxgfPPT73poV42Eo7kDQ08+O7JgnVPgxa6H6uYGI1hEwQ+YfiC9E kF5m1ZJN2KyN7eUnBFq74IKLT/J1l+ryPUjpFjw3Sx+kOJiDQsAFq0HO4/HLvzd0cbPy a927sMLrhEOMnTsk2YptrCA440FDKGEGGbXDpEXsnYzOcqnSi6GKjZAdSYp3zNanhnBc hKyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Gg480XhfMoGXqP/teEw7Tq4vypgx+NgStVfH6LwTrK4=; b=Hz9mJtClQTAj+QtK1nBeG3kVAaTFy1sr/YeQukNUTMNV4XUpv048eOxKmmID/94xy0 1YJMvCeVoFPGgCECLpnrJbD93+tBdxBV13GDmB6UVqnRqtw2irDwuF9swzCXS26FP+n8 PW7d23IsJ6XhmfwV/XXS4EkFXnN1twe1a7N/KjBuxJvfjU+5yxZgequHOqGibbBql23j Uy4XH0SoTWCN2Wf+IlB6wMBUX1f89eBbRf1NXYw4V4RO8USqrq21HlUU40z5wXNVueNh 1r/FyA95cCOOfTJR2WBHCApdeE1xjuBUER3kpU2oP7+RL46X61tXqfbWZ6fZ+us/fQ/m XpKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GAPw2+g8; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.04.57; Mon, 01 Feb 2021 02:04:57 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GAPw2+g8; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232508AbhBAKDg (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233009AbhBAKDf (ORCPT ); Mon, 1 Feb 2021 05:03:35 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA80AC061793 for ; Mon, 1 Feb 2021 02:02:22 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id s7so12882235wru.5 for ; Mon, 01 Feb 2021 02:02:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Gg480XhfMoGXqP/teEw7Tq4vypgx+NgStVfH6LwTrK4=; b=GAPw2+g8hF10x6QLkaCRtPjUnas1YfgHqJSLyJkmj6r0J1kybPL+9KQdOCkc27eJnS YntJQMcPex5g0d+Y9o6ptI6hNfKmW9JIZb9x+aVm/6SSZ0vsumNlVMGboAk5EDg84g4I kR71FAKBtou87mbd5Z8yha3PDgvjXUX34dbHgLGgfLBie/ncoxdf9c9Iva4pOGwsQ65X z2gjH5OCxeMNSfLyxWl2viXFDH8TfsZgt91NvfXWji8H4VwsW7gIb2D9y595BwIthGql WUI62EE9IPaJGORSm1qBe23IbqNfy3jk3AayjMjIpDDQfpRK1FLc9P0nG9l0o2Bx1c/5 vfXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Gg480XhfMoGXqP/teEw7Tq4vypgx+NgStVfH6LwTrK4=; b=R3XYKTm5vNVvdYIx8o9w3cIJfY6L95tdQLZiDE3tSOzvXSDuvkxjF4glCPqlC6cINZ MFzh8t17Ju2n1N0JhLzNPnFeLLvi5CPCw3Y7znIv5yXVYCWCUmjq7SvUI2NkH9WYgFJq qW2P/K/gGpKw479oBYjilCpjzHxuBQIUNwiYwttEgWwzXjNf1eORB36cJgwSM0nLKDe0 HkBtvLM8L7v+HUP/jWOEO5mB37u9FisakmksfBgKivoqR6F5LhURrDqkKg6czB9Brm1c iDckrUiCTavbq6rDH22Cs9l4C3FyEIEqo2ugqztXY/UWu3taikph9DHB4JK9YQbwVvy5 Y56g== X-Gm-Message-State: AOAM5319/cjBE+ehKX0XybQiHch2/HqqQnDoRwV0zrX8rAiMNzMwoTUu qHm8WePcbkvoiN0AHkuvBdWZkpyUub0bFXYq X-Received: by 2002:adf:eccc:: with SMTP id s12mr10973223wro.383.1612173741148; Mon, 01 Feb 2021 02:02:21 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:20 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 09/12] futex: Provide state handling for exec() as well Date: Mon, 1 Feb 2021 10:01:40 +0000 Message-Id: <20210201100143.2028618-10-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit af8cbda2cfcaa5515d61ec500498d46e9a8247e2 upstream. exec() attempts to handle potentially held futexes gracefully by running the futex exit handling code like exit() does. The current implementation has no protection against concurrent incoming waiters. The reason is that the futex state cannot be set to FUTEX_STATE_DEAD after the cleanup because the task struct is still active and just about to execute the new binary. While its arguably buggy when a task holds a futex over exec(), for consistency sake the state handling can at least cover the actual futex exit cleanup section. This provides state consistency protection accross the cleanup. As the futex state of the task becomes FUTEX_STATE_OK after the cleanup has been finished, this cannot prevent subsequent attempts to attach to the task in case that the cleanup was not successfull in mopping up all leftovers. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.753355618@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index f1e8ba64fe8ae..50f61d0e51b59 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3269,7 +3269,7 @@ static void exit_robust_list(struct task_struct *curr) curr, pip); } -void futex_exec_release(struct task_struct *tsk) +static void futex_cleanup(struct task_struct *tsk) { if (unlikely(tsk->robust_list)) { exit_robust_list(tsk); @@ -3309,7 +3309,7 @@ void futex_exit_recursive(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_DEAD; } -void futex_exit_release(struct task_struct *tsk) +static void futex_cleanup_begin(struct task_struct *tsk) { /* * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. @@ -3325,10 +3325,40 @@ void futex_exit_release(struct task_struct *tsk) raw_spin_lock_irq(&tsk->pi_lock); tsk->futex_state = FUTEX_STATE_EXITING; raw_spin_unlock_irq(&tsk->pi_lock); +} - futex_exec_release(tsk); +static void futex_cleanup_end(struct task_struct *tsk, int state) +{ + /* + * Lockless store. The only side effect is that an observer might + * take another loop until it becomes visible. + */ + tsk->futex_state = state; +} - tsk->futex_state = FUTEX_STATE_DEAD; +void futex_exec_release(struct task_struct *tsk) +{ + /* + * The state handling is done for consistency, but in the case of + * exec() there is no way to prevent futher damage as the PID stays + * the same. But for the unlikely and arguably buggy case that a + * futex is held on exec(), this provides at least as much state + * consistency protection which is possible. + */ + futex_cleanup_begin(tsk); + futex_cleanup(tsk); + /* + * Reset the state to FUTEX_STATE_OK. The task is alive and about + * exec a new binary. + */ + futex_cleanup_end(tsk, FUTEX_STATE_OK); +} + +void futex_exit_release(struct task_struct *tsk) +{ + futex_cleanup_begin(tsk); + futex_cleanup(tsk); + futex_cleanup_end(tsk, FUTEX_STATE_DEAD); } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 10:01:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374227 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120324jah; Mon, 1 Feb 2021 02:04:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJyiSHK3XtOlTOAys3GucmOyd8iBaRycicnhDr4mOl2gpTMKr6+stp6Gts3KURODpj+CNbn7 X-Received: by 2002:a05:6402:b6f:: with SMTP id cb15mr17704131edb.277.1612173897825; Mon, 01 Feb 2021 02:04:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173897; cv=none; d=google.com; s=arc-20160816; b=Ut52qIzb/AYRrTDgFPFdSRb8xuBOT3J8pbDdVYTpUzdEPlwUkAz3cU+O6adIgUBXTq VogTZMEtfkZPv9+WzYO8soP1uuMkqvC8gmvl3MXMk1DZ6Y3FKYBz7a480STDTYiLtu4u A8mXwLKZKHgqaIDn3gcqPTbAu20myhgaYWZ28TanOnvjsQVZejh2dvmR6Sd1Jbzd6AXu ZXGWVDGz6V7Kc4QQifNMu/G+EQdN7Wvg3l7A7OkWKOFGpRKA3MGOz1bGdci03BhloaSx HUBsA0+m+DTQrd6zkVEg+FBRhnVmZ6cNCTv6Kb04qaRGbg0pP6XUrkBuO6b85sn38AJk 1xDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aVoAzw89+a17gR7Osu1Lbh2IFkjRoOStrGb7pIoW1Us=; b=krIP0RePAvW2eIDNCh5SLr0i2dKlxtqsF+OXNhIMEjXMEdC4IrKg3N6Z/CLI8jdxPn iFHZAzj/o/JUts0NZ2X4mTcjKqpSp7iqTOk7iKyhkJTsp/6iHHYP3hGTf9nWkf9mf0xa Qwis7pP77TikTslv5/xojQ3f3Ci87LZZta81sFKTaJbbHES/1JCOAdl4bm92Sr6NxP5r Y1G2PGT36DJL3C9pYpaQ22JFEbcoLFeKhXoogr3CJDNuf9rlTTIAV4hPY0zX07J6t/sT HWGJX/FstMAwQJAwDNhYADNZMCy9N5MVgjhiZpjB0IOjRjfPLNaaae7w4VcJG2dQbwOC uNdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rT0MzlDN; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.04.57; Mon, 01 Feb 2021 02:04:57 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rT0MzlDN; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232752AbhBAKDg (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233011AbhBAKDf (ORCPT ); Mon, 1 Feb 2021 05:03:35 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F1EEC061794 for ; Mon, 1 Feb 2021 02:02:24 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id g10so15885102wrx.1 for ; Mon, 01 Feb 2021 02:02:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aVoAzw89+a17gR7Osu1Lbh2IFkjRoOStrGb7pIoW1Us=; b=rT0MzlDNrgArkLuFqncWDG0LBLW5ELPTy8ULBD15CvQRx8+gcP9t/8VSy4EdLromj5 VLWa0yGdxhZvLzYGMBGTxARnLG7JRg0pO/diAysLoxXlkwIluPn/TxfajZNlK9OaRZjW Bq0n8CXM5Jr3Wh03b/nmxT46Vb6ADpXcMP7WsSxXsTog9MwJWPvRqDMoBbwk5BtSJp/3 LhbteVvA8AW5Qthp8ahEpGZosxjiK/8CZ1qui/upDqbq7GVzf937jKiuwH0dp8PQaVG8 QNo5CkqMyupCYN5QWguuDrrq/NkhQ1b7PGkZ4L0GLY6lGLZmf7DuL1Q2OX45Q9H/A8CS Pt5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aVoAzw89+a17gR7Osu1Lbh2IFkjRoOStrGb7pIoW1Us=; b=L5VjWkcd2hrlei68RVdGnIZWqbbSGUQC65LZVyit7/xfBycAslqhJ1fZ82pLzK4/Q8 J4zpDYNsPI5WD5jo2QB5J9XOduQS1lKAJO/UUxn8V7RCnj5pH7esqf0g28CLSJtpV7U2 IzkS20cz6x/aOknivkO6KGoqQGLl4gBE8s8lMzajcJI3a5Gu2aTSJO9y3MBDOwpB6KsW giOCW8jiDGsPNK06pq+TzViMasXy8JggA8F/jVSPG7ehbxx3klHAFu+JO7xqtg/mQq6f dHJh7FuSLQmtDbrbu+zAFZYCbCUFJUsWMT+9yGtznHrCuWbTtoQacRIhEYSud5UAYjZl gGbQ== X-Gm-Message-State: AOAM533INe74joDy+eyGZTD2u9xFsQUk6MufbswmEDYQTjsoFF0T7mHS T8Qu6K2u4Kyyfm89n7I5tA8sPF7TLyPhZHRP X-Received: by 2002:adf:e48b:: with SMTP id i11mr17343924wrm.406.1612173742392; Mon, 01 Feb 2021 02:02:22 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:21 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman Subject: [PATCH 10/12] futex: Add mutex around futex exit Date: Mon, 1 Feb 2021 10:01:41 +0000 Message-Id: <20210201100143.2028618-11-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3f186d974826847a07bc7964d79ec4eded475ad9 upstream. The mutex will be used in subsequent changes to replace the busy looping of a waiter when the futex owner is currently executing the exit cleanup to prevent a potential live lock. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.845798895@linutronix.de Signed-off-by: Greg Kroah-Hartman --- include/linux/futex.h | 1 + include/linux/sched.h | 1 + kernel/futex.c | 16 ++++++++++++++++ 3 files changed, 18 insertions(+) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 805508373fcea..0f294ae63c78c 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -70,6 +70,7 @@ static inline void futex_init_task(struct task_struct *tsk) INIT_LIST_HEAD(&tsk->pi_state_list); tsk->pi_state_cache = NULL; tsk->futex_state = FUTEX_STATE_OK; + mutex_init(&tsk->futex_exit_mutex); } void futex_exit_recursive(struct task_struct *tsk); diff --git a/include/linux/sched.h b/include/linux/sched.h index fcbe5904cbd97..f094882822a63 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1815,6 +1815,7 @@ struct task_struct { #endif struct list_head pi_state_list; struct futex_pi_state *pi_state_cache; + struct mutex futex_exit_mutex; unsigned int futex_state; #endif #ifdef CONFIG_PERF_EVENTS diff --git a/kernel/futex.c b/kernel/futex.c index 50f61d0e51b59..e7798ef3b4b71 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3306,11 +3306,22 @@ static void futex_cleanup(struct task_struct *tsk) */ void futex_exit_recursive(struct task_struct *tsk) { + /* If the state is FUTEX_STATE_EXITING then futex_exit_mutex is held */ + if (tsk->futex_state == FUTEX_STATE_EXITING) + mutex_unlock(&tsk->futex_exit_mutex); tsk->futex_state = FUTEX_STATE_DEAD; } static void futex_cleanup_begin(struct task_struct *tsk) { + /* + * Prevent various race issues against a concurrent incoming waiter + * including live locks by forcing the waiter to block on + * tsk->futex_exit_mutex when it observes FUTEX_STATE_EXITING in + * attach_to_pi_owner(). + */ + mutex_lock(&tsk->futex_exit_mutex); + /* * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. * @@ -3334,6 +3345,11 @@ static void futex_cleanup_end(struct task_struct *tsk, int state) * take another loop until it becomes visible. */ tsk->futex_state = state; + /* + * Drop the exit protection. This unblocks waiters which observed + * FUTEX_STATE_EXITING to reevaluate the state. + */ + mutex_unlock(&tsk->futex_exit_mutex); } void futex_exec_release(struct task_struct *tsk) From patchwork Mon Feb 1 10:01:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374228 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120369jah; Mon, 1 Feb 2021 02:05:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJx9o64wR3FYYYsonfkepFkFiXcsRppT3ui7n+JAVuFUTYqgf2QGeF9JtKvWLa5cEu+PTbx+ X-Received: by 2002:a17:906:708f:: with SMTP id b15mr17172467ejk.267.1612173900988; Mon, 01 Feb 2021 02:05:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173900; cv=none; d=google.com; s=arc-20160816; b=ku6BncWBbjzPwUP39orCXlaB+WdiGIKo0jUp7YfNHLDa4GXyD6h0WM5hi5y6D07yDf I2ws/wZQDmQqb+dTLrdnFGGhVa2OYKSH7JH7S5Uu+FDLf30dR4Q6ZeaqHkJKuSoDHtkV dVkOVmolOL4lKfHMHLUNSnWWOOO4tznE2NhybG8JxzW3N15DF4jJHuR+boXdG0IvbkS4 27NpMyi1itg7H4HVYDlTMcenTri7QzV95/uOv+jC9y1INHtTO8vcZU50X6PRrROeK1Z8 kxR1ZT72LDXSzr0cLagrUXWb0ptLPPkFk6bniCWojfFoenCcu+Sio2IckSl7b0017I++ frQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YvIosSfqayR6zcFHOY8c0x/JdWmP1VyW0c+N7TdenS0=; b=M3HeYltmzjSnYwE0YWE52/UytAcBh5Om+kMAOAVO7AR7SWwb+7ZJGnXZ7HWw29oj1s YOCHvIogtgw+aNVYELJmZF5urWs9GmHNzLZNzCxwY15nn+8McmS9prjIH3xpoXzfn/b8 IS/BtuYCI8R/t2uZpTk/TsZ+ghFKYT7q5Y9f2UM+EY/ZE+3uiEfldKlRAzWHe7V0nMS1 9iVov9+iPqL1JFUwk5ypDwDhCrVYHCXqalzsEDHHd+5Me9KNugbJ31liM+4OZFuTe7ri X7Qnj8sjIux+bNkiMJojDe9qi8ZTsAjIu2lDlWf8Gvq3oTh5Eiuto9VFFtwJeloHb6Nl whcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dEO9Jyac; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.05.00; Mon, 01 Feb 2021 02:05:00 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dEO9Jyac; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233020AbhBAKDn (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233011AbhBAKDm (ORCPT ); Mon, 1 Feb 2021 05:03:42 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CC22C0617A7 for ; Mon, 1 Feb 2021 02:02:25 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id s7so12882436wru.5 for ; Mon, 01 Feb 2021 02:02:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YvIosSfqayR6zcFHOY8c0x/JdWmP1VyW0c+N7TdenS0=; b=dEO9JyacwmaNh722rkMqymWjpemauzfTZo9g6zy6O3h0+aVdJI0fT0egpN0lkOapR4 dEbSVoXXQLVegZrXuhI0JW/mzj/vGy1TE0lf9YVPfvApAkUO7AnULl3X0QkLmwnLgI5D 0QGKjd2uWNOTg5j3ll5of6xK00r5v3uEkqS1Ki3nN5VaaeHKL7Pr2UtiBvH1urnskihx bu0lC9kBMyiVCMLtXel/iNjToeKnGtQmoDPpnBK8wtGKuOCrmnQPVekKCn2zvVcPwFhs 9V3tUg2xpFZq89ABmzrKo2ZTkbdbsod2B4ULRnyA7qFX6yEUcsGpiR6vckmzpMD7+NrI 4fsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YvIosSfqayR6zcFHOY8c0x/JdWmP1VyW0c+N7TdenS0=; b=SR9E4uEZxOkVROZNfQsoANmhP+4K2f0Aa1dqGluGeOvl3g1yA0HovX+zWVdpZxXdyv YxAY2GF9BLr+3u0xV7eNCn+R0vqSv2PF7y8lVHfPHCej3UzlokvIJYv7+Y0rfB3ly9Yb yJrVANA+iYoEPFvIYIqz/ejLoOhDYj3NPArmUU/aiMJR0m5sC3qqk1ORLIRuRATkue1M rWMCG6VyJDgXOwh46GhTb8KmL7TGu9ovWeFM7CID1PSpmR7+1Z5TJP2DIOTlOYGHfCoQ hvW1PxbfL1agEqcs7LVsmNXV91FNNmWaB9O5uuffLfWJjD0YZmCx8UHJSlYt7sN+mQOs gXiQ== X-Gm-Message-State: AOAM533icZ0gHSwb4BSbzATSAlnWXwOgx7c2JcV5mLZTr/gFQahoObpr +ki8H3hQ2IMJ83MNiDf9MOiyQZ5IeDfglojF X-Received: by 2002:a5d:4f84:: with SMTP id d4mr16951524wru.374.1612173743342; Mon, 01 Feb 2021 02:02:23 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:22 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 11/12] futex: Provide distinct return value when owner is exiting Date: Mon, 1 Feb 2021 10:01:42 +0000 Message-Id: <20210201100143.2028618-12-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit ac31c7ff8624409ba3c4901df9237a616c187a5d upstream. attach_to_pi_owner() returns -EAGAIN for various cases: - Owner task is exiting - Futex value has changed The caller drops the held locks (hash bucket, mmap_sem) and retries the operation. In case of the owner task exiting this can result in a live lock. As a preparatory step for seperating those cases, provide a distinct return value (EBUSY) for the owner exiting case. No functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.935606117@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index e7798ef3b4b71..cc4590d9fe645 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1918,12 +1918,13 @@ static int futex_requeue(u32 __user *uaddr1, unsigned int flags, if (!ret) goto retry; goto out; + case -EBUSY: case -EAGAIN: /* * Two reasons for this: - * - Owner is exiting and we just wait for the + * - EBUSY: Owner is exiting and we just wait for the * exit to complete. - * - The user space value changed. + * - EAGAIN: The user space value changed. */ double_unlock_hb(hb1, hb2); hb_waiters_dec(hb2); @@ -2615,12 +2616,13 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, goto out_unlock_put_key; case -EFAULT: goto uaddr_faulted; + case -EBUSY: case -EAGAIN: /* * Two reasons for this: - * - Task is exiting and we just wait for the + * - EBUSY: Task is exiting and we just wait for the * exit to complete. - * - The user space value changed. + * - EAGAIN: The user space value changed. */ queue_unlock(hb); put_futex_key(&q.key); From patchwork Mon Feb 1 10:01:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374230 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1120377jah; Mon, 1 Feb 2021 02:05:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJyBs/XstYwQ2Zr+C1T6Vx0bGkIgH5KlE1H7GxdNAEsGILYLGgFabDLrMKG0QArcGhY2AOgE X-Received: by 2002:a05:6402:270e:: with SMTP id y14mr17613744edd.322.1612173901523; Mon, 01 Feb 2021 02:05:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612173901; cv=none; d=google.com; s=arc-20160816; b=JIau3nphLciUJLzN7dM+19Tub6bP56lSAIsiySwwQtwdXviXI74zeT45IysV9umaOK E3B6bMcR3uxbiyO1CQ5ke/mWD6iMaqW/FwRMrQPl55ajbDkB9hbV0QLjP1dgV/nGrcOX IDBrx+ZIIESKirVYT6dk6xD5C2yis7TC7KIGK4WBOdfx5xk+/a5wmagk8AyXnqUHePVU Wl2A4HjuQTydA9NpyUTJZRheRww2KsACI3PEu3sUP0YnSBZJCXQFznGqhuRrO4KwHrcC SXWwpnfFOdpvN/nBFIHeSPUomQ7k0nU2Ign0ejTMbji2/cl2YTsM/QGrAR48xI4M5CtS w4sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=GHgs52PMT6uUIxDpX89DS1Pb97XITDMq7XtRSlRACwM=; b=KgUWR+jt+/byMn3G0DEwajgbJ09aAxFLm/Eo/l9wNvg8dgz/dM0491XV/Gln9J7Rdj pCjOC6hJH1zFuHGX/U7Gb6Yxpu7pSUQZGMfRPP9gbZtI1utYvJyta1ltg1Riprr+wE7I v2GdLuyem2IrhWZG6OF55H0FWOBil1v6+3dLzVxfBYMH2Bpfte8VhVtd3Tvozgn83mhv 0MuX8t4mYU0fTWo436ndmwLAbnJZSoE3BGJwSCk6hKQPJxFTM4ncqQ/20BO+/bXA/NdF I9Rs8hkvZ6lFMVYwXz6fm8UooYlrP5h+YXcU9fI4whgICA9gwTPgCwvQi40qJoMum8x+ BD8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tOY33Uyb; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m6si13120787ejl.643.2021.02.01.02.05.01; Mon, 01 Feb 2021 02:05:01 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tOY33Uyb; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232937AbhBAKDt (ORCPT + 13 others); Mon, 1 Feb 2021 05:03:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233011AbhBAKDq (ORCPT ); Mon, 1 Feb 2021 05:03:46 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 140D1C0617A9 for ; Mon, 1 Feb 2021 02:02:27 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id u14so12060011wml.4 for ; Mon, 01 Feb 2021 02:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GHgs52PMT6uUIxDpX89DS1Pb97XITDMq7XtRSlRACwM=; b=tOY33Uyb2PVFfa0kO0YeTB0uYBJc6d9/9LzmUz64em3L5ie94qiF2iELRusmocEU6V SS4kRAtLVeR2HiQzapjQBy4myVusJE5eC+DQ5MjkwTbU0SzHBYf4ii+Rk2+Mny1g4lT4 UN/nWM8GFtkm3or+9pp32CwhvFwe5ylVicIsBSWoHkVeHyRTSt7CX9Tng5rwtxH6ITGu /Pz9LEKs38gRVbJQ0Tl4hD5AVqTLBIJXsN3fljbWcwu1G575CqMOvhMMSTTD+3CSVXsC RwKanqssv5H3Irprfo35tfvAa4DAduqzpd+UtIAe2rJUTHZAteNaEbTiqKraE7JT+xq3 m7gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GHgs52PMT6uUIxDpX89DS1Pb97XITDMq7XtRSlRACwM=; b=XNBDVmefJQ4bIpvs1bCcBjdw+Vys843EMR9Vk01skfgbk18kftkvR1xKc0ESWp6ghs AcjK6ZcKV3/Esx0lw/DeVgjLL35J+RiouUqjFIO8+Hkyj8LKw65a3f6Cur8/S77M4qVZ LdygnPF0G90WAMeHlahh5bM/l0+QSIAMobx7mlVTLF9ljUzbGoOAvRGE5+qMgrgo2yWv UJuGHFRlXaMidfCo2OixCDvlI8zKicCsRVXHqdqL1E/XgIU+zBA3tDETew9zy7t9H7sP 89RemL59oi8nV+FVssNRiqYdUYStXgq3KMb5pzw99fVipUPYKXbO/jUVYS6jbP8KkK8o hMZA== X-Gm-Message-State: AOAM530/ISixXj4zcr3Er9l25jFC8ti0SyeAvtZOryAs2UGnDTpHyoEY Jcpbup/+GeOVl1LdlDVUVkgQcCTGxbtOaHdV X-Received: by 2002:a1c:32c4:: with SMTP id y187mr3044385wmy.120.1612173744903; Mon, 01 Feb 2021 02:02:24 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id p15sm26151387wrt.15.2021.02.01.02.02.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 02:02:24 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Oleg Nesterov , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 12/12] futex: Prevent exit livelock Date: Mon, 1 Feb 2021 10:01:43 +0000 Message-Id: <20210201100143.2028618-13-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201100143.2028618-1-lee.jones@linaro.org> References: <20210201100143.2028618-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3ef240eaff36b8119ac9e2ea17cbf41179c930ba upstream. Oleg provided the following test case: int main(void) { struct sched_param sp = {}; sp.sched_priority = 2; assert(sched_setscheduler(0, SCHED_FIFO, &sp) == 0); int lock = vfork(); if (!lock) { sp.sched_priority = 1; assert(sched_setscheduler(0, SCHED_FIFO, &sp) == 0); _exit(0); } syscall(__NR_futex, &lock, FUTEX_LOCK_PI, 0,0,0); return 0; } This creates an unkillable RT process spinning in futex_lock_pi() on a UP machine or if the process is affine to a single CPU. The reason is: parent child set FIFO prio 2 vfork() -> set FIFO prio 1 implies wait_for_child() sched_setscheduler(...) exit() do_exit() .... mm_release() tsk->futex_state = FUTEX_STATE_EXITING; exit_futex(); (NOOP in this case) complete() --> wakes parent sys_futex() loop infinite because tsk->futex_state == FUTEX_STATE_EXITING The same problem can happen just by regular preemption as well: task holds futex ... do_exit() tsk->futex_state = FUTEX_STATE_EXITING; --> preemption (unrelated wakeup of some other higher prio task, e.g. timer) switch_to(other_task) return to user sys_futex() loop infinite as above Just for the fun of it the futex exit cleanup could trigger the wakeup itself before the task sets its futex state to DEAD. To cure this, the handling of the exiting owner is changed so: - A refcount is held on the task - The task pointer is stored in a caller visible location - The caller drops all locks (hash bucket, mmap_sem) and blocks on task::futex_exit_mutex. When the mutex is acquired then the exiting task has completed the cleanup and the state is consistent and can be reevaluated. This is not a pretty solution, but there is no choice other than returning an error code to user space, which would break the state consistency guarantee and open another can of problems including regressions. For stable backports the preparatory commits ac31c7ff8624 .. ba31c1a48538 are required as well, but for anything older than 5.3.y the backports are going to be provided when this hits mainline as the other dependencies for those kernels are definitely not stable material. Fixes: 778e9a9c3e71 ("pi-futex: fix exit races and locking problems") Reported-by: Oleg Nesterov Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Cc: Stable Team Link: https://lkml.kernel.org/r/20191106224557.041676471@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 106 ++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 91 insertions(+), 15 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index cc4590d9fe645..2ef8c5aef35d0 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1072,12 +1072,43 @@ static int attach_to_pi_state(u32 uval, struct futex_pi_state *pi_state, return 0; } +/** + * wait_for_owner_exiting - Block until the owner has exited + * @exiting: Pointer to the exiting task + * + * Caller must hold a refcount on @exiting. + */ +static void wait_for_owner_exiting(int ret, struct task_struct *exiting) +{ + if (ret != -EBUSY) { + WARN_ON_ONCE(exiting); + return; + } + + if (WARN_ON_ONCE(ret == -EBUSY && !exiting)) + return; + + mutex_lock(&exiting->futex_exit_mutex); + /* + * No point in doing state checking here. If the waiter got here + * while the task was in exec()->exec_futex_release() then it can + * have any FUTEX_STATE_* value when the waiter has acquired the + * mutex. OK, if running, EXITING or DEAD if it reached exit() + * already. Highly unlikely and not a problem. Just one more round + * through the futex maze. + */ + mutex_unlock(&exiting->futex_exit_mutex); + + put_task_struct(exiting); +} + /* * Lookup the task for the TID provided from user space and attach to * it after doing proper sanity checks. */ static int attach_to_pi_owner(u32 uval, union futex_key *key, - struct futex_pi_state **ps) + struct futex_pi_state **ps, + struct task_struct **exiting) { pid_t pid = uval & FUTEX_TID_MASK; struct futex_pi_state *pi_state; @@ -1113,7 +1144,19 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, int ret = (p->futex_state = FUTEX_STATE_DEAD) ? -ESRCH : -EAGAIN; raw_spin_unlock_irq(&p->pi_lock); - put_task_struct(p); + /* + * If the owner task is between FUTEX_STATE_EXITING and + * FUTEX_STATE_DEAD then store the task pointer and keep + * the reference on the task struct. The calling code will + * drop all locks, wait for the task to reach + * FUTEX_STATE_DEAD and then drop the refcount. This is + * required to prevent a live lock when the current task + * preempted the exiting task between the two states. + */ + if (ret == -EBUSY) + *exiting = p; + else + put_task_struct(p); return ret; } @@ -1144,7 +1187,8 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, } static int lookup_pi_state(u32 uval, struct futex_hash_bucket *hb, - union futex_key *key, struct futex_pi_state **ps) + union futex_key *key, struct futex_pi_state **ps, + struct task_struct **exiting) { struct futex_q *match = futex_top_waiter(hb, key); @@ -1159,7 +1203,7 @@ static int lookup_pi_state(u32 uval, struct futex_hash_bucket *hb, * We are the first waiter - try to look up the owner based on * @uval and attach to it. */ - return attach_to_pi_owner(uval, key, ps); + return attach_to_pi_owner(uval, key, ps, exiting); } static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) @@ -1185,6 +1229,8 @@ static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) * lookup * @task: the task to perform the atomic lock work for. This will * be "current" except in the case of requeue pi. + * @exiting: Pointer to store the task pointer of the owner task + * which is in the middle of exiting * @set_waiters: force setting the FUTEX_WAITERS bit (1) or not (0) * * Return: @@ -1193,11 +1239,17 @@ static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) * <0 - error * * The hb->lock and futex_key refs shall be held by the caller. + * + * @exiting is only set when the return value is -EBUSY. If so, this holds + * a refcount on the exiting task on return and the caller needs to drop it + * after waiting for the exit to complete. */ static int futex_lock_pi_atomic(u32 __user *uaddr, struct futex_hash_bucket *hb, union futex_key *key, struct futex_pi_state **ps, - struct task_struct *task, int set_waiters) + struct task_struct *task, + struct task_struct **exiting, + int set_waiters) { u32 uval, newval, vpid = task_pid_vnr(task); struct futex_q *match; @@ -1267,7 +1319,7 @@ static int futex_lock_pi_atomic(u32 __user *uaddr, struct futex_hash_bucket *hb, * attach to the owner. If that fails, no harm done, we only * set the FUTEX_WAITERS bit in the user space variable. */ - return attach_to_pi_owner(uval, key, ps); + return attach_to_pi_owner(uval, key, ps, exiting); } /** @@ -1693,6 +1745,8 @@ void requeue_pi_wake_futex(struct futex_q *q, union futex_key *key, * @key1: the from futex key * @key2: the to futex key * @ps: address to store the pi_state pointer + * @exiting: Pointer to store the task pointer of the owner task + * which is in the middle of exiting * @set_waiters: force setting the FUTEX_WAITERS bit (1) or not (0) * * Try and get the lock on behalf of the top waiter if we can do it atomically. @@ -1700,16 +1754,20 @@ void requeue_pi_wake_futex(struct futex_q *q, union futex_key *key, * then direct futex_lock_pi_atomic() to force setting the FUTEX_WAITERS bit. * hb1 and hb2 must be held by the caller. * + * @exiting is only set when the return value is -EBUSY. If so, this holds + * a refcount on the exiting task on return and the caller needs to drop it + * after waiting for the exit to complete. + * * Return: * 0 - failed to acquire the lock atomically; * >0 - acquired the lock, return value is vpid of the top_waiter * <0 - error */ -static int futex_proxy_trylock_atomic(u32 __user *pifutex, - struct futex_hash_bucket *hb1, - struct futex_hash_bucket *hb2, - union futex_key *key1, union futex_key *key2, - struct futex_pi_state **ps, int set_waiters) +static int +futex_proxy_trylock_atomic(u32 __user *pifutex, struct futex_hash_bucket *hb1, + struct futex_hash_bucket *hb2, union futex_key *key1, + union futex_key *key2, struct futex_pi_state **ps, + struct task_struct **exiting, int set_waiters) { struct futex_q *top_waiter = NULL; u32 curval; @@ -1746,7 +1804,7 @@ static int futex_proxy_trylock_atomic(u32 __user *pifutex, */ vpid = task_pid_vnr(top_waiter->task); ret = futex_lock_pi_atomic(pifutex, hb2, key2, ps, top_waiter->task, - set_waiters); + exiting, set_waiters); if (ret == 1) { requeue_pi_wake_futex(top_waiter, key2, hb2); return vpid; @@ -1866,6 +1924,8 @@ static int futex_requeue(u32 __user *uaddr1, unsigned int flags, } if (requeue_pi && (task_count - nr_wake < nr_requeue)) { + struct task_struct *exiting = NULL; + /* * Attempt to acquire uaddr2 and wake the top waiter. If we * intend to requeue waiters, force setting the FUTEX_WAITERS @@ -1873,7 +1933,8 @@ static int futex_requeue(u32 __user *uaddr1, unsigned int flags, * faults rather in the requeue loop below. */ ret = futex_proxy_trylock_atomic(uaddr2, hb1, hb2, &key1, - &key2, &pi_state, nr_requeue); + &key2, &pi_state, + &exiting, nr_requeue); /* * At this point the top_waiter has either taken uaddr2 or is @@ -1900,7 +1961,8 @@ static int futex_requeue(u32 __user *uaddr1, unsigned int flags, * If that call succeeds then we have pi_state and an * initial refcount on it. */ - ret = lookup_pi_state(ret, hb2, &key2, &pi_state); + ret = lookup_pi_state(ret, hb2, &key2, + &pi_state, &exiting); } switch (ret) { @@ -1930,6 +1992,12 @@ static int futex_requeue(u32 __user *uaddr1, unsigned int flags, hb_waiters_dec(hb2); put_futex_key(&key2); put_futex_key(&key1); + /* + * Handle the case where the owner is in the middle of + * exiting. Wait for the exit to complete otherwise + * this task might loop forever, aka. live lock. + */ + wait_for_owner_exiting(ret, exiting); cond_resched(); goto retry; default: @@ -2580,6 +2648,7 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int trylock) { struct hrtimer_sleeper timeout, *to = NULL; + struct task_struct *exiting = NULL; struct futex_hash_bucket *hb; struct futex_q q = futex_q_init; int res, ret; @@ -2603,7 +2672,8 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, retry_private: hb = queue_lock(&q); - ret = futex_lock_pi_atomic(uaddr, hb, &q.key, &q.pi_state, current, 0); + ret = futex_lock_pi_atomic(uaddr, hb, &q.key, &q.pi_state, current, + &exiting, 0); if (unlikely(ret)) { /* * Atomic work succeeded and we got the lock, @@ -2626,6 +2696,12 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, */ queue_unlock(hb); put_futex_key(&q.key); + /* + * Handle the case where the owner is in the middle of + * exiting. Wait for the exit to complete otherwise + * this task might loop forever, aka. live lock. + */ + wait_for_owner_exiting(ret, exiting); cond_resched(); goto retry; default: