From patchwork Mon Feb 1 15:12:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374258 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1381356jah; Mon, 1 Feb 2021 07:14:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJxoDi11q8J/KO4Y/Mahf+FUAoSiwAsXMp2zU2vBqspvJnA5saK1niP8mNjNkR7CYBupQoWQ X-Received: by 2002:aa7:d489:: with SMTP id b9mr19496940edr.374.1612192469131; Mon, 01 Feb 2021 07:14:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192469; cv=none; d=google.com; s=arc-20160816; b=VVHuE14lRAS/pB4wcGCEU7bMnT0wVVqvtDu52IcwyS4ApVRs5P/xx/2YTBUqXLHcPx eggNtPmo6HseOGCJKFMhWXZx35Fw+UGuxPe7gnddwVQ53lnJFNywOr84EUIhov+evJBD 3W/s8LxZ4vM8DqqR5Thf0cBvQkw9yRtVRdpgO5Sn+TjYpI1YtyW/QhG/pWZUmIIQRFC6 0M9z0BK2vZD7LtygYk1XdUhUV0qJg3Pv6VtrjJmH511o4sKStx8j6e7MbFr6UOeq1d2x kUWx6uZJr6Y1pzzMPArn/cAXLIs5g7ehuGvZM6eIB8AHiKQ2digugrgyM8Pv636WvV2K avpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=K5rbjOGkExgtzLhLra22CQVxtXBAO/XwKYwFCVpvkjQ=; b=uVSc/pNTw9cW+5Z5nzs+ph8vqNzD2J6rK/yTeXtaxqtK8CYlzMs7Cj3OGe2M+pRWij hyyDbL98/n4zruedxjmakFG+YeOOW+xfaub4XxUOnKci9IEb78bLOSElgHvemfgxeN8s pyh+kZlxOd2Mjev0CCVOZoosiiuvg7c1NFg/YeLDBELlyE/eDWjqCBwO/DH/DK/84TYl FI1D4VwdQH78/P69v2zvCF3yGJs6Obf9NbaRiG0aIDkQ1in/4U7HGA71I3Br9sZd41xl tkDKyqwWt1ckxhzicPyExKI+jY7LXfUhIRfWUbPCTItC0KXgddseACZrCFAOciueoOXE ZIeA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E6l16lbF; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o17si11346999edz.582.2021.02.01.07.14.28; Mon, 01 Feb 2021 07:14:29 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E6l16lbF; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231319AbhBAPOX (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231362AbhBAPOQ (ORCPT ); Mon, 1 Feb 2021 10:14:16 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8002C0613ED for ; Mon, 1 Feb 2021 07:13:35 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id d16so16928605wro.11 for ; Mon, 01 Feb 2021 07:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K5rbjOGkExgtzLhLra22CQVxtXBAO/XwKYwFCVpvkjQ=; b=E6l16lbFXzc3E5GVrhq8Gw1zEzI43yDD2QuHQRxlyJpBukoJxB1VL40A82i1xttjUc wWKSeODyfxqzO17MgmyED8TVG9YYb9cjyoVJZVUH85BjlVvjcPVxAMneDJ6ohwKNZmVw hLI+nRNnChA2mC874CJqqS6qS80TamSXjFBoWlAZxirXs8Uyz/m+9GdPpWZxPIrW8zjF glIm+VJ1fGomSxfiggsYen6RUgLSzvJBRlw6PLlFa8sZWpzW/d4jIdSTomWxPvpdDHQO Hkv/xz7sWO9+FAE039ccEO5vyGKgYiMo2p//infBpdrnPfTl/kIB5OGwk8+auaLswIX7 C7zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K5rbjOGkExgtzLhLra22CQVxtXBAO/XwKYwFCVpvkjQ=; b=dxWe7RAPAqjqHRpA8NqFseACM3KHN0BZ2itpr9PA3sEaW7UVHiD2teAHH3G3iUkqOg BrL9AiDcqLqVqj0ad+kO1UMepl5+TtxsiqaH226pIiYn0b3+2Hv2y3kbD6h69ASkRP0h MJC2msMSCrE9mXBZna+T4wFK6Ei+ItSgTE3lcbAQ+H6g0FD+XLnfm0s4eTcLfuLRFxqV KMgMlDBcWYdbykTvkEVM0t90iZ1yVZPrm5/Rae+Z1aQN3xfB/01xEctqYVdVPDBRGzRA OAZQPh/DS3h5Sryg8sqLsXsU+qx12gbX+ub9c+SvVdGW3h27KKPLDhUCShH53pmaaprN j0MQ== X-Gm-Message-State: AOAM530IwvipQGxqmfubUUhTNe1vO9bvj0ai+bdEpVDyMtzv7nDOrMxY 3mfwKTXespKVIPJvRXLz1MasmTVC7J4HF8p1 X-Received: by 2002:a5d:68cc:: with SMTP id p12mr18044054wrw.276.1612192414152; Mon, 01 Feb 2021 07:13:34 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:33 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Arnd Bergmann , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 01/12] y2038: futex: Move compat implementation into futex.c Date: Mon, 1 Feb 2021 15:12:03 +0000 Message-Id: <20210201151214.2193508-2-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Arnd Bergmann commit 04e7712f4460585e5eed5b853fd8b82a9943958f upstream. We are going to share the compat_sys_futex() handler between 64-bit architectures and 32-bit architectures that need to deal with both 32-bit and 64-bit time_t, and this is easier if both entry points are in the same file. In fact, most other system call handlers do the same thing these days, so let's follow the trend here and merge all of futex_compat.c into futex.c. In the process, a few minor changes have to be done to make sure everything still makes sense: handle_futex_death() and futex_cmpxchg_enabled() become local symbol, and the compat version of the fetch_robust_entry() function gets renamed to compat_fetch_robust_entry() to avoid a symbol clash. This is intended as a purely cosmetic patch, no behavior should change. Signed-off-by: Arnd Bergmann Signed-off-by: Greg Kroah-Hartman [Lee: Back-ported to satisfy a build dependency] Signed-off-by: Lee Jones --- include/linux/futex.h | 8 -- kernel/Makefile | 3 - kernel/futex.c | 195 +++++++++++++++++++++++++++++++++++++++- kernel/futex_compat.c | 201 ------------------------------------------ 4 files changed, 192 insertions(+), 215 deletions(-) delete mode 100644 kernel/futex_compat.c -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index c015fa91e7cce..fb4e12cbe887e 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -11,9 +11,6 @@ union ktime; long do_futex(u32 __user *uaddr, int op, u32 val, union ktime *timeout, u32 __user *uaddr2, u32 val2, u32 val3); -extern int -handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi); - /* * Futexes are matched on equal values of this key. * The key type depends on whether it's a shared or private mapping. @@ -58,11 +55,6 @@ union futex_key { #ifdef CONFIG_FUTEX extern void exit_robust_list(struct task_struct *curr); extern void exit_pi_state_list(struct task_struct *curr); -#ifdef CONFIG_HAVE_FUTEX_CMPXCHG -#define futex_cmpxchg_enabled 1 -#else -extern int futex_cmpxchg_enabled; -#endif #else static inline void exit_robust_list(struct task_struct *curr) { diff --git a/kernel/Makefile b/kernel/Makefile index 53abf008ecb39..a672bece1f499 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -36,9 +36,6 @@ obj-$(CONFIG_PROFILING) += profile.o obj-$(CONFIG_STACKTRACE) += stacktrace.o obj-y += time/ obj-$(CONFIG_FUTEX) += futex.o -ifeq ($(CONFIG_COMPAT),y) -obj-$(CONFIG_FUTEX) += futex_compat.o -endif obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o obj-$(CONFIG_SMP) += smp.o ifneq ($(CONFIG_SMP),y) diff --git a/kernel/futex.c b/kernel/futex.c index e50b67674ba25..2815b1801ec5a 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -44,6 +44,7 @@ * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +#include #include #include #include @@ -171,8 +172,10 @@ * double_lock_hb() and double_unlock_hb(), respectively. */ -#ifndef CONFIG_HAVE_FUTEX_CMPXCHG -int __read_mostly futex_cmpxchg_enabled; +#ifdef CONFIG_HAVE_FUTEX_CMPXCHG +#define futex_cmpxchg_enabled 1 +#else +static int __read_mostly futex_cmpxchg_enabled; #endif /* @@ -3088,7 +3091,7 @@ err_unlock: * Process a futex-list entry, check whether it's owned by the * dying task, and do notification if so: */ -int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi) +static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi) { u32 uval, uninitialized_var(nval), mval; @@ -3318,6 +3321,192 @@ SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); } +#ifdef CONFIG_COMPAT +/* + * Fetch a robust-list pointer. Bit 0 signals PI futexes: + */ +static inline int +compat_fetch_robust_entry(compat_uptr_t *uentry, struct robust_list __user **entry, + compat_uptr_t __user *head, unsigned int *pi) +{ + if (get_user(*uentry, head)) + return -EFAULT; + + *entry = compat_ptr((*uentry) & ~1); + *pi = (unsigned int)(*uentry) & 1; + + return 0; +} + +static void __user *futex_uaddr(struct robust_list __user *entry, + compat_long_t futex_offset) +{ + compat_uptr_t base = ptr_to_compat(entry); + void __user *uaddr = compat_ptr(base + futex_offset); + + return uaddr; +} + +/* + * Walk curr->robust_list (very carefully, it's a userspace list!) + * and mark any locks found there dead, and notify any waiters. + * + * We silently return on any sign of list-walking problem. + */ +void compat_exit_robust_list(struct task_struct *curr) +{ + struct compat_robust_list_head __user *head = curr->compat_robust_list; + struct robust_list __user *entry, *next_entry, *pending; + unsigned int limit = ROBUST_LIST_LIMIT, pi, pip; + unsigned int uninitialized_var(next_pi); + compat_uptr_t uentry, next_uentry, upending; + compat_long_t futex_offset; + int rc; + + if (!futex_cmpxchg_enabled) + return; + + /* + * Fetch the list head (which was registered earlier, via + * sys_set_robust_list()): + */ + if (compat_fetch_robust_entry(&uentry, &entry, &head->list.next, &pi)) + return; + /* + * Fetch the relative futex offset: + */ + if (get_user(futex_offset, &head->futex_offset)) + return; + /* + * Fetch any possibly pending lock-add first, and handle it + * if it exists: + */ + if (compat_fetch_robust_entry(&upending, &pending, + &head->list_op_pending, &pip)) + return; + + next_entry = NULL; /* avoid warning with gcc */ + while (entry != (struct robust_list __user *) &head->list) { + /* + * Fetch the next entry in the list before calling + * handle_futex_death: + */ + rc = compat_fetch_robust_entry(&next_uentry, &next_entry, + (compat_uptr_t __user *)&entry->next, &next_pi); + /* + * A pending lock might already be on the list, so + * dont process it twice: + */ + if (entry != pending) { + void __user *uaddr = futex_uaddr(entry, futex_offset); + + if (handle_futex_death(uaddr, curr, pi)) + return; + } + if (rc) + return; + uentry = next_uentry; + entry = next_entry; + pi = next_pi; + /* + * Avoid excessively long or circular lists: + */ + if (!--limit) + break; + + cond_resched(); + } + if (pending) { + void __user *uaddr = futex_uaddr(pending, futex_offset); + + handle_futex_death(uaddr, curr, pip); + } +} + +COMPAT_SYSCALL_DEFINE2(set_robust_list, + struct compat_robust_list_head __user *, head, + compat_size_t, len) +{ + if (!futex_cmpxchg_enabled) + return -ENOSYS; + + if (unlikely(len != sizeof(*head))) + return -EINVAL; + + current->compat_robust_list = head; + + return 0; +} + +COMPAT_SYSCALL_DEFINE3(get_robust_list, int, pid, + compat_uptr_t __user *, head_ptr, + compat_size_t __user *, len_ptr) +{ + struct compat_robust_list_head __user *head; + unsigned long ret; + struct task_struct *p; + + if (!futex_cmpxchg_enabled) + return -ENOSYS; + + rcu_read_lock(); + + ret = -ESRCH; + if (!pid) + p = current; + else { + p = find_task_by_vpid(pid); + if (!p) + goto err_unlock; + } + + ret = -EPERM; + if (!ptrace_may_access(p, PTRACE_MODE_READ_REALCREDS)) + goto err_unlock; + + head = p->compat_robust_list; + rcu_read_unlock(); + + if (put_user(sizeof(*head), len_ptr)) + return -EFAULT; + return put_user(ptr_to_compat(head), head_ptr); + +err_unlock: + rcu_read_unlock(); + + return ret; +} + +COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, + struct compat_timespec __user *, utime, u32 __user *, uaddr2, + u32, val3) +{ + struct timespec ts; + ktime_t t, *tp = NULL; + int val2 = 0; + int cmd = op & FUTEX_CMD_MASK; + + if (utime && (cmd == FUTEX_WAIT || cmd == FUTEX_LOCK_PI || + cmd == FUTEX_WAIT_BITSET || + cmd == FUTEX_WAIT_REQUEUE_PI)) { + if (compat_get_timespec(&ts, utime)) + return -EFAULT; + if (!timespec_valid(&ts)) + return -EINVAL; + + t = timespec_to_ktime(ts); + if (cmd == FUTEX_WAIT) + t = ktime_add_safe(ktime_get(), t); + tp = &t; + } + if (cmd == FUTEX_REQUEUE || cmd == FUTEX_CMP_REQUEUE || + cmd == FUTEX_CMP_REQUEUE_PI || cmd == FUTEX_WAKE_OP) + val2 = (int) (unsigned long) utime; + + return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); +} +#endif /* CONFIG_COMPAT */ + static void __init futex_detect_cmpxchg(void) { #ifndef CONFIG_HAVE_FUTEX_CMPXCHG diff --git a/kernel/futex_compat.c b/kernel/futex_compat.c deleted file mode 100644 index 4ae3232e7a28a..0000000000000 --- a/kernel/futex_compat.c +++ /dev/null @@ -1,201 +0,0 @@ -/* - * linux/kernel/futex_compat.c - * - * Futex compatibililty routines. - * - * Copyright 2006, Red Hat, Inc., Ingo Molnar - */ - -#include -#include -#include -#include -#include -#include - -#include - - -/* - * Fetch a robust-list pointer. Bit 0 signals PI futexes: - */ -static inline int -fetch_robust_entry(compat_uptr_t *uentry, struct robust_list __user **entry, - compat_uptr_t __user *head, unsigned int *pi) -{ - if (get_user(*uentry, head)) - return -EFAULT; - - *entry = compat_ptr((*uentry) & ~1); - *pi = (unsigned int)(*uentry) & 1; - - return 0; -} - -static void __user *futex_uaddr(struct robust_list __user *entry, - compat_long_t futex_offset) -{ - compat_uptr_t base = ptr_to_compat(entry); - void __user *uaddr = compat_ptr(base + futex_offset); - - return uaddr; -} - -/* - * Walk curr->robust_list (very carefully, it's a userspace list!) - * and mark any locks found there dead, and notify any waiters. - * - * We silently return on any sign of list-walking problem. - */ -void compat_exit_robust_list(struct task_struct *curr) -{ - struct compat_robust_list_head __user *head = curr->compat_robust_list; - struct robust_list __user *entry, *next_entry, *pending; - unsigned int limit = ROBUST_LIST_LIMIT, pi, pip; - unsigned int uninitialized_var(next_pi); - compat_uptr_t uentry, next_uentry, upending; - compat_long_t futex_offset; - int rc; - - if (!futex_cmpxchg_enabled) - return; - - /* - * Fetch the list head (which was registered earlier, via - * sys_set_robust_list()): - */ - if (fetch_robust_entry(&uentry, &entry, &head->list.next, &pi)) - return; - /* - * Fetch the relative futex offset: - */ - if (get_user(futex_offset, &head->futex_offset)) - return; - /* - * Fetch any possibly pending lock-add first, and handle it - * if it exists: - */ - if (fetch_robust_entry(&upending, &pending, - &head->list_op_pending, &pip)) - return; - - next_entry = NULL; /* avoid warning with gcc */ - while (entry != (struct robust_list __user *) &head->list) { - /* - * Fetch the next entry in the list before calling - * handle_futex_death: - */ - rc = fetch_robust_entry(&next_uentry, &next_entry, - (compat_uptr_t __user *)&entry->next, &next_pi); - /* - * A pending lock might already be on the list, so - * dont process it twice: - */ - if (entry != pending) { - void __user *uaddr = futex_uaddr(entry, futex_offset); - - if (handle_futex_death(uaddr, curr, pi)) - return; - } - if (rc) - return; - uentry = next_uentry; - entry = next_entry; - pi = next_pi; - /* - * Avoid excessively long or circular lists: - */ - if (!--limit) - break; - - cond_resched(); - } - if (pending) { - void __user *uaddr = futex_uaddr(pending, futex_offset); - - handle_futex_death(uaddr, curr, pip); - } -} - -COMPAT_SYSCALL_DEFINE2(set_robust_list, - struct compat_robust_list_head __user *, head, - compat_size_t, len) -{ - if (!futex_cmpxchg_enabled) - return -ENOSYS; - - if (unlikely(len != sizeof(*head))) - return -EINVAL; - - current->compat_robust_list = head; - - return 0; -} - -COMPAT_SYSCALL_DEFINE3(get_robust_list, int, pid, - compat_uptr_t __user *, head_ptr, - compat_size_t __user *, len_ptr) -{ - struct compat_robust_list_head __user *head; - unsigned long ret; - struct task_struct *p; - - if (!futex_cmpxchg_enabled) - return -ENOSYS; - - rcu_read_lock(); - - ret = -ESRCH; - if (!pid) - p = current; - else { - p = find_task_by_vpid(pid); - if (!p) - goto err_unlock; - } - - ret = -EPERM; - if (!ptrace_may_access(p, PTRACE_MODE_READ_REALCREDS)) - goto err_unlock; - - head = p->compat_robust_list; - rcu_read_unlock(); - - if (put_user(sizeof(*head), len_ptr)) - return -EFAULT; - return put_user(ptr_to_compat(head), head_ptr); - -err_unlock: - rcu_read_unlock(); - - return ret; -} - -COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, - struct compat_timespec __user *, utime, u32 __user *, uaddr2, - u32, val3) -{ - struct timespec ts; - ktime_t t, *tp = NULL; - int val2 = 0; - int cmd = op & FUTEX_CMD_MASK; - - if (utime && (cmd == FUTEX_WAIT || cmd == FUTEX_LOCK_PI || - cmd == FUTEX_WAIT_BITSET || - cmd == FUTEX_WAIT_REQUEUE_PI)) { - if (compat_get_timespec(&ts, utime)) - return -EFAULT; - if (!timespec_valid(&ts)) - return -EINVAL; - - t = timespec_to_ktime(ts); - if (cmd == FUTEX_WAIT) - t = ktime_add_safe(ktime_get(), t); - tp = &t; - } - if (cmd == FUTEX_REQUEUE || cmd == FUTEX_CMP_REQUEUE || - cmd == FUTEX_CMP_REQUEUE_PI || cmd == FUTEX_WAKE_OP) - val2 = (int) (unsigned long) utime; - - return do_futex(uaddr, op, val, tp, uaddr2, val2, val3); -} From patchwork Mon Feb 1 15:12:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374260 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1381577jah; Mon, 1 Feb 2021 07:14:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJyy2bY9wt8afSHedJFwZgBiXhfjr5Dj73DIjjyM4+pm9WvyoFInpTJoIYlkBkCV37SU8+D5 X-Received: by 2002:aa7:c94c:: with SMTP id h12mr19633390edt.40.1612192483655; Mon, 01 Feb 2021 07:14:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192483; cv=none; d=google.com; s=arc-20160816; b=t6dYYmGCPhS0fsVI4liiSiYsqC4aY8sPg6HAL7WppCqDh43DUe+rrk5XMotPwiM6WP y3gox/cehoFbDnf36Fu7ietBaiT4jmGfMXy8P0LLCAS0VtjLMkk4rLIm2t8UtnSFA/Yf Te0RY1jYWmvro7rjyoarsg1UDTv9bHlsQr4rVC3mJ1uGWcOizlJs/Qq2V6kRKJ55GZyy BX9zoKFpKuCtycU51sVw8b91Kc0HbyHRjvRa8LRq9dhGKqpHCooMPXEdCAlBP0wifd9F DaIqTG/7ZhRZiDxkGcjoxIcT4gu+Q7kum8EUDCCRQKFg2N/noz3JyP96UBgtRObNByOl IGNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jDC51W2JTzWT110KNMO3wDJiHHG/GCkuWxt8htB7+e0=; b=LfUvK5c6vZZT4Avcb3+C2H0j57oCtjZPba2HQrZ3kCv17+YDkCA84CO5ZOVG03pcH5 XzXW4YAUpQSQTY8bZyZdXvH0cdGXF5waVTGP4m6NfTq/H+LDrDvW3G7paqptK16EoZbk 4aM5+VbK9hRtIkjo4JSU7T7N9OFyFK6/2ArcWejoPLqW9uzvG2JEkW+V1OMoWT+iiJvo uDnWKMqlPPW8icZlUENqv+mukYDuSgLAMY0NdJJrF2ez7X09FzbhLBiCoxJdaVcU/Opl IqbZHhNeZjbMQhKWwjvrYspJwBhNeMY2YUaYi/+I2TTKgO0UrDN9AVoS3Ysn1DAnrWp2 /BwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a55neDiW; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o17si11346999edz.582.2021.02.01.07.14.43; Mon, 01 Feb 2021 07:14:43 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a55neDiW; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbhBAPOb (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231383AbhBAPOR (ORCPT ); Mon, 1 Feb 2021 10:14:17 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 068ECC061786 for ; Mon, 1 Feb 2021 07:13:37 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id u14so13492737wmq.4 for ; Mon, 01 Feb 2021 07:13:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jDC51W2JTzWT110KNMO3wDJiHHG/GCkuWxt8htB7+e0=; b=a55neDiWRBwHIjb18hk+wIAr5faGMbOaOwQKoxCerwEdPgtPsN15fvsZ/0V7kgORwu Jao2vOS4ZlfF/BhdtQtu6HO6tnAa8UYsrqgG/tVL4iF3em61L3yfKln5PKPL82rvPhNx WXNd04ymELKiddR5vCI/d/wDnBzzyYYMhLzk+rz+7OOwBvnCxI/81BEXtCh2F2SAwEiz GL7c/a5XeLBpKvNxYk/bTSH9JA3kIfCFQzTQ96Jj/2+sow7zXV3CVkwlb78hZBFuM8Xl Dbywt7PuKZjDERZ5pvchaqy7f6RSwaslxEaCIwuSV2ogjwvR7djS4l/detB9mxAvYBGf Zb6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jDC51W2JTzWT110KNMO3wDJiHHG/GCkuWxt8htB7+e0=; b=uiplobRC3uLbcux4O8XblRg97H8nEGb92nA5kuc4kAHo7f7y9ZRy7cbkgb7GFMhuyq +dlzIHraZevBETwZLZaT17YFQn67+9a9vofOAyvhRx7eYQtpgJBYJ7PXEsiJWvuRABFB bn6v1FK4t3TowJC3VnTuQPewOijWx1H6Xy7uORKTe4Yi37KeALkB9iJqSnXdGaYJdIEW kyNTaq6B4IcykANBzXmxjI7X1v+M1puoITNMYIVhFUZlq31MGNs4Fzf2bC06dgbzMexK wdr5dt1eQ7SECztoAb65ygBGgwxazR45vF5CdGTBKwL9prTSA/+iThYjmgrJiQjMTVq4 jbDg== X-Gm-Message-State: AOAM533YzqKt6wd3aJU6lVU+1hJcG0CFX4ZsAyatG7ehmAi/wLk+Oieo BwU423kTlYdrcILEYsc/eawzUQSDk/LDNN10 X-Received: by 2002:a1c:2e04:: with SMTP id u4mr63551wmu.79.1612192415316; Mon, 01 Feb 2021 07:13:35 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:34 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 02/12] futex: Move futex exit handling into futex code Date: Mon, 1 Feb 2021 15:12:04 +0000 Message-Id: <20210201151214.2193508-3-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit ba31c1a48538992316cc71ce94fa9cd3e7b427c0 upstream. The futex exit handling is #ifdeffed into mm_release() which is not pretty to begin with. But upcoming changes to address futex exit races need to add more functionality to this exit code. Split it out into a function, move it into futex code and make the various futex exit functions static. Preparatory only and no functional change. Folded build fix from Borislav. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.049705556@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/compat.h | 2 -- include/linux/futex.h | 24 +++++++++++++++++------- kernel/fork.c | 25 +++---------------------- kernel/futex.c | 28 ++++++++++++++++++++++++++-- 4 files changed, 46 insertions(+), 33 deletions(-) -- 2.25.1 diff --git a/include/linux/compat.h b/include/linux/compat.h index a76c9172b2eb0..24dd42910d7c2 100644 --- a/include/linux/compat.h +++ b/include/linux/compat.h @@ -306,8 +306,6 @@ struct compat_kexec_segment; struct compat_mq_attr; struct compat_msgbuf; -extern void compat_exit_robust_list(struct task_struct *curr); - asmlinkage long compat_sys_set_robust_list(struct compat_robust_list_head __user *head, compat_size_t len); diff --git a/include/linux/futex.h b/include/linux/futex.h index fb4e12cbe887e..63d353cedfcde 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -1,6 +1,8 @@ #ifndef _LINUX_FUTEX_H #define _LINUX_FUTEX_H +#include + #include struct inode; @@ -53,14 +55,22 @@ union futex_key { #define FUTEX_KEY_INIT (union futex_key) { .both = { .ptr = 0ULL } } #ifdef CONFIG_FUTEX -extern void exit_robust_list(struct task_struct *curr); -extern void exit_pi_state_list(struct task_struct *curr); -#else -static inline void exit_robust_list(struct task_struct *curr) -{ -} -static inline void exit_pi_state_list(struct task_struct *curr) +static inline void futex_init_task(struct task_struct *tsk) { + tsk->robust_list = NULL; +#ifdef CONFIG_COMPAT + tsk->compat_robust_list = NULL; +#endif + INIT_LIST_HEAD(&tsk->pi_state_list); + tsk->pi_state_cache = NULL; } + +void futex_mm_release(struct task_struct *tsk); + +long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, + u32 __user *uaddr2, u32 val2, u32 val3); +#else +static inline void futex_init_task(struct task_struct *tsk) { } +static inline void futex_mm_release(struct task_struct *tsk) { } #endif #endif diff --git a/kernel/fork.c b/kernel/fork.c index 5d35be1e0913b..d0ab6aff5efdc 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -890,20 +890,7 @@ static int wait_for_vfork_done(struct task_struct *child, void mm_release(struct task_struct *tsk, struct mm_struct *mm) { /* Get rid of any futexes when releasing the mm */ -#ifdef CONFIG_FUTEX - if (unlikely(tsk->robust_list)) { - exit_robust_list(tsk); - tsk->robust_list = NULL; - } -#ifdef CONFIG_COMPAT - if (unlikely(tsk->compat_robust_list)) { - compat_exit_robust_list(tsk); - tsk->compat_robust_list = NULL; - } -#endif - if (unlikely(!list_empty(&tsk->pi_state_list))) - exit_pi_state_list(tsk); -#endif + futex_mm_release(tsk); uprobe_free_utask(tsk); @@ -1511,14 +1498,8 @@ static struct task_struct *copy_process(unsigned long clone_flags, #ifdef CONFIG_BLOCK p->plug = NULL; #endif -#ifdef CONFIG_FUTEX - p->robust_list = NULL; -#ifdef CONFIG_COMPAT - p->compat_robust_list = NULL; -#endif - INIT_LIST_HEAD(&p->pi_state_list); - p->pi_state_cache = NULL; -#endif + futex_init_task(p); + /* * sigaltstack should be cleared when sharing the same VM */ diff --git a/kernel/futex.c b/kernel/futex.c index 2815b1801ec5a..5282b9b8d1ec1 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -331,6 +331,12 @@ static inline bool should_fail_futex(bool fshared) } #endif /* CONFIG_FAIL_FUTEX */ +#ifdef CONFIG_COMPAT +static void compat_exit_robust_list(struct task_struct *curr); +#else +static inline void compat_exit_robust_list(struct task_struct *curr) { } +#endif + static inline void futex_get_mm(union futex_key *key) { atomic_inc(&key->private.mm->mm_count); @@ -889,7 +895,7 @@ static struct task_struct * futex_find_get_task(pid_t pid) * Kernel cleans up PI-state, but userspace is likely hosed. * (Robust-futex cleanup is separate and might save the day for userspace.) */ -void exit_pi_state_list(struct task_struct *curr) +static void exit_pi_state_list(struct task_struct *curr) { struct list_head *next, *head = &curr->pi_state_list; struct futex_pi_state *pi_state; @@ -3166,7 +3172,7 @@ static inline int fetch_robust_entry(struct robust_list __user **entry, * * We silently return on any sign of list-walking problem. */ -void exit_robust_list(struct task_struct *curr) +static void exit_robust_list(struct task_struct *curr) { struct robust_list_head __user *head = curr->robust_list; struct robust_list __user *entry, *next_entry, *pending; @@ -3229,6 +3235,24 @@ void exit_robust_list(struct task_struct *curr) curr, pip); } +void futex_mm_release(struct task_struct *tsk) +{ + if (unlikely(tsk->robust_list)) { + exit_robust_list(tsk); + tsk->robust_list = NULL; + } + +#ifdef CONFIG_COMPAT + if (unlikely(tsk->compat_robust_list)) { + compat_exit_robust_list(tsk); + tsk->compat_robust_list = NULL; + } +#endif + + if (unlikely(!list_empty(&tsk->pi_state_list))) + exit_pi_state_list(tsk); +} + long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3) { From patchwork Mon Feb 1 15:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374261 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1381655jah; Mon, 1 Feb 2021 07:14:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJySYCOaPWovN/g3xFbPjv2ieNIIbCJUKMyALeiSazYM0Cm7XoOLj+Ira84PjtjOmF4mTeNc X-Received: by 2002:aa7:ce87:: with SMTP id y7mr19489088edv.211.1612192488377; Mon, 01 Feb 2021 07:14:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192488; cv=none; d=google.com; s=arc-20160816; b=NVi28CCxLvwltAcFYOWA4MwpAVkURTmjqRF2hMb2lOVZehGDDhGLXxhnsSDK1uOYqH RORrlZ4iRVuev3papSF83WVpjF/fmu0ojGz7i1h5LCN1yr/bgO7uD9sXzcHxvguBsh8L t4Wq8JDDnWZj/nnDhZCDmXMv/s3UlxlnjF0KfJrIzt0ZP4Zqx/+LIKERn8je2/Htz8Mv +bmc1hTnoxBssC9PRTKojOSIo0zuOkWzLqs3Mof/1yKYmuRCGCyZ8sH/RbtZAtIxWcVJ UIEgCMrm/Bv79eMti15tchLGxVMYDXQB6WOzlfy2wPKrbMjrVwfyd4yhE/rwwF2oLp8o /vuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wnSvDcp3TADKO1sLWE7uv6tZLcF/eEHjH7KbFW37keU=; b=V9VtdAvWFtmH+aBSCh0Y/MbA8B+mqrmpimDVhRsSdsZnsldNdTZANT1xOtNtnuKBti p4TKrsffT8FbuLfKV67zEQ6Hdd6so4OvRFh+QjEWd8ghDoLA1adQ8mXeziyPvg3oqyB1 VE+/cF//JiHlCN6o1lrk/I1jTd4gY3b/asN8L8YKCzww2I8AbVteEE8q9d/Bbrm+9WZj 87YGP0UMsmfsDBVMuxQ5BzUnufdPYe0todYPjMh8NaT8+9zpRELaOHwCNSNeNy6d05q2 fjUMKRzN2rZj48X/v+GTNfdmTDmccgfPZTCm3A68HPWLvlpYbtj0xrVebl1juGBZNcyQ Azow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jVYATYp+; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o17si11346999edz.582.2021.02.01.07.14.47; Mon, 01 Feb 2021 07:14:48 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jVYATYp+; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229819AbhBAPOj (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231429AbhBAPOU (ORCPT ); Mon, 1 Feb 2021 10:14:20 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CB64C061788 for ; Mon, 1 Feb 2021 07:13:38 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id z6so16909263wrq.10 for ; Mon, 01 Feb 2021 07:13:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wnSvDcp3TADKO1sLWE7uv6tZLcF/eEHjH7KbFW37keU=; b=jVYATYp+Qhnyao9rODtO7+MkGThui2iMwr40MzsTVdgd3J2PUDlcC0w9wDKCSw06JS 97lTPEqF0zGWuckx8uZfJ+EJVT8nJ+qdGjAU89X8oHbatZgNOJfMAgHRLXxJeqCsdzYn 9McMQkD6KK7nKfGFoeYm4175d+Vmc50oOFQAmA/sGym9ap2+V75c0s4rezvxx89u0WWe PEaCD796UWCitVWOOSevsD8ipsA/RFyEPvCVqwqZBNQ8DLfPE5CbDK4vd/RScuf603Z8 L6EmpbdABudZXr/QeAlnfU6inoavV1gTfgIHQZdaPHbkgGgD8QHYX+CZZjxlaVFqQBxc kW6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wnSvDcp3TADKO1sLWE7uv6tZLcF/eEHjH7KbFW37keU=; b=cnC1xGqEFVNVSRV0eyWr6lr8Jlh+/axJhuJXboajc5NMUolwT/db8OrwZAWfJsgQT0 B34K4lZnKBufb8QBmyW7+ao8g/CS4xgSNiA4WszsBzsmbX4jza7M3t+EGk2O5AjtC2Gv We1lJQSM8DXaVGDfccCiSE/X8+x2HhIB+GsK3ME/xU+PXYlXXmkMg/KRvIc2ehNnL35Y LKdcdnRUtAUl8c2IikEEn4OkNAkvWJw3hxhH+xol1MtlBnQV0qBSdI+AR9ou2vBlnoIX 4kdv5Q4FIZUwO2JKDFJaVHANjd1fwvJdYhUD8T4oLER7XHsJ8dDguiNpe27Tky47skQ7 6ODQ== X-Gm-Message-State: AOAM532KksLxNQNA8A3WzUa+dXK+ut9lGVw7KqJ5CnunPgy9NN5L6zBM b7ChK8yiJzqoeuhy0qsGL940ggUCjsw1YbZi X-Received: by 2002:a05:6000:1141:: with SMTP id d1mr18478922wrx.47.1612192416917; Mon, 01 Feb 2021 07:13:36 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:35 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 03/12] futex: Replace PF_EXITPIDONE with a state Date: Mon, 1 Feb 2021 15:12:05 +0000 Message-Id: <20210201151214.2193508-4-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3d4775df0a89240f671861c6ab6e8d59af8e9e41 upstream. The futex exit handling relies on PF_ flags. That's suboptimal as it requires a smp_mb() and an ugly lock/unlock of the exiting tasks pi_lock in the middle of do_exit() to enforce the observability of PF_EXITING in the futex code. Add a futex_state member to task_struct and convert the PF_EXITPIDONE logic over to the new state. The PF_EXITING dependency will be cleaned up in a later step. This prepares for handling various futex exit issues later. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.149449274@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 34 ++++++++++++++++++++++++++++++++++ include/linux/sched.h | 2 +- kernel/exit.c | 18 ++---------------- kernel/futex.c | 17 ++++++++--------- 4 files changed, 45 insertions(+), 26 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 63d353cedfcde..a0de6fe28e00b 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -55,6 +55,11 @@ union futex_key { #define FUTEX_KEY_INIT (union futex_key) { .both = { .ptr = 0ULL } } #ifdef CONFIG_FUTEX +enum { + FUTEX_STATE_OK, + FUTEX_STATE_DEAD, +}; + static inline void futex_init_task(struct task_struct *tsk) { tsk->robust_list = NULL; @@ -63,6 +68,34 @@ static inline void futex_init_task(struct task_struct *tsk) #endif INIT_LIST_HEAD(&tsk->pi_state_list); tsk->pi_state_cache = NULL; + tsk->futex_state = FUTEX_STATE_OK; +} + +/** + * futex_exit_done - Sets the tasks futex state to FUTEX_STATE_DEAD + * @tsk: task to set the state on + * + * Set the futex exit state of the task lockless. The futex waiter code + * observes that state when a task is exiting and loops until the task has + * actually finished the futex cleanup. The worst case for this is that the + * waiter runs through the wait loop until the state becomes visible. + * + * This has two callers: + * + * - futex_mm_release() after the futex exit cleanup has been done + * + * - do_exit() from the recursive fault handling path. + * + * In case of a recursive fault this is best effort. Either the futex exit + * code has run already or not. If the OWNER_DIED bit has been set on the + * futex then the waiter can take it over. If not, the problem is pushed + * back to user space. If the futex exit code did not run yet, then an + * already queued waiter might block forever, but there is nothing which + * can be done about that. + */ +static inline void futex_exit_done(struct task_struct *tsk) +{ + tsk->futex_state = FUTEX_STATE_DEAD; } void futex_mm_release(struct task_struct *tsk); @@ -72,5 +105,6 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, #else static inline void futex_init_task(struct task_struct *tsk) { } static inline void futex_mm_release(struct task_struct *tsk) { } +static inline void futex_exit_done(struct task_struct *tsk) { } #endif #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index df5f53ea2f86c..bdd41a0127d10 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1704,6 +1704,7 @@ struct task_struct { #endif struct list_head pi_state_list; struct futex_pi_state *pi_state_cache; + unsigned int futex_state; #endif #ifdef CONFIG_PERF_EVENTS struct perf_event_context *perf_event_ctxp[perf_nr_task_contexts]; @@ -2099,7 +2100,6 @@ extern void thread_group_cputime_adjusted(struct task_struct *p, cputime_t *ut, * Per process flags */ #define PF_EXITING 0x00000004 /* getting shut down */ -#define PF_EXITPIDONE 0x00000008 /* pi exit done on shut down */ #define PF_VCPU 0x00000010 /* I'm a virtual CPU */ #define PF_WQ_WORKER 0x00000020 /* I'm a workqueue worker */ #define PF_FORKNOEXEC 0x00000040 /* forked but didn't exec */ diff --git a/kernel/exit.c b/kernel/exit.c index 5c20a32c95392..274a3c3834a15 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -695,16 +695,7 @@ void do_exit(long code) */ if (unlikely(tsk->flags & PF_EXITING)) { pr_alert("Fixing recursive fault but reboot is needed!\n"); - /* - * We can do this unlocked here. The futex code uses - * this flag just to verify whether the pi state - * cleanup has been done or not. In the worst case it - * loops once more. We pretend that the cleanup was - * done as there is no way to return. Either the - * OWNER_DIED bit is set by now or we push the blocked - * task into the wait for ever nirwana as well. - */ - tsk->flags |= PF_EXITPIDONE; + futex_exit_done(tsk); set_current_state(TASK_UNINTERRUPTIBLE); schedule(); } @@ -793,12 +784,7 @@ void do_exit(long code) * Make sure we are holding no locks: */ debug_check_no_locks_held(); - /* - * We can do this unlocked here. The futex code uses this flag - * just to verify whether the pi state cleanup has been done - * or not. In the worst case it loops once more. - */ - tsk->flags |= PF_EXITPIDONE; + futex_exit_done(tsk); if (tsk->io_context) exit_io_context(tsk); diff --git a/kernel/futex.c b/kernel/futex.c index 5282b9b8d1ec1..e531789aa440a 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1094,19 +1094,18 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, } /* - * We need to look at the task state flags to figure out, - * whether the task is exiting. To protect against the do_exit - * change of the task flags, we do this protected by - * p->pi_lock: + * We need to look at the task state to figure out, whether the + * task is exiting. To protect against the change of the task state + * in futex_exit_release(), we do this protected by p->pi_lock: */ raw_spin_lock_irq(&p->pi_lock); - if (unlikely(p->flags & PF_EXITING)) { + if (unlikely(p->futex_state != FUTEX_STATE_OK)) { /* - * The task is on the way out. When PF_EXITPIDONE is - * set, we know that the task has finished the - * cleanup: + * The task is on the way out. When the futex state is + * FUTEX_STATE_DEAD, we know that the task has finished + * the cleanup: */ - int ret = (p->flags & PF_EXITPIDONE) ? -ESRCH : -EAGAIN; + int ret = (p->futex_state = FUTEX_STATE_DEAD) ? -ESRCH : -EAGAIN; raw_spin_unlock_irq(&p->pi_lock); put_task_struct(p); From patchwork Mon Feb 1 15:12:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374259 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1381520jah; Mon, 1 Feb 2021 07:14:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJx+CuXN85ASXoifbcvhqtITWJRcxhaYlOoTVY208+hujEWK6FG2jyTj/Zh1TrLI2SLKeUE0 X-Received: by 2002:aa7:d755:: with SMTP id a21mr18929084eds.301.1612192479215; Mon, 01 Feb 2021 07:14:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192479; cv=none; d=google.com; s=arc-20160816; b=uqNInYKr99+r72E/lHHZ7oHZszGyzO4O2lnCPTyWJkBIsaASXTDQyEWrp+3htT+HB/ 9Qn+wMqa1mFmry71yA+OMsrlHmLKnkaOII719hbIIeaFsURmUf4u11XTNI4R1fnOHI5U Ch2Br25w7Ckpt19z+dHoxfBMLkA0nl10tsqfQjLjBGvkFzqmSzTwbsqC9jvTjR4eXxWD JOLRbJ3eGGcPP+fSc37mlN22w71+bSqmyVI9e8Uth5GaIjDwULfBANG+m1qxQs4OCHYT sHC4r/7IncLz9X1ohBAUXavapBozd9Vb+oQfuKIcwmGymx19soK/UmElI7GFP2C/eeqN y18g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=iXZwRIns2is4X1nRY0skb3yDaf3QquMnywmX7WSfZSk=; b=vjYdcM5IMCOUw/HZg9OvDXBMiHB9qIruYuRYp6dmtWEyyCR3T/YqsKmUwBEBYsBoV6 Z/E93iVZp+Uyx7RDnt+4hE9/60iStT+kO4/1LvufYp86sI6FiQ3myRxwoCUUlCKveVQU W1jVF55kADSORIkv5s8QBqkXGPEGjTo5c51qHNwW7NweRGah9x5ToxUGOGXoElTlo/wg LoXvUnH9kxmyMeELWVgKvny8U0ImvPnJjhAVn6PO/oCo5xhYAzo46YNbqvkFi8owFDBT WXrq2AFQTxfoREca05IYXGu/G/bY4/lYkUexL955/3uBbRpitVX38y3IDbRhkKlSsh8a Tx0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=x22wUzCM; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o17si11346999edz.582.2021.02.01.07.14.39; Mon, 01 Feb 2021 07:14:39 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=x22wUzCM; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbhBAPOZ (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231440AbhBAPOU (ORCPT ); Mon, 1 Feb 2021 10:14:20 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA7ABC06178A for ; Mon, 1 Feb 2021 07:13:39 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id i9so13511400wmq.1 for ; Mon, 01 Feb 2021 07:13:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iXZwRIns2is4X1nRY0skb3yDaf3QquMnywmX7WSfZSk=; b=x22wUzCMVwDMyDfsMBLNuyiHE8hITKH7qezvMs8a9lM9BHIC2xz3YpA+26fN3vTiLL dCmCKmayTkBpiTpg/43H7jg/hAdWzoONYBEKr73tIbbR6JY7+pf8Pa+EhvtnQGoZ132T eq+YJz+whCohqI01knRb89dXRFcN4Sf6sfH1819d3Plm3WkevCX+x22O2DrF7KdeikfY aFyI7VRWBun8oqC6Np4vjg/WcngLmP5ybjCBHWkOyT+s8LrA+04LAzn5Vusxe8weZrLG 9+ptxjqmOlovuGomtWN1H+J34/pXQMDMLiG/yUXeE/VoaS2HQsXuCGplwrVkcW3i3nw2 V7fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iXZwRIns2is4X1nRY0skb3yDaf3QquMnywmX7WSfZSk=; b=GQyYHjhRZ6zEQ8nwbUJapDSyVbpc2OAZfducYDPMjC27jtteKLU7Hm+3/pqOEYJaTN m0ZrkqgiRViq59hUZkKyUC0bt/mhO7KcdnxSSCdPfyXPJ6Yurzn1hur9fR6RuomMbWBR lcGI+ZTxCzBWtnIhLnKUw+oLJkzSUo4IO92TfkkeVMsKqK94A7mL+3fzB/msnFrE6aYo Y6SVBdPtNcWZCgM+mnm58ylv3ndff3RvkeCKNMQwADXRd2LF99y9H9JaH9ecySy0rx06 a8MylhtXXAWcftvaLgGgtIEYoLhL5kL4A1pcDgtTN3nE2z+5v/B1m8ssQvpKlhecH5t4 TUSA== X-Gm-Message-State: AOAM532u0a+gmz0h5FYf6lZ+S81WRaQ7os/2i15cW6qCeVSMpYCjJWRe XT419+U7+nOJ0EjFI3p4gxWC1oxKmAXCA0gZ X-Received: by 2002:a05:600c:4ed3:: with SMTP id g19mr15071996wmq.95.1612192417907; Mon, 01 Feb 2021 07:13:37 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:37 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 04/12] exit/exec: Seperate mm_release() Date: Mon, 1 Feb 2021 15:12:06 +0000 Message-Id: <20210201151214.2193508-5-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 4610ba7ad877fafc0a25a30c6c82015304120426 upstream. mm_release() contains the futex exit handling. mm_release() is called from do_exit()->exit_mm() and from exec()->exec_mm(). In the exit_mm() case PF_EXITING and the futex state is updated. In the exec_mm() case these states are not touched. As the futex exit code needs further protections against exit races, this needs to be split into two functions. Preparatory only, no functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.240518241@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- fs/exec.c | 2 +- include/linux/sched.h | 6 ++++-- kernel/exit.c | 2 +- kernel/fork.c | 12 +++++++++++- 4 files changed, 17 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/fs/exec.c b/fs/exec.c index 46cc0c072246d..ce111af5784be 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -875,7 +875,7 @@ static int exec_mmap(struct mm_struct *mm) /* Notify parent that we're no longer interested in the old VM */ tsk = current; old_mm = current->mm; - mm_release(tsk, old_mm); + exec_mm_release(tsk, old_mm); if (old_mm) { sync_mm_rss(old_mm); diff --git a/include/linux/sched.h b/include/linux/sched.h index bdd41a0127d10..aba34bba5e9e3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2647,8 +2647,10 @@ extern struct mm_struct *get_task_mm(struct task_struct *task); * succeeds. */ extern struct mm_struct *mm_access(struct task_struct *task, unsigned int mode); -/* Remove the current tasks stale references to the old mm_struct */ -extern void mm_release(struct task_struct *, struct mm_struct *); +/* Remove the current tasks stale references to the old mm_struct on exit() */ +extern void exit_mm_release(struct task_struct *, struct mm_struct *); +/* Remove the current tasks stale references to the old mm_struct on exec() */ +extern void exec_mm_release(struct task_struct *, struct mm_struct *); #ifdef CONFIG_HAVE_COPY_THREAD_TLS extern int copy_thread_tls(unsigned long, unsigned long, unsigned long, diff --git a/kernel/exit.c b/kernel/exit.c index 274a3c3834a15..a098d76a9877e 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -389,7 +389,7 @@ static void exit_mm(struct task_struct *tsk) struct mm_struct *mm = tsk->mm; struct core_state *core_state; - mm_release(tsk, mm); + exit_mm_release(tsk, mm); if (!mm) return; sync_mm_rss(mm); diff --git a/kernel/fork.c b/kernel/fork.c index d0ab6aff5efdc..43a50072dd5b4 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -887,7 +887,7 @@ static int wait_for_vfork_done(struct task_struct *child, * restoring the old one. . . * Eric Biederman 10 January 1998 */ -void mm_release(struct task_struct *tsk, struct mm_struct *mm) +static void mm_release(struct task_struct *tsk, struct mm_struct *mm) { /* Get rid of any futexes when releasing the mm */ futex_mm_release(tsk); @@ -924,6 +924,16 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm) complete_vfork_done(tsk); } +void exit_mm_release(struct task_struct *tsk, struct mm_struct *mm) +{ + mm_release(tsk, mm); +} + +void exec_mm_release(struct task_struct *tsk, struct mm_struct *mm) +{ + mm_release(tsk, mm); +} + /* * Allocate a new mm structure and copy contents from the * mm structure of the passed in task structure. From patchwork Mon Feb 1 15:12:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374263 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383107jah; Mon, 1 Feb 2021 07:16:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJx9/VCdGEjzMcHcf6YLI67eLvEMT9c7CR3BgshWjcTF2so3qKZFuGp3VcP73nsBL88+g+XH X-Received: by 2002:adf:9261:: with SMTP id 88mr18559889wrj.227.1612192568390; Mon, 01 Feb 2021 07:16:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192568; cv=none; d=google.com; s=arc-20160816; b=N1FSxSiTO3zojlSIC7nOf6gpNBrzRJpZghkmQTRCdhX7OGO13VGXWzDSQdEy+yy7yV a8JfSmjRRxhYIRBdLwkuFTbtqAmre0pNIR2V1UgG2fw9vMzkJ5uRbqSEwA5DiOWVCsCv wVRGUvGEWDCiG+fTXkWlZ+bT0gkLrj41LQL309NJmXL5qIbCg/dP32UtMK7iGjP2y0MY iR9fYAOUlAPMJb/AKrUPqH2AV8DYrJtmTjKedNwJvEIGctXBnYK85ClRZvsOdpOxvKsp 043QFhuUugyoP+oG9+FDMXb4iAxJrokF605/fY0SIQdecg8i3+Snzd5Q+PhamlQNk3/q 8YtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Iu4YQ/UttuQPFJ7M1Rq64N1axTPYV12HexbBC2SJPVQ=; b=LecqVJtNoDXvz+RCWv2836IViDEySbgENRG91irxYRKRz7aBTmVZri4kw+Rx7Ob1lr sK8rMcfC6+28tm9SwDrGsHZl0/7d5Bdx5onAvOVpIcnZBAd17jO4ATqQ6NbqH+72GmX+ y8zfLBSkNJ8lfdU9GBh+VICzYgFTdIYc/D/G+cCG6IibA8WvkJN7rMpUrwq14ZTbwRwS 5jLCSKUhP977rn8+5VQXbCqJBvMtzJtn+k2QaMfAbL20Sb7wWAwTe8I3jVRhVvRmZOTE cen9Czd/2vxpyddTeavslS54XOZMaKsomGTAv79mtClIFCdxaZRh55t2WnrvEjuyQHlh mCcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T+hbEweK; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.08; Mon, 01 Feb 2021 07:16:08 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T+hbEweK; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230437AbhBAPOt (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231292AbhBAPOW (ORCPT ); Mon, 1 Feb 2021 10:14:22 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13F0CC06174A for ; Mon, 1 Feb 2021 07:13:41 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id e15so13486797wme.0 for ; Mon, 01 Feb 2021 07:13:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iu4YQ/UttuQPFJ7M1Rq64N1axTPYV12HexbBC2SJPVQ=; b=T+hbEweKWZKT+yyzPDlE7nZL1uJChbb+ZeP7/5YjXKEvCCYfJbqpVNKKTmrLttaf5j 5byHTBf13l0ub+VtF5zmOShMTvLn6Y7IojKkQm7t3nWKPnGQ2QkbcHDePRnflCoQoUZh M6Uw+JxbVxtHHnzYfu8cLgvSvFB/Xm3jPaZxD/Sg+SXBkSP7pvRiZkMd002R11JtAHlq p/8nWhqwU7XXpCHDjkfWzlR9tWhEExp87pgxim2WlLAZzuUPzbafosEHJyznJkZX79mG Uect+6qqdVLCDbeaAZnHYeViaJSwgEi2JOXtCTyTiqh2IoHMmSbrLWtRQBFDB6nQlpUx S6Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iu4YQ/UttuQPFJ7M1Rq64N1axTPYV12HexbBC2SJPVQ=; b=jEVnB1HdmkjMYKZ+9xBKUrKwBT92NRZcXjjFMUlOUArfLw0+BgGLa7b+HiVnSwA8RD quPlmCaTu2WDvjaTnSgsHqGPR6ib+EVnkPonefspTYB7zmG4Bjuo9wI+4ZkZ2wC3pWxQ sDmdMrZLtzcDfLktyM/i2N9tNdPLBn7SEHoaoUKogZj/3/vKuHd6ANGX0Sn4XcCx4fIs cQS6CZuKDkcpcpx7WEs3ENH9GAdpz/6V8MJvVeBlsvixU0RQgqHdoypWAqpsEWB3F1uI 8900t1P95dDGTNcuU5fTcVw1O6sRViIhe+95bHc/jT/xLSAld6d04XO3MSodV+V5ZiPn UbDw== X-Gm-Message-State: AOAM533KofLHrBKdH88mXEsmS16O0ll7uC38ir3Sa69TMnh68WW8SaWa 4RgyaYi5ff0/TD/+V+YIgIchQu2J2u2Q3M62 X-Received: by 2002:a1c:7e4e:: with SMTP id z75mr15778850wmc.168.1612192419484; Mon, 01 Feb 2021 07:13:39 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:38 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 05/12] futex: Split futex_mm_release() for exit/exec Date: Mon, 1 Feb 2021 15:12:07 +0000 Message-Id: <20210201151214.2193508-6-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 150d71584b12809144b8145b817e83b81158ae5f upstream. To allow separate handling of the futex exit state in the futex exit code for exit and exec, split futex_mm_release() into two functions and invoke them from the corresponding exit/exec_mm_release() callsites. Preparatory only, no functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.332094221@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 6 ++++-- kernel/fork.c | 5 ++--- kernel/futex.c | 7 ++++++- 3 files changed, 12 insertions(+), 6 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index a0de6fe28e00b..063a5cd00d770 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -98,13 +98,15 @@ static inline void futex_exit_done(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_DEAD; } -void futex_mm_release(struct task_struct *tsk); +void futex_exit_release(struct task_struct *tsk); +void futex_exec_release(struct task_struct *tsk); long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3); #else static inline void futex_init_task(struct task_struct *tsk) { } -static inline void futex_mm_release(struct task_struct *tsk) { } static inline void futex_exit_done(struct task_struct *tsk) { } +static inline void futex_exit_release(struct task_struct *tsk) { } +static inline void futex_exec_release(struct task_struct *tsk) { } #endif #endif diff --git a/kernel/fork.c b/kernel/fork.c index 43a50072dd5b4..2bd4c38efa095 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -889,9 +889,6 @@ static int wait_for_vfork_done(struct task_struct *child, */ static void mm_release(struct task_struct *tsk, struct mm_struct *mm) { - /* Get rid of any futexes when releasing the mm */ - futex_mm_release(tsk); - uprobe_free_utask(tsk); /* Get rid of any cached register state */ @@ -926,11 +923,13 @@ static void mm_release(struct task_struct *tsk, struct mm_struct *mm) void exit_mm_release(struct task_struct *tsk, struct mm_struct *mm) { + futex_exit_release(tsk); mm_release(tsk, mm); } void exec_mm_release(struct task_struct *tsk, struct mm_struct *mm) { + futex_exec_release(tsk); mm_release(tsk, mm); } diff --git a/kernel/futex.c b/kernel/futex.c index e531789aa440a..32a606b605cbb 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3234,7 +3234,7 @@ static void exit_robust_list(struct task_struct *curr) curr, pip); } -void futex_mm_release(struct task_struct *tsk) +void futex_exec_release(struct task_struct *tsk) { if (unlikely(tsk->robust_list)) { exit_robust_list(tsk); @@ -3252,6 +3252,11 @@ void futex_mm_release(struct task_struct *tsk) exit_pi_state_list(tsk); } +void futex_exit_release(struct task_struct *tsk) +{ + futex_exec_release(tsk); +} + long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3) { From patchwork Mon Feb 1 15:12:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374262 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1381713jah; Mon, 1 Feb 2021 07:14:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJxqX8dafm80zR4M9GrK2aKQoAFV3/ntxcheXoLRTCJ1Zc3LHRcoMLWQrT5EW5ODl8GCGjmn X-Received: by 2002:a17:906:f102:: with SMTP id gv2mr1863693ejb.47.1612192490824; Mon, 01 Feb 2021 07:14:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192490; cv=none; d=google.com; s=arc-20160816; b=pyM87Pvp8GeBuk1K1LNxkOABFYgvboV8oxkIUGF8QPv4DLzcymW8Dr1KOXza+/p2yg HRFpYabvoPNVcWvwGWACDxsUqdOeJqBhZqegECzwqROdpPQlo2gxvrZjOyfimVkSH1Dc HLBvkgCB4VuB9V5df0zCXwx12jsB3Cz9YPIDchsunMkGbzFHCKgiBApKKffxFzYmBm0y 4qfC2F1H0/UE6S7eYpjV5PeEx9kX8j18MCVuwe+P5A2ppgVe4N450watdz6cXD3jls+Q Eyc5v3eDfhZ080FxTZSRJoL4WSs0Dz1/hY4HUaa3hsqm7fNoSSDGXqIFRk1PzBbDFH7a 1uzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ltrudIytk8l60pTfQhCRnlUeawvarsYISOCC5mS7dpk=; b=fKBM/gpF37hkZ10FkWKYqrZd2qdZmea1R/XkmHh3BA+BqUK2zXk49/VC3JNEdvo2lz aWOlngUYLIKk3U0JKBUQ5zA7BRQLxuFLSD4CZtx2zXZwnA5OnVHUtWvlwleRhpwpjiKh Jl5wlWly6dlqbKpl4+A6qA19YmQyIwsE/UNekCpF5qSvtOO95nITKPlT6hrv6OHOMV/h iqjy0OeFlNQIXKcr3z0rtWTLFj/Ijm2sMfJDHHkx/9L+3DzdEA1QUX8sbF/IdpLFiG3L eczJ+YhbDH2N4lqQjywtTrRTZFwThbaovt9CSdCCCHwBeoSyeyOqtiXRqhDCyvVHG0eH QpFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YBCL+lGG; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o17si11346999edz.582.2021.02.01.07.14.50; Mon, 01 Feb 2021 07:14:50 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YBCL+lGG; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231346AbhBAPOl (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230437AbhBAPOW (ORCPT ); Mon, 1 Feb 2021 10:14:22 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43B73C061756 for ; Mon, 1 Feb 2021 07:13:42 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id u14so12937929wml.4 for ; Mon, 01 Feb 2021 07:13:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ltrudIytk8l60pTfQhCRnlUeawvarsYISOCC5mS7dpk=; b=YBCL+lGGCAyPwLyPt/JVUWMlZrUPnYgZ44oOBXgwxAaOPw4BMSJfY2jMh1SBE9jqrz WbZfK7pqQW/1mOLjorBzr+VaZLo5M0aBpxB9fZdeklGiQxjs+It/15vM9fCjYk/yZRYj 6PGhRDgcYgttNlUV7ETurNk5XU9ZTfeXoaupfBkCGN6DKDFa3QXgNx8OxINtmK6w63zy l0GOyY3nBnluVyi25d81wZsG7PnXD3gw9EAGxGYvuoPpCBHwlGnWmbps6xWT3EerXyV/ zszCNGvjei/yt0eVe8T4A/31VtRIuF9cobiQ9iIt1LJqcg/+XMRgmIT3DTcJ/yRKI88D hsdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ltrudIytk8l60pTfQhCRnlUeawvarsYISOCC5mS7dpk=; b=T0VTh5Jj39S9pHXVGyibG0WjJ2qCj1P/DzijitMHIQK4UaFlNR2XPskwoflaWoS/Xi BJ6+pEpL1aM3MEaAgxKMdfn+O+ujVm8aNPUiHdiesNyc3flH1OVOW6oHBoMcgCq2o2v1 9shpE/imZsGvO9Vgc/iZtu9wy+xAitC7iM35Ynpav/jvUROjtLHlZZuhv3o83V2FOH5d 3cG1fOziYgFlXaIujHwJlD0fCIolAhYQ6M4fwphvNmG0LOLdsk+evHgfT+r0AuoR8PYM CGMER2kxVM/vkomU+NsP64tSSyHJaO9ltMkqzd6N69jVI4SLQK8x2u9jFgVcwr8At7LL 1RgA== X-Gm-Message-State: AOAM530O/ex+NXlHR5RddLAaTrmtaT4PkW81bljN8fH6Xn/6Cxkolg9N KAzHIglPlZATljo2QLhv1vKMh/Le6eP+JrWi X-Received: by 2002:a1c:20d8:: with SMTP id g207mr3076950wmg.77.1612192420706; Mon, 01 Feb 2021 07:13:40 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:40 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 06/12] futex: Set task::futex_state to DEAD right after handling futex exit Date: Mon, 1 Feb 2021 15:12:08 +0000 Message-Id: <20210201151214.2193508-7-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit f24f22435dcc11389acc87e5586239c1819d217c upstream. Setting task::futex_state in do_exit() is rather arbitrarily placed for no reason. Move it into the futex code. Note, this is only done for the exit cleanup as the exec cleanup cannot set the state to FUTEX_STATE_DEAD because the task struct is still in active use. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.439511191@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/exit.c | 1 - kernel/futex.c | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) -- 2.25.1 diff --git a/kernel/exit.c b/kernel/exit.c index a098d76a9877e..b39f4b3c0f37c 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -784,7 +784,6 @@ void do_exit(long code) * Make sure we are holding no locks: */ debug_check_no_locks_held(); - futex_exit_done(tsk); if (tsk->io_context) exit_io_context(tsk); diff --git a/kernel/futex.c b/kernel/futex.c index 32a606b605cbb..f85635ff2fce1 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3255,6 +3255,7 @@ void futex_exec_release(struct task_struct *tsk) void futex_exit_release(struct task_struct *tsk) { futex_exec_release(tsk); + futex_exit_done(tsk); } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 15:12:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374264 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383114jah; Mon, 1 Feb 2021 07:16:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJxsSyBmO0WFAjZtrDC4K6aBIbv2TRjoM8eaMXL8G92p5VE3N5xbytKwUlnQEpJNIvRqASHl X-Received: by 2002:a5d:4988:: with SMTP id r8mr18878327wrq.26.1612192568949; Mon, 01 Feb 2021 07:16:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192568; cv=none; d=google.com; s=arc-20160816; b=z/PUoQ6BAEGKcDA7pjhr2o9Z2/YTAoPdZ/d4fhwUr3cep+F5KJIFURuIbOo0uytU3w nqU6QCC5UVPx82XCwGprdVZO5c7CYRc0ePO6WF/n0UN0lLtIkUrOQhwKIt4kujrocCsa 4nX3QM1qPvWwYSr/VVMbFIWd3xLay6FMXQKyh9jbx8QiVMi/qaIXNTBW0XKpHds9mGKK UA/8ayYKH5MWFTEiCzvIYhgZBHpnY0pqUhb7EKGml1MthqFOcwvNjUDsVxB8j2uWNcXU qCuTICXZg/LM5+DTgDyDdd/g0EZqOaTtfRAzxWquoyvZPKCPRP8ruYxhvGmdHrVhvcJc 1I7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QzQj1EdG176w+bcW/LUmQ/cOr3Onvdscuk/WhMM2DZg=; b=ZA7u2Mxz8gzrVWEgVQewGIzaEr3mx62mpfl31WGDsQUXgKijML5Z1/WhT+GbWdZBZN Z/oml2KPknExcGx0QtNPd6Sc5W3jZijRQmDgVsDUnYXtgIgJNQ5nRl/Sisni71FICV5q yv/xq8+5RS7z9If39wBv6c1gNdYx+XTLgaZQOi4rHedho0eOARV8j88ahlMNiBEbkeVo vSESnUCYmdc1RnjEoRmb7d7+cOWciIijqMDx/78vWzrOamXefTdB52dHzZrnh5vDi4Fd StBgtSS3yhJXwAbA8bjhSe91/fop0RsG5dhqgwuH+zuUMSm+hTtRVheTB2D1SvoYKp+i Rp/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xQgq5dWO; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.08; Mon, 01 Feb 2021 07:16:08 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xQgq5dWO; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231451AbhBAPO6 (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231292AbhBAPOz (ORCPT ); Mon, 1 Feb 2021 10:14:55 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80C5BC06178B for ; Mon, 1 Feb 2021 07:13:43 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id z6so16909593wrq.10 for ; Mon, 01 Feb 2021 07:13:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QzQj1EdG176w+bcW/LUmQ/cOr3Onvdscuk/WhMM2DZg=; b=xQgq5dWOnqZGP4Cwr00PBH50Nqo2h5cQTfJiDLh4MArixKV0YGdCU8DEuN02AB5M+K R4/1G2+7iQACv35wdiRHwrpH/KCvllQXF7KH4afBp1fRQtycOgRK9kNggF6OG4HHwayj ZA/QxbB+vlbOuMBsQ72jYG0t+iyGm2IY+rWZusY6NFJIeEUT+DdOU6VrpCggJeYXLvzb UpMBDhWqgKjLofEarjn1TuGznQjTHIf9ad3XcVhGIrdZ0AVM/25Q94+o9gnDV94mAUxt g9bxeISDp4eHQwfmyILTspyNg6UAbc7Q23Dam1zP6J5OhcEEC32qYrb9U2vWqHsnbpF3 uBxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QzQj1EdG176w+bcW/LUmQ/cOr3Onvdscuk/WhMM2DZg=; b=isjRxU4CdEWSBGPlKvlo9d+qV/u4eT7Ook8/5rHCSRoVwCNhIbCgK/jA5lHjr7lJNj LgBQFzNodjtXqeBzCJ6ztLEBlMT9kpWT75dOVg8EnaNcncE6jDlkZVsd+H1Std0JgUuj XtCx9J5M0RyXj36OBFyDiILGBzFVDauh9wvn5/KZ5rR9xx7yeivri8n58a6EZsM2H9Zu MHobqYT14kwEdHus0srGhQYsZv79y+9gRLusaFJrjWHuwDZo0znAiD3LoeFOlYBkMmQX BnrGRbJ9YN2yNZqrIeaE9wbfm71s3aLNKs3OiwYZNoJKMBkgUbJyT8/Rbp4Sx4lzqWZM s0Wg== X-Gm-Message-State: AOAM531hBI2oyYdw1YPKShbONLsYvz6w3YwuAks17x+vnu7Sd+s+kR9o 6YoitSpaZSk8XkUFKgAkfXNJeyj9j2Lc4SNb X-Received: by 2002:adf:f189:: with SMTP id h9mr12200603wro.286.1612192421834; Mon, 01 Feb 2021 07:13:41 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:41 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 07/12] futex: Mark the begin of futex exit explicitly Date: Mon, 1 Feb 2021 15:12:09 +0000 Message-Id: <20210201151214.2193508-8-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 18f694385c4fd77a09851fd301236746ca83f3cb upstream. Instead of relying on PF_EXITING use an explicit state for the futex exit and set it in the futex exit function. This moves the smp barrier and the lock/unlock serialization into the futex code. As with the DEAD state this is restricted to the exit path as exec continues to use the same task struct. This allows to simplify that logic in a next step. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.539409004@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 31 +++---------------------------- kernel/exit.c | 8 +------- kernel/futex.c | 37 ++++++++++++++++++++++++++++++++++++- 3 files changed, 40 insertions(+), 36 deletions(-) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 063a5cd00d770..805508373fcea 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -57,6 +57,7 @@ union futex_key { #ifdef CONFIG_FUTEX enum { FUTEX_STATE_OK, + FUTEX_STATE_EXITING, FUTEX_STATE_DEAD, }; @@ -71,33 +72,7 @@ static inline void futex_init_task(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_OK; } -/** - * futex_exit_done - Sets the tasks futex state to FUTEX_STATE_DEAD - * @tsk: task to set the state on - * - * Set the futex exit state of the task lockless. The futex waiter code - * observes that state when a task is exiting and loops until the task has - * actually finished the futex cleanup. The worst case for this is that the - * waiter runs through the wait loop until the state becomes visible. - * - * This has two callers: - * - * - futex_mm_release() after the futex exit cleanup has been done - * - * - do_exit() from the recursive fault handling path. - * - * In case of a recursive fault this is best effort. Either the futex exit - * code has run already or not. If the OWNER_DIED bit has been set on the - * futex then the waiter can take it over. If not, the problem is pushed - * back to user space. If the futex exit code did not run yet, then an - * already queued waiter might block forever, but there is nothing which - * can be done about that. - */ -static inline void futex_exit_done(struct task_struct *tsk) -{ - tsk->futex_state = FUTEX_STATE_DEAD; -} - +void futex_exit_recursive(struct task_struct *tsk); void futex_exit_release(struct task_struct *tsk); void futex_exec_release(struct task_struct *tsk); @@ -105,7 +80,7 @@ long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3); #else static inline void futex_init_task(struct task_struct *tsk) { } -static inline void futex_exit_done(struct task_struct *tsk) { } +static inline void futex_exit_recursive(struct task_struct *tsk) { } static inline void futex_exit_release(struct task_struct *tsk) { } static inline void futex_exec_release(struct task_struct *tsk) { } #endif diff --git a/kernel/exit.c b/kernel/exit.c index b39f4b3c0f37c..8d3c268fb1b8d 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -695,18 +695,12 @@ void do_exit(long code) */ if (unlikely(tsk->flags & PF_EXITING)) { pr_alert("Fixing recursive fault but reboot is needed!\n"); - futex_exit_done(tsk); + futex_exit_recursive(tsk); set_current_state(TASK_UNINTERRUPTIBLE); schedule(); } exit_signals(tsk); /* sets PF_EXITING */ - /* - * tsk->flags are checked in the futex code to protect against - * an exiting task cleaning up the robust pi futexes. - */ - smp_mb(); - raw_spin_unlock_wait(&tsk->pi_lock); if (unlikely(in_atomic())) { pr_info("note: %s[%d] exited with preempt_count %d\n", diff --git a/kernel/futex.c b/kernel/futex.c index f85635ff2fce1..5bd3afee4e139 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3252,10 +3252,45 @@ void futex_exec_release(struct task_struct *tsk) exit_pi_state_list(tsk); } +/** + * futex_exit_recursive - Set the tasks futex state to FUTEX_STATE_DEAD + * @tsk: task to set the state on + * + * Set the futex exit state of the task lockless. The futex waiter code + * observes that state when a task is exiting and loops until the task has + * actually finished the futex cleanup. The worst case for this is that the + * waiter runs through the wait loop until the state becomes visible. + * + * This is called from the recursive fault handling path in do_exit(). + * + * This is best effort. Either the futex exit code has run already or + * not. If the OWNER_DIED bit has been set on the futex then the waiter can + * take it over. If not, the problem is pushed back to user space. If the + * futex exit code did not run yet, then an already queued waiter might + * block forever, but there is nothing which can be done about that. + */ +void futex_exit_recursive(struct task_struct *tsk) +{ + tsk->futex_state = FUTEX_STATE_DEAD; +} + void futex_exit_release(struct task_struct *tsk) { + tsk->futex_state = FUTEX_STATE_EXITING; + /* + * Ensure that all new tsk->pi_lock acquisitions must observe + * FUTEX_STATE_EXITING. Serializes against attach_to_pi_owner(). + */ + smp_mb(); + /* + * Ensure that we must observe the pi_state in exit_pi_state_list(). + */ + raw_spin_lock_irq(&tsk->pi_lock); + raw_spin_unlock_irq(&tsk->pi_lock); + futex_exec_release(tsk); - futex_exit_done(tsk); + + tsk->futex_state = FUTEX_STATE_DEAD; } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 15:12:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374269 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383120jah; Mon, 1 Feb 2021 07:16:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJyp67qqm5qaGcFkgQqJznus81zTSPjRbDIw4l35MjAJI3FD3zhalNW6yh9XAU9z7m7tx7S9 X-Received: by 2002:a17:906:b047:: with SMTP id bj7mr18425757ejb.335.1612192569370; Mon, 01 Feb 2021 07:16:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192569; cv=none; d=google.com; s=arc-20160816; b=A42vUz+zbtL/dHisQF0nwmV6hg9xEHLsdYW2uFDnVfYB3GVUiAp+wS7uNjN5TXt9F1 lPr/5Rz8byj8RqOy5UEe3PLZJsEGpWuuoJEvZcnXWG4wFQxI5FcvshQYQcHXVqqIflnc h39ggujBcifJU5W8wgmD5YalYku9HZRUHvwsZjdy7yXwtbpUDzG+IMbEo7stQ3PBTcuc /OJjwjzAW5XEAElls9qBj9TJHdYLjq3kTHbpgPcYo8JjZ1ZT85gK+U/SaLD/QvNjusMa eX5a87buuT+6eYeVxQIj6DaGmTZj08m2R9Zdu9AcIt82m8oXwD7m54h2bgrVA88NIpKx Tv2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XpuIWF2HW9dIBgX/Ld0+OKyOSiCvAVgygw7iJWAFZGs=; b=YWDbgKOkT49UINlCeGaFDiAPhf5cQIZEzselRkeRwvoMWi7qGmj1y7C4ioda72xx25 chxKzMNe+nMviwScALerl5Qv0HctNe7sfJjMycXuW94qJ2kiNuUvArjj9RPkz1Lg4EsU TRdITEX4oCpM0En30mGmQfI4cIuvuSwnsfWX6pcRzsGzQL5IMOpcaX673DuYpng+MxrT vMJLdaS4cApkgQcFiK6WMyE0/j01e0GIezBwT5wb71fM68eQlu19k8ZjXyzGh002jna+ QK8nRL6s9FBDZgbAzskz5S67yYvEGG2bR4jIOJ8qlzDHWDBzeDi77mbXWqmDqmeI6ygg BAcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IWN4UOeg; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.09; Mon, 01 Feb 2021 07:16:09 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IWN4UOeg; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231292AbhBAPO7 (ORCPT + 13 others); Mon, 1 Feb 2021 10:14:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231376AbhBAPOz (ORCPT ); Mon, 1 Feb 2021 10:14:55 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68402C06178C for ; Mon, 1 Feb 2021 07:13:44 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id c127so13472345wmf.5 for ; Mon, 01 Feb 2021 07:13:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XpuIWF2HW9dIBgX/Ld0+OKyOSiCvAVgygw7iJWAFZGs=; b=IWN4UOegACrcqKI3wDyqv54yB5lKHxlCgWlnzDfxAFCvoNZwStXF57Ez5Bi7LfCps0 FJJQrbw2tK6cwO/zw4QAs+4UWRguf3Lt7a3SfVd0k01kOvCMxUOQ0kQ2rs+cXA2cvyg4 PDCBR25VlCVQfwupiGjTWsKjvdiDDkiwx6TV/TbQpOTY3GCbzj8OgXdKxJEEFNIg6OJY zAE7/Q4VqgQKETPQ2foFtiiCVlK22BdJCp9O3qUQZFrnUHjDwQm8V4vMcX/c3WUhKwDJ vJqS6JMgauD9aL6d3moCVpUBqupaR0EeYAup7qPB+Z2Vx22AH+tD7ZmDyJULu4v4Kt25 rBeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XpuIWF2HW9dIBgX/Ld0+OKyOSiCvAVgygw7iJWAFZGs=; b=OVzpBSoFVgFRSkPdEK90/petiJ6JXfHMk9Vm8V3xtClwMoJWVx38EKA54ZXOwoxm7y wVv9PgjwmiOKiUiLuq4hOnfg7gpJZs3ZBt7luMvWNoodQc7pVguV2fjvqDU3FMQdF6a2 aNnwU9F04v6XVGvvgCWdhpPrb4ydu5IXPm4kcif2rIvXxIxiZFpb60TlkmGuOJ2rc7qJ jJoIrwWXtb0zhJtsYKyNvO6J6cucSFGXTuOXWmOFGQseguF+h26C6tm/I+QW/k7drzb1 gK5tYDNNE1jmu2UUT8vEHIbvm3zqFDPMGG2SErasxQ6Sy1nWSe3XG/b6f1DxtktKRYoD aXHA== X-Gm-Message-State: AOAM531mAX/admf7/t531vdv1NaL9WanSnBnZnATeq0KyDpO5ZinEoim T9Way+aRqh2dsrqugPFwXv6mG+gtmoFK5/BT X-Received: by 2002:a7b:c8c3:: with SMTP id f3mr15560824wml.110.1612192422859; Mon, 01 Feb 2021 07:13:42 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:42 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 08/12] futex: Sanitize exit state handling Date: Mon, 1 Feb 2021 15:12:10 +0000 Message-Id: <20210201151214.2193508-9-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 4a8e991b91aca9e20705d434677ac013974e0e30 upstream. Instead of having a smp_mb() and an empty lock/unlock of task::pi_lock move the state setting into to the lock section. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.645603214@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 5bd3afee4e139..0efcb55455c2a 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3276,16 +3276,19 @@ void futex_exit_recursive(struct task_struct *tsk) void futex_exit_release(struct task_struct *tsk) { - tsk->futex_state = FUTEX_STATE_EXITING; - /* - * Ensure that all new tsk->pi_lock acquisitions must observe - * FUTEX_STATE_EXITING. Serializes against attach_to_pi_owner(). - */ - smp_mb(); /* - * Ensure that we must observe the pi_state in exit_pi_state_list(). + * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. + * + * This ensures that all subsequent checks of tsk->futex_state in + * attach_to_pi_owner() must observe FUTEX_STATE_EXITING with + * tsk->pi_lock held. + * + * It guarantees also that a pi_state which was queued right before + * the state change under tsk->pi_lock by a concurrent waiter must + * be observed in exit_pi_state_list(). */ raw_spin_lock_irq(&tsk->pi_lock); + tsk->futex_state = FUTEX_STATE_EXITING; raw_spin_unlock_irq(&tsk->pi_lock); futex_exec_release(tsk); From patchwork Mon Feb 1 15:12:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374267 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383175jah; Mon, 1 Feb 2021 07:16:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJxzNlLLWk9TvsGiDHqU3eRxS8f4qB/YrPAhB5sfIy8FtCR0lCTm9GdG/WB8gTa54HQdy8a8 X-Received: by 2002:a17:906:3ad0:: with SMTP id z16mr18355091ejd.72.1612192571675; Mon, 01 Feb 2021 07:16:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192571; cv=none; d=google.com; s=arc-20160816; b=ajuTQMN3eiUjBaIP0ZFc9NeBybjYKrcdhbzHCAYfRkUW1P8yLMyt5ip/g2gIvt1mn3 vY5ZSICczFIJNSM/r9leb4jh8YmgjXiwUyOu7Li+AuDivu9ZikMunO+kZcJDLRTA43S4 KYngDTL9Y3fVrxCrjy8gyXUo5JkWZEw1KMIx43tkRpkoj3n1npbfVH94mgVi+G4ylq88 Q9NJQrI1YgUd4F671+ESOVmr2K5B3Fo/rRMbYI4G7IwiqeG9OeThp0uhQjHSrG4u7UAC 8LTjXVfu9mnpxXN+mgn+TWQtBt8HHQYU/ICMMl8B6HUA6yF/jVJrhM1sEqIDvfSU9qFn 2C+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MCVDRoiJdy/Do7csl/30kMZE1zZamBX1lgP8roH5D6M=; b=XC7PSRXXzuwGSIkN+7w7R0P5FaFfM+Er+JiU08fQMDNQpDk0feadFMGDLrs61Whh0X lB6fNYEsXkKV2z/S5n0/9tVBGJ5tN61dUn1thXetfi5MexuZsuwDed41I5JBYiBu0SLi TjZe7TfPWJWvLGI2kwu+xP7MqT37SI/WHneg8mYlL4dah68JAo2J/bK3aiQcpx/7YWb9 imyBmsqNry6IDiRjvo83X2wZmi+4+kcQxcusz4xRjWG1P+hOl6LcI1xaniIe6vBDoYfN MHw7oezLeqcJdFNHIJ0jDdASMiE2ZSAt9PQ7ecUSRym9ADH5jt8F7enMc1/Li4nnCs7M kugA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=J0w3ovp+; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.11; Mon, 01 Feb 2021 07:16:11 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=J0w3ovp+; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231320AbhBAPPC (ORCPT + 13 others); Mon, 1 Feb 2021 10:15:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231440AbhBAPO5 (ORCPT ); Mon, 1 Feb 2021 10:14:57 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F13FC061793 for ; Mon, 1 Feb 2021 07:13:45 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id u14so12938060wml.4 for ; Mon, 01 Feb 2021 07:13:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MCVDRoiJdy/Do7csl/30kMZE1zZamBX1lgP8roH5D6M=; b=J0w3ovp+vCnLYkGRkQmCZ5FrnFU7ex17dn7x5K7Lanyd942kM8Ds9aw6A37qAVCaqE gKyJADcqGLD3xtd/vGi51UshfYvnfNO7ZoyOBaFnRTl+93uMy50Z3lkKP8CEjzcyQCqI y9OM7lMce+gVaCF63DzI9EDh/MzOckGMmFAlvACObd2MMSFscAMeFDkkjunhYwEvnvft m6DmK97o8D3E1ir8x6snqV5Jg8hAhH5sbGYZCDPAK+s1Bzxr+2G0lH5qQEuL9MgMQfkz sHHEGc68AfLV4/aLLTJtaBvrIWv32XGakJBFmTKfbRW4Gm+Ho9rTdN6kvbVvgT91uBer fW8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MCVDRoiJdy/Do7csl/30kMZE1zZamBX1lgP8roH5D6M=; b=t4UrTgqGLk/eeGVkqsedaSAM3Wy6qTjZRTbUtmhNv6gACqxrb0KZc1t9sRaCAHQGFw UYbOlnoX6+ZOoWf9glxJyJFgeBjYZRMTjVy2vl0K1OBCu9R5rzgqDObEGLhBqMTtlBL5 V+bcyd/BEtHFiKd0IQY/hGCZrDD6P94yPumpBVb4sGTsXaGCTNXPDvOm1XC949oSQ8DC rFvslfM5QPbNdusSRoc9P791iPvKQi8TPWB+qjAC0CWycrw5s0LAWrjY/rKjvT+rxVBn XIaz2E90GZk7sE5Z5ncrH8XlyPAScuh6lTimU96M3DlbhHmZsdjmi5HU1rqJVtpVirsL B/VQ== X-Gm-Message-State: AOAM530yGlKIdhJfXBfQaIFS/t2xlqPqEzvIK15uftEVOhrgVsQVRbOo UFNM8MKqAYW8TsGadCKBFIQn9+SMeT8sWF8P X-Received: by 2002:a1c:4c0a:: with SMTP id z10mr7427390wmf.163.1612192423862; Mon, 01 Feb 2021 07:13:43 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:43 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 09/12] futex: Provide state handling for exec() as well Date: Mon, 1 Feb 2021 15:12:11 +0000 Message-Id: <20210201151214.2193508-10-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit af8cbda2cfcaa5515d61ec500498d46e9a8247e2 upstream. exec() attempts to handle potentially held futexes gracefully by running the futex exit handling code like exit() does. The current implementation has no protection against concurrent incoming waiters. The reason is that the futex state cannot be set to FUTEX_STATE_DEAD after the cleanup because the task struct is still active and just about to execute the new binary. While its arguably buggy when a task holds a futex over exec(), for consistency sake the state handling can at least cover the actual futex exit cleanup section. This provides state consistency protection accross the cleanup. As the futex state of the task becomes FUTEX_STATE_OK after the cleanup has been finished, this cannot prevent subsequent attempts to attach to the task in case that the cleanup was not successfull in mopping up all leftovers. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.753355618@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 0efcb55455c2a..feef5ce071aa5 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3234,7 +3234,7 @@ static void exit_robust_list(struct task_struct *curr) curr, pip); } -void futex_exec_release(struct task_struct *tsk) +static void futex_cleanup(struct task_struct *tsk) { if (unlikely(tsk->robust_list)) { exit_robust_list(tsk); @@ -3274,7 +3274,7 @@ void futex_exit_recursive(struct task_struct *tsk) tsk->futex_state = FUTEX_STATE_DEAD; } -void futex_exit_release(struct task_struct *tsk) +static void futex_cleanup_begin(struct task_struct *tsk) { /* * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. @@ -3290,10 +3290,40 @@ void futex_exit_release(struct task_struct *tsk) raw_spin_lock_irq(&tsk->pi_lock); tsk->futex_state = FUTEX_STATE_EXITING; raw_spin_unlock_irq(&tsk->pi_lock); +} - futex_exec_release(tsk); +static void futex_cleanup_end(struct task_struct *tsk, int state) +{ + /* + * Lockless store. The only side effect is that an observer might + * take another loop until it becomes visible. + */ + tsk->futex_state = state; +} - tsk->futex_state = FUTEX_STATE_DEAD; +void futex_exec_release(struct task_struct *tsk) +{ + /* + * The state handling is done for consistency, but in the case of + * exec() there is no way to prevent futher damage as the PID stays + * the same. But for the unlikely and arguably buggy case that a + * futex is held on exec(), this provides at least as much state + * consistency protection which is possible. + */ + futex_cleanup_begin(tsk); + futex_cleanup(tsk); + /* + * Reset the state to FUTEX_STATE_OK. The task is alive and about + * exec a new binary. + */ + futex_cleanup_end(tsk, FUTEX_STATE_OK); +} + +void futex_exit_release(struct task_struct *tsk) +{ + futex_cleanup_begin(tsk); + futex_cleanup(tsk); + futex_cleanup_end(tsk, FUTEX_STATE_DEAD); } long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, From patchwork Mon Feb 1 15:12:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374265 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383148jah; Mon, 1 Feb 2021 07:16:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJwBKn+8TL2I2eL0ra49zNPzPX4R/BqhYSRRqLs0rxUGuhIRiXTL2v8gv8sV5tZsBOOSzf66 X-Received: by 2002:aa7:d692:: with SMTP id d18mr5737596edr.327.1612192570449; Mon, 01 Feb 2021 07:16:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192570; cv=none; d=google.com; s=arc-20160816; b=Sx1yvxzSDXTx4sO7QkHDvKE+UN0LJVxlxLlALOWJflevEXa2mWya8jpMuL/I7to8fx CI/KGBR570Zmt3HhVzPnlDopOAG/ekX1f+ijuLwtRNwWhYFjhvhnV+EsyZZS1l7gfjHX 5pRPiSaFn6i3NsW62Jhj0BPhKbqjU9Y8ksEoEstFq41HIGHdK0BogjvNfWdXtakndpzf L5TIdsfQCSncUIT0tDnWzgBYfLzSLjiPP2TpZ6hwDb9u7jPq73E39HAmOCEN0vu9XdbS 2oaFqptXNCcNrJd30YaTCBWN98AYPWnk+C8Hu75SAb9mq9F8HFZwG1Z1s7tPDmP5GY39 t0Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=q27H37/zpcTikuASjIMD4FhxtWUcta8dDgeh38mJ6rY=; b=QGkoRkD+9QfD4Tn91Pkzu0SC6va6ZOYBdAGO7bbt6VjOusEcbqsdWHgpmj7XtyDRCm V3CaNdwzqGREc2+DpBbrc15RXeEgTls55cvxxufY7o9hpGB3fdBlwCBEqHr+KRXIbwQm qOrHcUeKpcsTkSEqCL6Wb9brM0D313ZM6faf0wkDuwZjZL6me6TwfsxRSMOmOYTFcuaj Ya0Nep8/GQoMXGW2yfoYmoQ8/kyYp0q8D/k/qeLCDfy1QjHAGaeatsSdB48Avk1znqBC V0mYvP6FnnwOOhrSRcPstKFAVvuU2GLr6E51fwEkGJms0Yq6oIXelWs+0FROawrjgLNP l+eA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q4MQbzN7; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.10; Mon, 01 Feb 2021 07:16:10 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q4MQbzN7; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231390AbhBAPPB (ORCPT + 13 others); Mon, 1 Feb 2021 10:15:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231463AbhBAPO5 (ORCPT ); Mon, 1 Feb 2021 10:14:57 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C18FC061794 for ; Mon, 1 Feb 2021 07:13:46 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id g10so16980654wrx.1 for ; Mon, 01 Feb 2021 07:13:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q27H37/zpcTikuASjIMD4FhxtWUcta8dDgeh38mJ6rY=; b=Q4MQbzN7KagSSyZ68E9POBDyRqxUcOspqukH42Stuk/h19Vv+2ghdy8TUd6Yhtw+sJ fJPNPL1vInwJXsofZ+4Py7dCY4waVqj6sjKpzQ+Eu1RcsQHD1LxQAzAx59jMjWyvz6yN cdVNCz0j661b86g4pKYv9U5Pg6ffG9Bjxzk26TljQ4lc1oR6cb434Jkn2JLQeI1lUFz9 wbkXTykPYT2TuEvOYP+NrPd99OEaAe+VZsmpwDIgLBYypY8FhbxdvYt5LpCglEbZKYlh bI9pHuUUlsjYf8lGiIs1WyE847ZLZVfGTd25NrhlkG/jx7dV9EI8A864c8+9mC41qxlo dyCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q27H37/zpcTikuASjIMD4FhxtWUcta8dDgeh38mJ6rY=; b=nfcTX7yCyAkv1b4M9K1B5muxATTNq6/2D0nxhj/raM3b6j+U61WUV5fslSssO757WL Zz27ly0XH17QD3zqIA3+0TkFiZ0nVdMrGI+lpXBWLvnqYCSwtd0DEzsk1AHe1g3QCdwb ePf2cVdiysK5bPVTPH0TSfAckWoryghThI4GwHCB6fTr5MRJwmlYXx2IARhUswkcCH1A snBlqjYqh6Xro1dBxbANzm0A3Q8fI0YGK+z7uH4EC3fmfDFHLYToK7nprzE/UajZrxCr c97DpTHy6/CguwhtXXCBMVaP2j+Xmc5kT8jNlEy0QBnd0MKWhd4+h1Q4+z6Hny4OpJQH 0bCA== X-Gm-Message-State: AOAM531kHml/x8Xugk2eBdsMXUJKgkhKXB1KQ90Poyfs1iYm1ZnXzDOu aOD/qqK0ANnxmMqUXdcek9l36Q6Hgz63asOR X-Received: by 2002:a5d:47ae:: with SMTP id 14mr18321414wrb.378.1612192424811; Mon, 01 Feb 2021 07:13:44 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:44 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 10/12] futex: Add mutex around futex exit Date: Mon, 1 Feb 2021 15:12:12 +0000 Message-Id: <20210201151214.2193508-11-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3f186d974826847a07bc7964d79ec4eded475ad9 upstream. The mutex will be used in subsequent changes to replace the busy looping of a waiter when the futex owner is currently executing the exit cleanup to prevent a potential live lock. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.845798895@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- include/linux/futex.h | 1 + include/linux/sched.h | 1 + kernel/futex.c | 16 ++++++++++++++++ 3 files changed, 18 insertions(+) -- 2.25.1 diff --git a/include/linux/futex.h b/include/linux/futex.h index 805508373fcea..0f294ae63c78c 100644 --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -70,6 +70,7 @@ static inline void futex_init_task(struct task_struct *tsk) INIT_LIST_HEAD(&tsk->pi_state_list); tsk->pi_state_cache = NULL; tsk->futex_state = FUTEX_STATE_OK; + mutex_init(&tsk->futex_exit_mutex); } void futex_exit_recursive(struct task_struct *tsk); diff --git a/include/linux/sched.h b/include/linux/sched.h index aba34bba5e9e3..8c10e97f94fea 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1704,6 +1704,7 @@ struct task_struct { #endif struct list_head pi_state_list; struct futex_pi_state *pi_state_cache; + struct mutex futex_exit_mutex; unsigned int futex_state; #endif #ifdef CONFIG_PERF_EVENTS diff --git a/kernel/futex.c b/kernel/futex.c index feef5ce071aa5..d21b151216aa3 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -3271,11 +3271,22 @@ static void futex_cleanup(struct task_struct *tsk) */ void futex_exit_recursive(struct task_struct *tsk) { + /* If the state is FUTEX_STATE_EXITING then futex_exit_mutex is held */ + if (tsk->futex_state == FUTEX_STATE_EXITING) + mutex_unlock(&tsk->futex_exit_mutex); tsk->futex_state = FUTEX_STATE_DEAD; } static void futex_cleanup_begin(struct task_struct *tsk) { + /* + * Prevent various race issues against a concurrent incoming waiter + * including live locks by forcing the waiter to block on + * tsk->futex_exit_mutex when it observes FUTEX_STATE_EXITING in + * attach_to_pi_owner(). + */ + mutex_lock(&tsk->futex_exit_mutex); + /* * Switch the state to FUTEX_STATE_EXITING under tsk->pi_lock. * @@ -3299,6 +3310,11 @@ static void futex_cleanup_end(struct task_struct *tsk, int state) * take another loop until it becomes visible. */ tsk->futex_state = state; + /* + * Drop the exit protection. This unblocks waiters which observed + * FUTEX_STATE_EXITING to reevaluate the state. + */ + mutex_unlock(&tsk->futex_exit_mutex); } void futex_exec_release(struct task_struct *tsk) From patchwork Mon Feb 1 15:12:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374268 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383191jah; Mon, 1 Feb 2021 07:16:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJzezvLxxk6/OUiB4hofP5ijTx/d2Y42PN0BMoo7drFqMntxc+G3cnsOQuTtzz02/wnh8NvU X-Received: by 2002:a17:906:a453:: with SMTP id cb19mr18258391ejb.459.1612192573309; Mon, 01 Feb 2021 07:16:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192573; cv=none; d=google.com; s=arc-20160816; b=pQSIQRqz2iffkuGi5JGJuonn64rCCjE9yJ0lGz2YPPyPxsu+w9kSoaiifta5drqyAK EE1QVhYfp3yAZ9graQNW7ySyhyEdguJ/PzlkoaePq9ivSNsIhg9c6aCtgbgD8mKud1UY srdzw8BmV9NhH5AkkQL2yyiqm3oKRv4xg/VfNJvLXuxhKoZJU42P5pskDrFe8FmqA/k+ 20HAWGAB/8N1lKwNA6jixxgPSQFDLReO85nNk8+w0qpYtwvH129eBgKMc0RajHdtYFOk geGlnoHt4zSTIGoy4tF4/OJN7XTlY8BJWkwZe6WaaDxPkhW29TQgiz9SDc623z9VUyhW RyGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xELEKrhqhTmDdRYtgPgdfO9RZGP1Kih4SIlHlfBN9xk=; b=BMfsCin8xAonK3ReUoqwje7163sKOvQQVLMSJfEhtRHoQImuVK/lRLJUtX8yqyGGkd WVgzUKBxaUOLuE4axR85uM0b3A8VeyGSVz7MjOpVOg3DXq64F7w6y/o9BzWEh8EIWuej +12d4lOaNotyXjik93HBn+fWN7vvFZfXQXRJO+fsQgO4A8mIjyjga5DzhO3FZhlsBuO9 heBGxSC5cYuB8/T+LrCJe/fT3JIjfWWL9yVzKZ7Go15qOPHoAqJTMo6ovzXdYyu0FEMS yJM9/v3TntC4tFijBuxmV8I1tORk0hpD7EPfSwpbENPKxKqI5bqjWbWW0Hf5mwatLy4H pRxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xvHNKVH5; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.11; Mon, 01 Feb 2021 07:16:13 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xvHNKVH5; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231440AbhBAPPD (ORCPT + 13 others); Mon, 1 Feb 2021 10:15:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231352AbhBAPPA (ORCPT ); Mon, 1 Feb 2021 10:15:00 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2E6DC061797 for ; Mon, 1 Feb 2021 07:13:47 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id m1so10220915wml.2 for ; Mon, 01 Feb 2021 07:13:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xELEKrhqhTmDdRYtgPgdfO9RZGP1Kih4SIlHlfBN9xk=; b=xvHNKVH52mIQidJjuSVbHHFtT7epo7ypTi7PBlUXI0RYeeNUQ9swqA5YUA8xrTCXTK 32DIeKf4AlJhiotmKXi277YWUR3W4GGQ26+ZSueMiRIXq3nUFbR/+pb5CJjYfZE46zPT XOiwEZa2839QI/lm5UhF5iQ80MQmbYhLViDa5kbfRIEqFJFDr6J4SXTa886EmT4GqMGQ qf0P4cb5m4Ud6BxAMi0g3t4xl0yvgjLDc1q2mSUMn7zmoWbfAVNGqvDjXvOitefM3YkT cPJ4KpciJiz1AGWuXZThiWA8XRkKYgVmlW580BuWKyzFUeWajN4u7kitz/mxxvEXnww9 Y7dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xELEKrhqhTmDdRYtgPgdfO9RZGP1Kih4SIlHlfBN9xk=; b=VjqNHRnrKvrCisuYZI6y4qTYPCD6X9PdJpIVDEkIP+aRiEUGwV4W/W1OkfKXDDJoiV 7nQNPaATOLM3t5RME+zvCdfXv25RCyyFnM9HOXoYgO4jHcNIPtjvh+/txKMMf6AAa6w7 nYEIOyNsy5SgB1LOknGJRpx7vMb6xL1enCj7qc5MxWumS8cPg27bnrv3lfE7Z1AbIxrP ijJ1WydmjgIgVMzZely5v0ewP1hD22cpH9BSbE1m59wmsLgGivLL5pIXt1RHXN4Ltg5O bd99Ieoxe1WmFux3ih9+jftnW0GC839KVvSbABnGCZRKFRfT32KFRXgZo2JsB+rXl7jW u99A== X-Gm-Message-State: AOAM532waEPhBjpj+rHF3Uu/4pQ/w/dPvxRSl2YSBeWMcfWdEakIhi1E +SrLgiQV1IK7hAgUBRqIipKX6W3XVcsfesIv X-Received: by 2002:a05:600c:35d6:: with SMTP id r22mr15430041wmq.44.1612192426025; Mon, 01 Feb 2021 07:13:46 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:45 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 11/12] futex: Provide distinct return value when owner is exiting Date: Mon, 1 Feb 2021 15:12:13 +0000 Message-Id: <20210201151214.2193508-12-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit ac31c7ff8624409ba3c4901df9237a616c187a5d upstream. attach_to_pi_owner() returns -EAGAIN for various cases: - Owner task is exiting - Futex value has changed The caller drops the held locks (hash bucket, mmap_sem) and retries the operation. In case of the owner task exiting this can result in a live lock. As a preparatory step for seperating those cases, provide a distinct return value (EBUSY) for the owner exiting case. No functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20191106224556.935606117@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index d21b151216aa3..32d799b9bd205 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1909,12 +1909,13 @@ retry_private: if (!ret) goto retry; goto out; + case -EBUSY: case -EAGAIN: /* * Two reasons for this: - * - Owner is exiting and we just wait for the + * - EBUSY: Owner is exiting and we just wait for the * exit to complete. - * - The user space value changed. + * - EAGAIN: The user space value changed. */ free_pi_state(pi_state); pi_state = NULL; @@ -2580,12 +2581,13 @@ retry_private: goto out_unlock_put_key; case -EFAULT: goto uaddr_faulted; + case -EBUSY: case -EAGAIN: /* * Two reasons for this: - * - Task is exiting and we just wait for the + * - EBUSY: Task is exiting and we just wait for the * exit to complete. - * - The user space value changed. + * - EAGAIN: The user space value changed. */ queue_unlock(hb); put_futex_key(&q.key); From patchwork Mon Feb 1 15:12:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 374266 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1383157jah; Mon, 1 Feb 2021 07:16:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJxbzBxF+6Sy/ceADGjxUWLB/JpGBWCy9B1LctfWPgvT4XITaF+YIZ7Bkzv0/VXG+ZsUre5K X-Received: by 2002:a5d:6686:: with SMTP id l6mr18546318wru.236.1612192571003; Mon, 01 Feb 2021 07:16:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612192570; cv=none; d=google.com; s=arc-20160816; b=ZGYgB6wYvVOKNoM8aQIh60x66As+tYXZDw9KU5D0g0Coy7a9pirfaiAD3sf/VvnZDH VGLHQ5Zy0TnmBBWunfQeROQ5/tt6XkNZOjuqI19iIObQO7BxRDyhh/zc12OLiBCCcsfc EGqFKa68RwYoLV6buPNUE6n+MWVtMtb83gQHBpV8zDJ2rVBUuvA/eogdIGSPQW6XDoCq GYa7uCD7z9qG224+ILQg3Y4ExdpyxNhm8Y9p1gqiG+BWiq23fkL7rgNd5f/NGg8Z2MIm OGFDujvrSV7tTMiGptfJBHg1aj8ORLWJV0n3LG64mmHcMnZXcqmjy/ESRiI46BCRF5P0 l/ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8K2hHwn4TIJXOCXwlxx1E35acoZbLVwNgZTL7vVpJEE=; b=gpp7kQdODtPZLL4NItzz/jOqLSarLN7YhTCYhCMIxCq0Wf6H9B02XlujCDyXVAw5Xa UOjT7tyTVWVq1rRsTiOIG0Nk/jJj3XYTQf1Dv6SE6CvykOXDkErB0Yakl6pqX7UkQq8m ayRhx4lArb1wrJDoH8vOZVi3oDOd1IU0pXply7EoYa8zXWbm3X0kMuEfpa2adgsEQDAz w1MTaoo9IKUpSGV+FxgfkeJO4Fj0hpK+93QGM3kdiZlCmu4Uvtas60wv6QFav8QkjHsA 15e14UG0yTwq0Kg3uk75QHP712PdZFo59pG1a9Wm2vM9v9R8BRfHT5HIAUyNFi0b7FQW f83w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xn0zj9xC; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f29si11574766edj.297.2021.02.01.07.16.10; Mon, 01 Feb 2021 07:16:10 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xn0zj9xC; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231290AbhBAPPC (ORCPT + 13 others); Mon, 1 Feb 2021 10:15:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231499AbhBAPO7 (ORCPT ); Mon, 1 Feb 2021 10:14:59 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA75BC0617A7 for ; Mon, 1 Feb 2021 07:13:48 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id z6so16909976wrq.10 for ; Mon, 01 Feb 2021 07:13:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8K2hHwn4TIJXOCXwlxx1E35acoZbLVwNgZTL7vVpJEE=; b=xn0zj9xCF0jy22v3W7JLDimHJUCOqh2Ocw5gTT/lYXaj8knqmd4GK1Q5TQ4wfcQZyj WxrrNY0+VrlU2/6bFqKIGdeqtS7QtXHhNIC8KAUW/D3KsUEZGoZZ62DyQRwWg4uawBfp h42xwjfOH4PD+2JycZz8oO79JgW90XVkU7M2+RVr4i7XuYMboBUy9Uaf1t2Kgi488fiS cZP8noVxTya7gSvWg1fGoKHLAazhoKLnsu6kD5mmvtDHQteIKYCFShu3HonmYOcOBBfP AR2Zaug09j1FbNszmQ+iAdn39GclP7aYwI9hVPretzMtcRh+VW8pnJM3htSprSOWYos+ ciEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8K2hHwn4TIJXOCXwlxx1E35acoZbLVwNgZTL7vVpJEE=; b=NCqas4NGDSheAil6t+q55wkD32UgaiLGJ7EGzZ/xuesUCFnmybjT6GHRPn8GG1xM8I 0eUELHDnjklCWeolkSoLMY0RQDiWloy6Dqx6jTcLqDIrME6hUmvxQW9ggM1poJsdS45c gg3vUuJ46dOALkEHVDkaWBM5Rw9pbmTj+C7Pp0w76I2BNa+wrD21VmRO1XW+LOaVUo0M d/6lED94a9FsNAnlMZvQoGTZlnHzN1I2ze1fqkFnTVE+14soNl6U0fYmjtHEU/NUpUhS sgExc/EOX1zjjQAweFhgdRt0yYkbC+hiFAcJYoGC+/6z3jnoDSjfZ6qjnswkACAQp/qE v0bw== X-Gm-Message-State: AOAM530e8C4PwPZgH4wdVckSWB66w1hrAXj9aeLhVsYtCJB6cBwCJVuk MykjGtIp3ArkmBPRTTvkWAArWUbkYK9NQ0O+ X-Received: by 2002:a5d:68cc:: with SMTP id p12mr18045262wrw.276.1612192427189; Mon, 01 Feb 2021 07:13:47 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id 192sm23323381wme.27.2021.02.01.07.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 07:13:46 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Oleg Nesterov , Ingo Molnar , Peter Zijlstra , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 12/12] futex: Prevent exit livelock Date: Mon, 1 Feb 2021 15:12:14 +0000 Message-Id: <20210201151214.2193508-13-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210201151214.2193508-1-lee.jones@linaro.org> References: <20210201151214.2193508-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner commit 3ef240eaff36b8119ac9e2ea17cbf41179c930ba upstream. Oleg provided the following test case: int main(void) { struct sched_param sp = {}; sp.sched_priority = 2; assert(sched_setscheduler(0, SCHED_FIFO, &sp) == 0); int lock = vfork(); if (!lock) { sp.sched_priority = 1; assert(sched_setscheduler(0, SCHED_FIFO, &sp) == 0); _exit(0); } syscall(__NR_futex, &lock, FUTEX_LOCK_PI, 0,0,0); return 0; } This creates an unkillable RT process spinning in futex_lock_pi() on a UP machine or if the process is affine to a single CPU. The reason is: parent child set FIFO prio 2 vfork() -> set FIFO prio 1 implies wait_for_child() sched_setscheduler(...) exit() do_exit() .... mm_release() tsk->futex_state = FUTEX_STATE_EXITING; exit_futex(); (NOOP in this case) complete() --> wakes parent sys_futex() loop infinite because tsk->futex_state == FUTEX_STATE_EXITING The same problem can happen just by regular preemption as well: task holds futex ... do_exit() tsk->futex_state = FUTEX_STATE_EXITING; --> preemption (unrelated wakeup of some other higher prio task, e.g. timer) switch_to(other_task) return to user sys_futex() loop infinite as above Just for the fun of it the futex exit cleanup could trigger the wakeup itself before the task sets its futex state to DEAD. To cure this, the handling of the exiting owner is changed so: - A refcount is held on the task - The task pointer is stored in a caller visible location - The caller drops all locks (hash bucket, mmap_sem) and blocks on task::futex_exit_mutex. When the mutex is acquired then the exiting task has completed the cleanup and the state is consistent and can be reevaluated. This is not a pretty solution, but there is no choice other than returning an error code to user space, which would break the state consistency guarantee and open another can of problems including regressions. For stable backports the preparatory commits ac31c7ff8624 .. ba31c1a48538 are required as well, but for anything older than 5.3.y the backports are going to be provided when this hits mainline as the other dependencies for those kernels are definitely not stable material. Fixes: 778e9a9c3e71 ("pi-futex: fix exit races and locking problems") Reported-by: Oleg Nesterov Signed-off-by: Thomas Gleixner Reviewed-by: Ingo Molnar Acked-by: Peter Zijlstra (Intel) Cc: Stable Team Link: https://lkml.kernel.org/r/20191106224557.041676471@linutronix.de Signed-off-by: Greg Kroah-Hartman Signed-off-by: Lee Jones --- kernel/futex.c | 106 ++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 91 insertions(+), 15 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 32d799b9bd205..f1990e2a51e5a 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1067,12 +1067,43 @@ out_state: return 0; } +/** + * wait_for_owner_exiting - Block until the owner has exited + * @exiting: Pointer to the exiting task + * + * Caller must hold a refcount on @exiting. + */ +static void wait_for_owner_exiting(int ret, struct task_struct *exiting) +{ + if (ret != -EBUSY) { + WARN_ON_ONCE(exiting); + return; + } + + if (WARN_ON_ONCE(ret == -EBUSY && !exiting)) + return; + + mutex_lock(&exiting->futex_exit_mutex); + /* + * No point in doing state checking here. If the waiter got here + * while the task was in exec()->exec_futex_release() then it can + * have any FUTEX_STATE_* value when the waiter has acquired the + * mutex. OK, if running, EXITING or DEAD if it reached exit() + * already. Highly unlikely and not a problem. Just one more round + * through the futex maze. + */ + mutex_unlock(&exiting->futex_exit_mutex); + + put_task_struct(exiting); +} + /* * Lookup the task for the TID provided from user space and attach to * it after doing proper sanity checks. */ static int attach_to_pi_owner(u32 uval, union futex_key *key, - struct futex_pi_state **ps) + struct futex_pi_state **ps, + struct task_struct **exiting) { pid_t pid = uval & FUTEX_TID_MASK; struct futex_pi_state *pi_state; @@ -1108,7 +1139,19 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, int ret = (p->futex_state = FUTEX_STATE_DEAD) ? -ESRCH : -EAGAIN; raw_spin_unlock_irq(&p->pi_lock); - put_task_struct(p); + /* + * If the owner task is between FUTEX_STATE_EXITING and + * FUTEX_STATE_DEAD then store the task pointer and keep + * the reference on the task struct. The calling code will + * drop all locks, wait for the task to reach + * FUTEX_STATE_DEAD and then drop the refcount. This is + * required to prevent a live lock when the current task + * preempted the exiting task between the two states. + */ + if (ret == -EBUSY) + *exiting = p; + else + put_task_struct(p); return ret; } @@ -1139,7 +1182,8 @@ static int attach_to_pi_owner(u32 uval, union futex_key *key, } static int lookup_pi_state(u32 uval, struct futex_hash_bucket *hb, - union futex_key *key, struct futex_pi_state **ps) + union futex_key *key, struct futex_pi_state **ps, + struct task_struct **exiting) { struct futex_q *match = futex_top_waiter(hb, key); @@ -1154,7 +1198,7 @@ static int lookup_pi_state(u32 uval, struct futex_hash_bucket *hb, * We are the first waiter - try to look up the owner based on * @uval and attach to it. */ - return attach_to_pi_owner(uval, key, ps); + return attach_to_pi_owner(uval, key, ps, exiting); } static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) @@ -1180,6 +1224,8 @@ static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) * lookup * @task: the task to perform the atomic lock work for. This will * be "current" except in the case of requeue pi. + * @exiting: Pointer to store the task pointer of the owner task + * which is in the middle of exiting * @set_waiters: force setting the FUTEX_WAITERS bit (1) or not (0) * * Return: @@ -1188,11 +1234,17 @@ static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) * <0 - error * * The hb->lock and futex_key refs shall be held by the caller. + * + * @exiting is only set when the return value is -EBUSY. If so, this holds + * a refcount on the exiting task on return and the caller needs to drop it + * after waiting for the exit to complete. */ static int futex_lock_pi_atomic(u32 __user *uaddr, struct futex_hash_bucket *hb, union futex_key *key, struct futex_pi_state **ps, - struct task_struct *task, int set_waiters) + struct task_struct *task, + struct task_struct **exiting, + int set_waiters) { u32 uval, newval, vpid = task_pid_vnr(task); struct futex_q *match; @@ -1262,7 +1314,7 @@ static int futex_lock_pi_atomic(u32 __user *uaddr, struct futex_hash_bucket *hb, * attach to the owner. If that fails, no harm done, we only * set the FUTEX_WAITERS bit in the user space variable. */ - return attach_to_pi_owner(uval, key, ps); + return attach_to_pi_owner(uval, key, ps, exiting); } /** @@ -1688,6 +1740,8 @@ void requeue_pi_wake_futex(struct futex_q *q, union futex_key *key, * @key1: the from futex key * @key2: the to futex key * @ps: address to store the pi_state pointer + * @exiting: Pointer to store the task pointer of the owner task + * which is in the middle of exiting * @set_waiters: force setting the FUTEX_WAITERS bit (1) or not (0) * * Try and get the lock on behalf of the top waiter if we can do it atomically. @@ -1695,16 +1749,20 @@ void requeue_pi_wake_futex(struct futex_q *q, union futex_key *key, * then direct futex_lock_pi_atomic() to force setting the FUTEX_WAITERS bit. * hb1 and hb2 must be held by the caller. * + * @exiting is only set when the return value is -EBUSY. If so, this holds + * a refcount on the exiting task on return and the caller needs to drop it + * after waiting for the exit to complete. + * * Return: * 0 - failed to acquire the lock atomically; * >0 - acquired the lock, return value is vpid of the top_waiter * <0 - error */ -static int futex_proxy_trylock_atomic(u32 __user *pifutex, - struct futex_hash_bucket *hb1, - struct futex_hash_bucket *hb2, - union futex_key *key1, union futex_key *key2, - struct futex_pi_state **ps, int set_waiters) +static int +futex_proxy_trylock_atomic(u32 __user *pifutex, struct futex_hash_bucket *hb1, + struct futex_hash_bucket *hb2, union futex_key *key1, + union futex_key *key2, struct futex_pi_state **ps, + struct task_struct **exiting, int set_waiters) { struct futex_q *top_waiter = NULL; u32 curval; @@ -1741,7 +1799,7 @@ static int futex_proxy_trylock_atomic(u32 __user *pifutex, */ vpid = task_pid_vnr(top_waiter->task); ret = futex_lock_pi_atomic(pifutex, hb2, key2, ps, top_waiter->task, - set_waiters); + exiting, set_waiters); if (ret == 1) { requeue_pi_wake_futex(top_waiter, key2, hb2); return vpid; @@ -1861,6 +1919,8 @@ retry_private: } if (requeue_pi && (task_count - nr_wake < nr_requeue)) { + struct task_struct *exiting = NULL; + /* * Attempt to acquire uaddr2 and wake the top waiter. If we * intend to requeue waiters, force setting the FUTEX_WAITERS @@ -1868,7 +1928,8 @@ retry_private: * faults rather in the requeue loop below. */ ret = futex_proxy_trylock_atomic(uaddr2, hb1, hb2, &key1, - &key2, &pi_state, nr_requeue); + &key2, &pi_state, + &exiting, nr_requeue); /* * At this point the top_waiter has either taken uaddr2 or is @@ -1892,7 +1953,8 @@ retry_private: * rereading and handing potential crap to * lookup_pi_state. */ - ret = lookup_pi_state(ret, hb2, &key2, &pi_state); + ret = lookup_pi_state(ret, hb2, &key2, + &pi_state, &exiting); } switch (ret) { @@ -1923,6 +1985,12 @@ retry_private: hb_waiters_dec(hb2); put_futex_key(&key2); put_futex_key(&key1); + /* + * Handle the case where the owner is in the middle of + * exiting. Wait for the exit to complete otherwise + * this task might loop forever, aka. live lock. + */ + wait_for_owner_exiting(ret, exiting); cond_resched(); goto retry; default: @@ -2545,6 +2613,7 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int trylock) { struct hrtimer_sleeper timeout, *to = NULL; + struct task_struct *exiting = NULL; struct futex_hash_bucket *hb; struct futex_q q = futex_q_init; int res, ret; @@ -2568,7 +2637,8 @@ retry: retry_private: hb = queue_lock(&q); - ret = futex_lock_pi_atomic(uaddr, hb, &q.key, &q.pi_state, current, 0); + ret = futex_lock_pi_atomic(uaddr, hb, &q.key, &q.pi_state, current, + &exiting, 0); if (unlikely(ret)) { /* * Atomic work succeeded and we got the lock, @@ -2591,6 +2661,12 @@ retry_private: */ queue_unlock(hb); put_futex_key(&q.key); + /* + * Handle the case where the owner is in the middle of + * exiting. Wait for the exit to complete otherwise + * this task might loop forever, aka. live lock. + */ + wait_for_owner_exiting(ret, exiting); cond_resched(); goto retry; default: