From patchwork Wed Feb 3 13:45:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375402 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417130ejt; Wed, 3 Feb 2021 05:47:33 -0800 (PST) X-Google-Smtp-Source: ABdhPJwaIcY5e4YYg3fS0sKkfMXX07w7DUELOogqo6s2MhVMhUIcBmVNMhAvQRQUnz8SeMuWPQvh X-Received: by 2002:a05:6402:4312:: with SMTP id m18mr3029483edc.99.1612360053209; Wed, 03 Feb 2021 05:47:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360053; cv=none; d=google.com; s=arc-20160816; b=YB7fLLPphNzqcvVx55fgB2Excra4yHFckkpBSA2CuyVwJI7IhaVFbxnz3Ybppt/JSj oVNotX5YWB2RkDtxTqM2fe4+f9oddknGEPU7rf4La0JYEgUOr/mi2Ve+qQodhU2P0GHQ 1uwJxx60il9VFo68PNFnqrOuguCe9R78MA5sKRjn6PK6K7cpojOT2lTBvGuj8WdfLB62 7Bxq4UILF2ykFKrLuXLF0KrD3v3JiMntqAECfHAdi5PvKhBXJlj4GnGdhTA7z2j/gULW H5bXh0EQjOtVTcCn/g4//W7IwaWgZ3q9zlGjpDNqE7nL/9v9hFtKx1kemO5AhyqGHNBp aYkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KBJgTERN4u5dqqvue5Bwso6/AEy/i3vNaSOQipjk4AE=; b=itcmGdbDG2FPGI5eLerEHBmCGU26G8h4u86eefRP7BKqh52hwr19ILOvEw1uwJw8kn eAYOLawcnaixuvR+YCFXF3H+K3WPeL4AGXHNWF/+SGCFQ2BTZ3h+BAPVqaDjQ+vv8Zbm ChYC3GrsRz36I2N46ovIA97ce9ATvA7PQa6brJMGY5o6dg7QNSkyYPx83o6v1yOaT1Pw BiFIvLocdMz+6wx9YWTwLgppoIUVE930PN64Pjj6VGO/egBuyuiSSQMqymO/nDZRopxw LPHEoFVo6KuAN9qdWmSNlEi0dWTpfj/MyjKlicSz2fELfjxGofUXa95zYuJJiOVcu4dW aN2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dRVIFDjS; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.32; Wed, 03 Feb 2021 05:47:33 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dRVIFDjS; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230248AbhBCNql (ORCPT + 13 others); Wed, 3 Feb 2021 08:46:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232181AbhBCNqb (ORCPT ); Wed, 3 Feb 2021 08:46:31 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EABACC0613ED for ; Wed, 3 Feb 2021 05:45:49 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id c4so21643490wru.9 for ; Wed, 03 Feb 2021 05:45:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KBJgTERN4u5dqqvue5Bwso6/AEy/i3vNaSOQipjk4AE=; b=dRVIFDjSGbXs0Mh7mEKLOWfO04UCEQAkZDoX+x00aNE6UIrK78vB4laOwNUjrCJEYt zpxpFHnBCC4/MD5qJ07Ph0+97AjOtdDOn3NIYwWSYtQNjMNdZDeV4gjEV+KivZ81xN8X bmdE5VBLj3Kanj2ANRhmdgyvA5H2ignr5dn/O6wvu5SgEDyo8jts9pA6Kpgu4II6Uy7C ExAyZaZtSvoiPVWJ88jM1OosW+gfz9vYdnmIRaYltBYxv7MUnTxEgE+PiQn3Mt1qadek 3YY744IYZoAkbiuGRUdE2i4Lqb0UUmz6q+EJbKviFXdXQi0vJvATryWySQX1hyhixBhG Nq2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KBJgTERN4u5dqqvue5Bwso6/AEy/i3vNaSOQipjk4AE=; b=rkxuUNB9cWOb8XGeNnoLwRLz8K3f4f9XzYEgd+f8Rnu5h9lr8B9mPPNkFHLT7PUGHG kqkxDyQ2SYwtFX4eRCGIaLxe49Rd/Okw8BK5DOM2mHB2ciUMdRQRfSdl6a0brXVNGFMv iLkO1bebqMxR86niG2Jox/pAmg6uph1c4nykTsLasanUs2ZxcsJ5R4WwAIgvmI2BMLj6 qlHFQIkv2gI6BpDCBQdtLu4/bkTtCF9Lnl5fR8h5/w72eALvNb/gRkpbi0orIQ1qb7ts v1WyJNVeQICkwSrBG0xBhbWev/86Eok2KzQ4MEVyqx9pFeNLhMX1jfE1Nw0iJFoqmfue qECQ== X-Gm-Message-State: AOAM530rCH62jOOlwHSt0VRJuLW9bCrgDv1/CTvg/PkVhucnfZYLU8Td ekcjJpRQTbFdLnfVTkNzsm+IISHEFAl+eg== X-Received: by 2002:a5d:4f86:: with SMTP id d6mr3683179wru.246.1612359948424; Wed, 03 Feb 2021 05:45:48 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:47 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 01/10] futex,rt_mutex: Provide futex specific rt_mutex API Date: Wed, 3 Feb 2021 13:45:30 +0000 Message-Id: <20210203134539.2583943-2-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra [ Upstream commit 5293c2efda37775346885c7e924d4ef7018ea60b ] Part of what makes futex_unlock_pi() intricate is that rt_mutex_futex_unlock() -> rt_mutex_slowunlock() can drop rt_mutex::wait_lock. This means it cannot rely on the atomicy of wait_lock, which would be preferred in order to not rely on hb->lock so much. The reason rt_mutex_slowunlock() needs to drop wait_lock is because it can race with the rt_mutex fastpath, however futexes have their own fast path. Since futexes already have a bunch of separate rt_mutex accessors, complete that set and implement a rt_mutex variant without fastpath for them. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.702962446@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 30 +++++++++--------- kernel/locking/rtmutex.c | 56 ++++++++++++++++++++++++--------- kernel/locking/rtmutex_common.h | 8 +++-- 3 files changed, 61 insertions(+), 33 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 2ef8c5aef35d0..97799bf825434 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -941,7 +941,7 @@ static void exit_pi_state_list(struct task_struct *curr) pi_state->owner = NULL; raw_spin_unlock_irq(&curr->pi_lock); - rt_mutex_unlock(&pi_state->pi_mutex); + rt_mutex_futex_unlock(&pi_state->pi_mutex); spin_unlock(&hb->lock); @@ -1441,20 +1441,18 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, pi_state->owner = new_owner; raw_spin_unlock(&new_owner->pi_lock); - raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); - - deboost = rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); - /* - * First unlock HB so the waiter does not spin on it once he got woken - * up. Second wake up the waiter before the priority is adjusted. If we - * deboost first (and lose our higher priority), then the task might get - * scheduled away before the wake up can take place. + * We've updated the uservalue, this unlock cannot fail. */ + deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); spin_unlock(&hb->lock); - wake_up_q(&wake_q); - if (deboost) + + if (deboost) { + wake_up_q(&wake_q); rt_mutex_adjust_prio(current); + } return 0; } @@ -2397,7 +2395,7 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) * task acquired the rt_mutex after we removed ourself from the * rt_mutex waiters list. */ - if (rt_mutex_trylock(&q->pi_state->pi_mutex)) { + if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { locked = 1; goto out; } @@ -2721,7 +2719,7 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, if (!trylock) { ret = rt_mutex_timed_futex_lock(&q.pi_state->pi_mutex, to); } else { - ret = rt_mutex_trylock(&q.pi_state->pi_mutex); + ret = rt_mutex_futex_trylock(&q.pi_state->pi_mutex); /* Fixup the trylock return value: */ ret = ret ? 0 : -EWOULDBLOCK; } @@ -2744,7 +2742,7 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, * it and return the fault to userspace. */ if (ret && (rt_mutex_owner(&q.pi_state->pi_mutex) == current)) - rt_mutex_unlock(&q.pi_state->pi_mutex); + rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* Unqueue and drop the lock */ unqueue_me_pi(&q); @@ -3051,7 +3049,7 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, spin_lock(q.lock_ptr); ret = fixup_pi_state_owner(uaddr2, &q, current); if (ret && rt_mutex_owner(&q.pi_state->pi_mutex) == current) - rt_mutex_unlock(&q.pi_state->pi_mutex); + rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* * Drop the reference to the pi state which * the requeue_pi() code acquired for us. @@ -3094,7 +3092,7 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, * userspace. */ if (ret && rt_mutex_owner(pi_mutex) == current) - rt_mutex_unlock(pi_mutex); + rt_mutex_futex_unlock(pi_mutex); /* Unqueue and drop the lock. */ unqueue_me_pi(&q); diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 7615e7722258c..38e6cd23d2e76 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1519,15 +1519,23 @@ EXPORT_SYMBOL_GPL(rt_mutex_lock_interruptible); /* * Futex variant with full deadlock detection. + * Futex variants must not use the fast-path, see __rt_mutex_futex_unlock(). */ -int rt_mutex_timed_futex_lock(struct rt_mutex *lock, +int __sched rt_mutex_timed_futex_lock(struct rt_mutex *lock, struct hrtimer_sleeper *timeout) { might_sleep(); - return rt_mutex_timed_fastlock(lock, TASK_INTERRUPTIBLE, timeout, - RT_MUTEX_FULL_CHAINWALK, - rt_mutex_slowlock); + return rt_mutex_slowlock(lock, TASK_INTERRUPTIBLE, + timeout, RT_MUTEX_FULL_CHAINWALK); +} + +/* + * Futex variant, must not use fastpath. + */ +int __sched rt_mutex_futex_trylock(struct rt_mutex *lock) +{ + return rt_mutex_slowtrylock(lock); } /** @@ -1586,20 +1594,38 @@ void __sched rt_mutex_unlock(struct rt_mutex *lock) EXPORT_SYMBOL_GPL(rt_mutex_unlock); /** - * rt_mutex_futex_unlock - Futex variant of rt_mutex_unlock - * @lock: the rt_mutex to be unlocked - * - * Returns: true/false indicating whether priority adjustment is - * required or not. + * Futex variant, that since futex variants do not use the fast-path, can be + * simple and will not need to retry. */ -bool __sched rt_mutex_futex_unlock(struct rt_mutex *lock, - struct wake_q_head *wqh) +bool __sched __rt_mutex_futex_unlock(struct rt_mutex *lock, + struct wake_q_head *wake_q) { - if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) { - rt_mutex_deadlock_account_unlock(current); - return false; + lockdep_assert_held(&lock->wait_lock); + + debug_rt_mutex_unlock(lock); + + if (!rt_mutex_has_waiters(lock)) { + lock->owner = NULL; + return false; /* done */ + } + + mark_wakeup_next_waiter(wake_q, lock); + return true; /* deboost and wakeups */ +} + +void __sched rt_mutex_futex_unlock(struct rt_mutex *lock) +{ + WAKE_Q(wake_q); + bool deboost; + + raw_spin_lock_irq(&lock->wait_lock); + deboost = __rt_mutex_futex_unlock(lock, &wake_q); + raw_spin_unlock_irq(&lock->wait_lock); + + if (deboost) { + wake_up_q(&wake_q); + rt_mutex_adjust_prio(current); } - return rt_mutex_slowunlock(lock, wqh); } /** diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 14cbafed00142..882d84eda50aa 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -113,8 +113,12 @@ extern int rt_mutex_wait_proxy_lock(struct rt_mutex *lock, extern bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter); extern int rt_mutex_timed_futex_lock(struct rt_mutex *l, struct hrtimer_sleeper *to); -extern bool rt_mutex_futex_unlock(struct rt_mutex *lock, - struct wake_q_head *wqh); +extern int rt_mutex_futex_trylock(struct rt_mutex *l); + +extern void rt_mutex_futex_unlock(struct rt_mutex *lock); +extern bool __rt_mutex_futex_unlock(struct rt_mutex *lock, + struct wake_q_head *wqh); + extern void rt_mutex_adjust_prio(struct task_struct *task); #ifdef CONFIG_DEBUG_RT_MUTEXES From patchwork Wed Feb 3 13:45:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375404 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417172ejt; Wed, 3 Feb 2021 05:47:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJzVcYV3IxY/3vvnqD1StXxfBDWUbldO/AHGKnr0zQpn25/sb4Vl3W9OagJ2o6UYMXix5mxb X-Received: by 2002:aa7:dd57:: with SMTP id o23mr3031649edw.252.1612360055960; Wed, 03 Feb 2021 05:47:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360055; cv=none; d=google.com; s=arc-20160816; b=B4nWtE3J+XIxZKmIMbJV5zM+LS6mTnRQgKQs/g6ZcVaQnIGY5DDrXNblnimyEkGYdE EGnSiDuQfWsZpiribEtKXcD0+ib6OFKDG555vlbrPARY6AE+kuhyzJRkhvEbj61z10Ua b2+rSzC2e0aNJndrNT34Kg2sJAMnxR8+oJmHdkXOc+KINow6R3tA2ba4jtlekmwz4ueL j1w6QHRymSwx5GQ+2r5l3qvdbJ7ktTNYjkOa7fEo8Xbrlu4mw5tFk4jdaWljF3zoZSHS X4FJyRCZ0OsPR0Sw2GH2r7JOi8soW6NPLvZX8t6NZEQ1uoMqgWTb62Z3AhDeMMkuyMAI H52g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6FNiNs5lJeBkWHcEwgaGKVxFwLZYESISi48d/nAi7B0=; b=ms6tN6RQaoDkAK/WF6qYBOx8G8GLw91l39hWavguQ8WhDqtIFoXiaHSNnKr8RAlvMg TFZuZ0q2FunzdfBWAiFinzSg+T3l7R8xNbwwHJgRQ1fo+e0QlL6pJFJTeZ62ts2STS4O KyeFd2qf2KHcDZ1Odq9aWT4xm7e8sW/zsxJ+37YIcQgl92+YvZk5kyBiYA5WXNQH/22m BBM9VzshtzyAokPab71k1xt+0m8cqPcYwRdGEU3RvFhAZnSCkGuCawt8EGokfy9su1cW BekVK7Eb1Q5Esq0v808TQH+6pIY/0y6tixCPDb6AGST8KeTFADCIPU4ExZVT8d4DDE2M g6Pw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dy6YcRnT; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.35; Wed, 03 Feb 2021 05:47:35 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dy6YcRnT; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230123AbhBCNqy (ORCPT + 13 others); Wed, 3 Feb 2021 08:46:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231721AbhBCNqb (ORCPT ); Wed, 3 Feb 2021 08:46:31 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FE2CC061786 for ; Wed, 3 Feb 2021 05:45:51 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id c127so5262965wmf.5 for ; Wed, 03 Feb 2021 05:45:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6FNiNs5lJeBkWHcEwgaGKVxFwLZYESISi48d/nAi7B0=; b=dy6YcRnTIvwFhxdcZuT43NpbGPf7LGpIOS37jhupT4Itan6CM0NW7nqarAhPqDsNM3 IW5k/IUIS77fa7BgLNfOj5EKe59MpMgAULUg7INgr4h/Do3BH5wKB8Y1lmFjmj/ybzO7 +1eWg8sjeXH7drEHBBLDmRsrqVmlaJRBXLVIlmHFM1rMX9Hpn7xdhQ/R43u2gJTwFmgb O/h8m0yd24DbsgKbIXfQHPKi6dWcpON8wCX/VAiZ5q8Ux+aKyq44tqHyNduAJAj1mClD JNz9bZ1LjxcUAbZD4JK34bhwOhcEFWcRPYhFo3zLYfDhXZ1NXYuiWl2da/rjvB/FzX62 jGFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6FNiNs5lJeBkWHcEwgaGKVxFwLZYESISi48d/nAi7B0=; b=clztdXDeT721Zl76Xe2f8Tw1aRqItttFhVDpNu/oRU2AhUmS7q5r0Zaqmg0diOqTL2 OY6uzJkQhABcKp+iBY4j5g/ZQUHD56AzO2LS1ogmkMfpaGE3vIggvKKmkl379vrBpqdP a03Jptmfna2pTdLaY4rHqNn2/UzvXgP00DgU+c/kcRYvQW0GSJbNCMglS3Q7jhUTGCtf D5WQzYnSSJKkvlq02zsnCyeinFfKCUVaoJVCQ6FE68PS2Ezj+5uXvJtQj3YSzs+R7c9a 85OwWCSR5hmH57ACAGKwiV03uYbPby41PomwrPBNl6R/4ze/nxQpi1DbwwsWA7cHAh0A h/bQ== X-Gm-Message-State: AOAM533vCe1eT4nr60ANyguXRUlE3uPlMuufH6CNw6qWz7N5jRlu57hb VXfQJUjIXybSGWY9AFvkYEOW+ggnJiTixg== X-Received: by 2002:a05:600c:4ec7:: with SMTP id g7mr2873543wmq.56.1612359949525; Wed, 03 Feb 2021 05:45:49 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:48 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 02/10] futex: Remove rt_mutex_deadlock_account_*() Date: Wed, 3 Feb 2021 13:45:31 +0000 Message-Id: <20210203134539.2583943-3-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra These are unused and clutter up the code. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.652692478@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/locking/rtmutex-debug.c | 9 -------- kernel/locking/rtmutex-debug.h | 3 --- kernel/locking/rtmutex.c | 42 +++++++++++++--------------------- kernel/locking/rtmutex.h | 2 -- 4 files changed, 16 insertions(+), 40 deletions(-) -- 2.25.1 diff --git a/kernel/locking/rtmutex-debug.c b/kernel/locking/rtmutex-debug.c index 62b6cee8ea7f9..0613c4b1d0596 100644 --- a/kernel/locking/rtmutex-debug.c +++ b/kernel/locking/rtmutex-debug.c @@ -173,12 +173,3 @@ void debug_rt_mutex_init(struct rt_mutex *lock, const char *name) lock->name = name; } -void -rt_mutex_deadlock_account_lock(struct rt_mutex *lock, struct task_struct *task) -{ -} - -void rt_mutex_deadlock_account_unlock(struct task_struct *task) -{ -} - diff --git a/kernel/locking/rtmutex-debug.h b/kernel/locking/rtmutex-debug.h index d0519c3432b67..b585af9a1b508 100644 --- a/kernel/locking/rtmutex-debug.h +++ b/kernel/locking/rtmutex-debug.h @@ -9,9 +9,6 @@ * This file contains macros used solely by rtmutex.c. Debug version. */ -extern void -rt_mutex_deadlock_account_lock(struct rt_mutex *lock, struct task_struct *task); -extern void rt_mutex_deadlock_account_unlock(struct task_struct *task); extern void debug_rt_mutex_init_waiter(struct rt_mutex_waiter *waiter); extern void debug_rt_mutex_free_waiter(struct rt_mutex_waiter *waiter); extern void debug_rt_mutex_init(struct rt_mutex *lock, const char *name); diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 38e6cd23d2e76..de302c580d65d 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -956,8 +956,6 @@ static int try_to_take_rt_mutex(struct rt_mutex *lock, struct task_struct *task, */ rt_mutex_set_owner(lock, task); - rt_mutex_deadlock_account_lock(lock, task); - return 1; } @@ -1365,8 +1363,6 @@ static bool __sched rt_mutex_slowunlock(struct rt_mutex *lock, debug_rt_mutex_unlock(lock); - rt_mutex_deadlock_account_unlock(current); - /* * We must be careful here if the fast path is enabled. If we * have no waiters queued we cannot set owner to NULL here @@ -1432,11 +1428,10 @@ rt_mutex_fastlock(struct rt_mutex *lock, int state, struct hrtimer_sleeper *timeout, enum rtmutex_chainwalk chwalk)) { - if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 0; - } else - return slowfn(lock, state, NULL, RT_MUTEX_MIN_CHAINWALK); + + return slowfn(lock, state, NULL, RT_MUTEX_MIN_CHAINWALK); } static inline int @@ -1448,21 +1443,19 @@ rt_mutex_timed_fastlock(struct rt_mutex *lock, int state, enum rtmutex_chainwalk chwalk)) { if (chwalk == RT_MUTEX_MIN_CHAINWALK && - likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 0; - } else - return slowfn(lock, state, timeout, chwalk); + + return slowfn(lock, state, timeout, chwalk); } static inline int rt_mutex_fasttrylock(struct rt_mutex *lock, int (*slowfn)(struct rt_mutex *lock)) { - if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 1; - } + return slowfn(lock); } @@ -1472,19 +1465,18 @@ rt_mutex_fastunlock(struct rt_mutex *lock, struct wake_q_head *wqh)) { WAKE_Q(wake_q); + bool deboost; - if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) { - rt_mutex_deadlock_account_unlock(current); + if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) + return; - } else { - bool deboost = slowfn(lock, &wake_q); + deboost = slowfn(lock, &wake_q); - wake_up_q(&wake_q); + wake_up_q(&wake_q); - /* Undo pi boosting if necessary: */ - if (deboost) - rt_mutex_adjust_prio(current); - } + /* Undo pi boosting if necessary: */ + if (deboost) + rt_mutex_adjust_prio(current); } /** @@ -1682,7 +1674,6 @@ void rt_mutex_init_proxy_locked(struct rt_mutex *lock, __rt_mutex_init(lock, NULL); debug_rt_mutex_proxy_lock(lock, proxy_owner); rt_mutex_set_owner(lock, proxy_owner); - rt_mutex_deadlock_account_lock(lock, proxy_owner); } /** @@ -1698,7 +1689,6 @@ void rt_mutex_proxy_unlock(struct rt_mutex *lock, { debug_rt_mutex_proxy_unlock(lock); rt_mutex_set_owner(lock, NULL); - rt_mutex_deadlock_account_unlock(proxy_owner); } /** diff --git a/kernel/locking/rtmutex.h b/kernel/locking/rtmutex.h index c4060584c4076..6607802efa8bd 100644 --- a/kernel/locking/rtmutex.h +++ b/kernel/locking/rtmutex.h @@ -11,8 +11,6 @@ */ #define rt_mutex_deadlock_check(l) (0) -#define rt_mutex_deadlock_account_lock(m, t) do { } while (0) -#define rt_mutex_deadlock_account_unlock(l) do { } while (0) #define debug_rt_mutex_init_waiter(w) do { } while (0) #define debug_rt_mutex_free_waiter(w) do { } while (0) #define debug_rt_mutex_lock(l) do { } while (0) From patchwork Wed Feb 3 13:45:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375405 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417194ejt; Wed, 3 Feb 2021 05:47:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJz6BFT/AdAyxM2vbF6MhNPFxAbs39pJzME8pcQZ3BifkTC8faAv9DaxSIaHDAuShN+SiQo/ X-Received: by 2002:a17:906:33c5:: with SMTP id w5mr3335928eja.319.1612360057565; Wed, 03 Feb 2021 05:47:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360057; cv=none; d=google.com; s=arc-20160816; b=BYQvwDLS6DzfP0/DF8ueaDrMoQ2+FJ/IyYoF/dGKABeE02Ax77YtwCPfMj8h5y7PLG Po7KJ52gOKdwYEFFfFLxP8l2S+5BCuvmKySL5yl8TYR0GUr1PMoEzLWVSBw0LBFy6yB0 VainZ1Hvu8mw0PA6cC75K3N2b3yS/zxqn7p/LMEhkSBsWmg/hCF1+XOBNKJqgMuFSMBg C91T0Ul0lRXoe1Pfa+vk/UoMwoaEAkYYiupVhwNn6pBiYyxG87YeKdhvx0E/tczwBfxS q1JnxBj2T55qHe52qJ+2ap4RFBugaIjVdd8zyR7vpPCmaHiPfdbnYIJXzaSHL9UsuiFQ wGog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fJ5xguPQGJ7hpmRtzwhPbme9DSuJNTs4eyp+KrTETFQ=; b=FOm/LO8K3w70NEmB5nLxNQjHxuuwqXD/G9m1Ur36iELYNoXZB4aIGAZY3KrCP6tz7P qyxaTcnP5FepHpo67ODqu0rMN+rr7fuJOobD1fyVZaxAULQ6nfa5IwhxFx13K3FuIP+Z 7drYCFgD+9+Z7C6kANX6MDciV6amY41SXEYUiYqu70MV6mFSTnkkSBOAp0KwDJm2tOwx M7/C3+SUrMoo2W+HLY3nYpgB7eGUfGBlDAy/QBHMGlp7AaEZup0mIoHU4s2Jppr/q3HW AnP5ZPGs+tb/fShNVDypVlFWSzsNezNmIB2cFH8NSWVQ3SYjApM03wEptTY61uj8ZazG tROA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Mtl9Cco0; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.36; Wed, 03 Feb 2021 05:47:37 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Mtl9Cco0; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231535AbhBCNq4 (ORCPT + 13 others); Wed, 3 Feb 2021 08:46:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232161AbhBCNqc (ORCPT ); Wed, 3 Feb 2021 08:46:32 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 380CDC061788 for ; Wed, 3 Feb 2021 05:45:52 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id w4so4284981wmi.4 for ; Wed, 03 Feb 2021 05:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fJ5xguPQGJ7hpmRtzwhPbme9DSuJNTs4eyp+KrTETFQ=; b=Mtl9Cco0lsJuYn+jHqn9Khwf8oe/R86CMV84xwwMxRc/zIm5/Uv7pjWGjdnUn8ZaF7 CufHmFitI80Urpl44PLcGPXU5taEwk4cvAaQ2vFnSjKPhxFJZyyl67OWjnkE01pftdYc vmAm3cAaQkABYwrdktBgzQi+dcZ5SNbFN4wqK2kt+tTbHWMP1MD7o+DjJXR6D+SaRneb NWMYHwHsBkGR+08cPDxXgeMRUy/xu3SOpAtMTVdoVWBbBminhi6tvsir6gHYTxIlBa2e Zqh5pYJDEpuEaCxSfEHj8duD07nmjXY0dRTe/CX4tBIZToXCpONSYm1zMB6MuOE6DLpL A3xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fJ5xguPQGJ7hpmRtzwhPbme9DSuJNTs4eyp+KrTETFQ=; b=aR/usX/t4iN1Zt43cpFo+OJQ42R2HpIM8RcRkkm/3OaMDilkeiom/Sr20dg/PkUk19 +BPB74oXxwKIaQjNBPZKd5xs2jwDWV682Us1VBz2jnykzljCeF1c/ooA5L1BZ5eoYXBG Upzxt18ZP/A5tmGzfES4BjR+9QvjmEfUHFuy3LYjWaVgaiZJsaXjwMpF1Nm+xL2fxHLn xEKtgJYg8yBYeII06fBdAerulDSs9rzxxLlcOnw+bMn8J2Lhw3l7laA9Znw5AYBnp80s e6t1L6mz0O4kKVb0JhYEO6jTogzLk2PrSK4Yx+zTb0XUhNdJUsF8Gh+WKgp/G7Fiz3oa dW2g== X-Gm-Message-State: AOAM533lYiscMpMkzWaAT5lDgo3NHTEopNLrlAg7LZ2CkKu7pxkItitF UFaJ0CXvrd2zecTtCzgT+So2ik35ukEowQ== X-Received: by 2002:a1c:8109:: with SMTP id c9mr2853937wmd.137.1612359950588; Wed, 03 Feb 2021 05:45:50 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:50 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 03/10] futex: Rework inconsistent rt_mutex/futex_q state Date: Wed, 3 Feb 2021 13:45:32 +0000 Message-Id: <20210203134539.2583943-4-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra [Upstream commit 73d786bd043ebc855f349c81ea805f6b11cbf2aa ] There is a weird state in the futex_unlock_pi() path when it interleaves with a concurrent futex_lock_pi() at the point where it drops hb->lock. In this case, it can happen that the rt_mutex wait_list and the futex_q disagree on pending waiters, in particular rt_mutex will find no pending waiters where futex_q thinks there are. In this case the rt_mutex unlock code cannot assign an owner. The futex side fixup code has to cleanup the inconsistencies with quite a bunch of interesting corner cases. Simplify all this by changing wake_futex_pi() to return -EAGAIN when this situation occurs. This then gives the futex_lock_pi() code the opportunity to continue and the retried futex_unlock_pi() will now observe a coherent state. The only problem is that this breaks RT timeliness guarantees. That is, consider the following scenario: T1 and T2 are both pinned to CPU0. prio(T2) > prio(T1) CPU0 T1 lock_pi() queue_me() <- Waiter is visible preemption T2 unlock_pi() loops with -EAGAIN forever Which is undesirable for PI primitives. Future patches will rectify this. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.850383690@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 50 ++++++++++++++------------------------------------ 1 file changed, 14 insertions(+), 36 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 97799bf825434..2594bc4dc5a19 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1394,12 +1394,19 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, new_owner = rt_mutex_next_owner(&pi_state->pi_mutex); /* - * It is possible that the next waiter (the one that brought - * this owner to the kernel) timed out and is no longer - * waiting on the lock. + * When we interleave with futex_lock_pi() where it does + * rt_mutex_timed_futex_lock(), we might observe @this futex_q waiter, + * but the rt_mutex's wait_list can be empty (either still, or again, + * depending on which side we land). + * + * When this happens, give up our locks and try again, giving the + * futex_lock_pi() instance time to complete, either by waiting on the + * rtmutex or removing itself from the futex queue. */ - if (!new_owner) - new_owner = this->task; + if (!new_owner) { + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); + return -EAGAIN; + } /* * We pass it to the next owner. The WAITERS bit is always @@ -2372,7 +2379,6 @@ static long futex_wait_restart(struct restart_block *restart); */ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) { - struct task_struct *owner; int ret = 0; if (locked) { @@ -2385,44 +2391,16 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) goto out; } - /* - * Catch the rare case, where the lock was released when we were on the - * way back before we locked the hash bucket. - */ - if (q->pi_state->owner == current) { - /* - * Try to get the rt_mutex now. This might fail as some other - * task acquired the rt_mutex after we removed ourself from the - * rt_mutex waiters list. - */ - if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { - locked = 1; - goto out; - } - - /* - * pi_state is incorrect, some other task did a lock steal and - * we returned due to timeout or signal without taking the - * rt_mutex. Too late. - */ - raw_spin_lock_irq(&q->pi_state->pi_mutex.wait_lock); - owner = rt_mutex_owner(&q->pi_state->pi_mutex); - if (!owner) - owner = rt_mutex_next_owner(&q->pi_state->pi_mutex); - raw_spin_unlock_irq(&q->pi_state->pi_mutex.wait_lock); - ret = fixup_pi_state_owner(uaddr, q, owner); - goto out; - } - /* * Paranoia check. If we did not take the lock, then we should not be * the owner of the rt_mutex. */ - if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) + if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) { printk(KERN_ERR "fixup_owner: ret = %d pi-mutex: %p " "pi-state %p\n", ret, q->pi_state->pi_mutex.owner, q->pi_state->owner); + } out: return ret ? ret : locked; From patchwork Wed Feb 3 13:45:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375406 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417197ejt; Wed, 3 Feb 2021 05:47:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJyl4sFgNYqpzXhw+LHRTQHswpuv78iwXvQCOGiSmq9NSpiDlURlhiwrQLn/SVVuV1Tg2ZZA X-Received: by 2002:a05:6402:2707:: with SMTP id y7mr2911586edd.5.1612360058175; Wed, 03 Feb 2021 05:47:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360058; cv=none; d=google.com; s=arc-20160816; b=lMi2xQ9Epdf85bHxYzfMhRFNSDdZ+pvpNjREOC4pk5rATW3bJxtwOvVuS9+/rPSbeJ B9/AJ7WP2IrDOUjkpcQfZLEU4R2kWjHG1J/rHWh1Sn4iztL/jk58H1oPOh57STHZAG1m PmcE5MrcA79bjGp+D4JjyIFdUn1DVKaeC4gHvW//hSbdBzkGDnZGn576AtVstIubYYp8 +C4BLs3u7JmFI1LrfC/IQyjTucH0MwloBdl7m8gsFR+iUlJw5txFjgNR/XDo0A/hUiOF SUPFPOqYgAiZhlQa/bO64TmDoRn9gVQ9EiHny6SNJHD/S/PcLvHNo2nFyk/KLbdisEEZ cnrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Z+STRFMFB3+xvUfGDHfXNdOi4XMfKGKJ3ao/IxPLzCA=; b=nXDLs3wlp+2xMoWQ18SCcqr1wYI1Qg8FZe6j2d/3bSP0Y7YlwIH2KBQuaJwiDwigvu 6zt2qvjW3vQSHcGdaODBzNkzyKnZQqCUOz4g6oqGk6qqsEFcUhrkOtJ4yLGBvxpiToSZ clke9/kNIPWMlrAGxtsQASJMSY/8+1S8nk9Soav9nrSFCxmBv1MCgCGn6mqtlQNivTIX WLNrrkdaJ7cLMSuCYMXruTKCOxOnQZ8eqEOpuJuOQynSwP2OG4s84baAiMe/j9/MVklm dVv1rWBTTLYbkrR7DWm4o+GT1RI4nD1T/DRQKLx5Zl/BigPPpH0pRYKycopY5u+yGVpY AfAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="g/W1GWRM"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.37; Wed, 03 Feb 2021 05:47:38 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="g/W1GWRM"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231721AbhBCNq5 (ORCPT + 13 others); Wed, 3 Feb 2021 08:46:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232181AbhBCNqn (ORCPT ); Wed, 3 Feb 2021 08:46:43 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78BBCC06178A for ; Wed, 3 Feb 2021 05:45:53 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id u14so5275933wmq.4 for ; Wed, 03 Feb 2021 05:45:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z+STRFMFB3+xvUfGDHfXNdOi4XMfKGKJ3ao/IxPLzCA=; b=g/W1GWRMJsnxZfHfdReS4rRay3pBK2BJXWxYCjtjLqow/E40jf197WalXE3DHTMqiE KvIFp9lloeF2vd8lea3OhBCpzIy3zNT5Q22wLAoL2gIHxZXnEdHb48Daa75Od9Rkk6gk xPYXJpyGkyPtHXGUaPxhchL+fi2roZx+2Q4N3gzXUu/zluW2YsBCLRNrR2yPdD0S7B1J v0FHauB+4cvtI8CCOr6BtCxCCUYaOxjokwm5xDpQNk+fucjjDyzGoOMk51/2FRBmq9ZR WEndaLZHFTLpQ0jfYJ50/2PZIOH87zm6B2IbkN4/oQjW7ULxw8IBw0g1V2xwcretvQUm poTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z+STRFMFB3+xvUfGDHfXNdOi4XMfKGKJ3ao/IxPLzCA=; b=g/xL+JK6FhQ6SbjK9Vc0LNqp90XZdrVVY+U4MRsFF4JrKRrNF8D1dgg3nkLarLg3S6 GlH4e13XhtLOr8qKfbPdk/iO4zUtGZKEOa61AIaWxU/JpT0Fy8UDsZnvO/WV6GVNgoJo d13g+Si6Keb5LAyuOzEDHbVW7iHUGJdw7/rcroXO7VVsTOjwYxqO73ofQ3Q/ollIz9dk QwLHCkE6J+zdB5aP9arEEafGnIhuhi+lzXfrrffAo3SVlatJSHco7dYoz5JP+rgXWpH6 6sy+Bimp01rvUBi+Dwe4UHbuj/n7jUwWbb6U5MAr5gEgbSS9TAl+0yS7PvwNJZJF53ug jypQ== X-Gm-Message-State: AOAM5305tTRW/wv+GWWqRLkbUunIdgN3U/LhgreuqMNrx9gRz2NbQw8u pjt16CDmECHVQjzpM/6n7twutg2yQu9Gqg== X-Received: by 2002:a1c:541d:: with SMTP id i29mr2777357wmb.19.1612359951655; Wed, 03 Feb 2021 05:45:51 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:51 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , Julia Cartwright , Gratian Crisan , Thomas Gleixner , Darren Hart , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 04/10] futex: Avoid violating the 10th rule of futex Date: Wed, 3 Feb 2021 13:45:33 +0000 Message-Id: <20210203134539.2583943-5-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra commit c1e2f0eaf015fb7076d51a339011f2383e6dd389 upstream. Julia reported futex state corruption in the following scenario: waiter waker stealer (prio > waiter) futex(WAIT_REQUEUE_PI, uaddr, uaddr2, timeout=[N ms]) futex_wait_requeue_pi() futex_wait_queue_me() freezable_schedule() futex(LOCK_PI, uaddr2) futex(CMP_REQUEUE_PI, uaddr, uaddr2, 1, 0) /* requeues waiter to uaddr2 */ futex(UNLOCK_PI, uaddr2) wake_futex_pi() cmp_futex_value_locked(uaddr2, waiter) wake_up_q() task> futex(LOCK_PI, uaddr2) __rt_mutex_start_proxy_lock() try_to_take_rt_mutex() /* steals lock */ rt_mutex_set_owner(lock, stealer) rt_mutex_wait_proxy_lock() __rt_mutex_slowlock() try_to_take_rt_mutex() /* fails, lock held by stealer */ if (timeout && !timeout->task) return -ETIMEDOUT; fixup_owner() /* lock wasn't acquired, so, fixup_pi_state_owner skipped */ return -ETIMEDOUT; /* At this point, we've returned -ETIMEDOUT to userspace, but the * futex word shows waiter to be the owner, and the pi_mutex has * stealer as the owner */ futex_lock(LOCK_PI, uaddr2) -> bails with EDEADLK, futex word says we're owner. And suggested that what commit: 73d786bd043e ("futex: Rework inconsistent rt_mutex/futex_q state") removes from fixup_owner() looks to be just what is needed. And indeed it is -- I completely missed that requeue_pi could also result in this case. So we need to restore that, except that subsequent patches, like commit: 16ffa12d7425 ("futex: Pull rt_mutex_futex_unlock() out from under hb->lock") changed all the locking rules. Even without that, the sequence: - if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { - locked = 1; - goto out; - } - raw_spin_lock_irq(&q->pi_state->pi_mutex.wait_lock); - owner = rt_mutex_owner(&q->pi_state->pi_mutex); - if (!owner) - owner = rt_mutex_next_owner(&q->pi_state->pi_mutex); - raw_spin_unlock_irq(&q->pi_state->pi_mutex.wait_lock); - ret = fixup_pi_state_owner(uaddr, q, owner); already suggests there were races; otherwise we'd never have to look at next_owner. So instead of doing 3 consecutive wait_lock sections with who knows what races, we do it all in a single section. Additionally, the usage of pi_state->owner in fixup_owner() was only safe because only the rt_mutex owner would modify it, which this additional case wrecks. Luckily the values can only change away and not to the value we're testing, this means we can do a speculative test and double check once we have the wait_lock. Fixes: 73d786bd043e ("futex: Rework inconsistent rt_mutex/futex_q state") Reported-by: Julia Cartwright Reported-by: Gratian Crisan Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Thomas Gleixner Tested-by: Julia Cartwright Tested-by: Gratian Crisan Cc: Darren Hart Link: https://lkml.kernel.org/r/20171208124939.7livp7no2ov65rrc@hirez.programming.kicks-ass.net Signed-off-by: Greg Kroah-Hartman [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 80 +++++++++++++++++++++++++++------ kernel/locking/rtmutex.c | 26 ++++++++--- kernel/locking/rtmutex_common.h | 1 + 3 files changed, 87 insertions(+), 20 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 2594bc4dc5a19..8b137505fb502 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2262,30 +2262,34 @@ static void unqueue_me_pi(struct futex_q *q) spin_unlock(q->lock_ptr); } -/* - * Fixup the pi_state owner with the new owner. - * - * Must be called with hash bucket lock held and mm->sem held for non - * private futexes. - */ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, - struct task_struct *newowner) + struct task_struct *argowner) { - u32 newtid = task_pid_vnr(newowner) | FUTEX_WAITERS; struct futex_pi_state *pi_state = q->pi_state; - struct task_struct *oldowner = pi_state->owner; u32 uval, uninitialized_var(curval), newval; + struct task_struct *oldowner, *newowner; + u32 newtid; int ret; + lockdep_assert_held(q->lock_ptr); + + oldowner = pi_state->owner; /* Owner died? */ if (!pi_state->owner) newtid |= FUTEX_OWNER_DIED; /* - * We are here either because we stole the rtmutex from the - * previous highest priority waiter or we are the highest priority - * waiter but failed to get the rtmutex the first time. - * We have to replace the newowner TID in the user space variable. + * We are here because either: + * + * - we stole the lock and pi_state->owner needs updating to reflect + * that (@argowner == current), + * + * or: + * + * - someone stole our lock and we need to fix things to point to the + * new owner (@argowner == NULL). + * + * Either way, we have to replace the TID in the user space variable. * This must be atomic as we have to preserve the owner died bit here. * * Note: We write the user space value _before_ changing the pi_state @@ -2299,6 +2303,39 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, * in lookup_pi_state. */ retry: + if (!argowner) { + if (oldowner != current) { + /* + * We raced against a concurrent self; things are + * already fixed up. Nothing to do. + */ + return 0; + } + + if (__rt_mutex_futex_trylock(&pi_state->pi_mutex)) { + /* We got the lock after all, nothing to fix. */ + return 0; + } + + /* + * Since we just failed the trylock; there must be an owner. + */ + newowner = rt_mutex_owner(&pi_state->pi_mutex); + BUG_ON(!newowner); + } else { + WARN_ON_ONCE(argowner != current); + if (oldowner == current) { + /* + * We raced against a concurrent self; things are + * already fixed up. Nothing to do. + */ + return 0; + } + newowner = argowner; + } + + newtid = task_pid_vnr(newowner) | FUTEX_WAITERS; + if (get_futex_value_locked(&uval, uaddr)) goto handle_fault; @@ -2385,12 +2422,29 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) /* * Got the lock. We might not be the anticipated owner if we * did a lock-steal - fix up the PI-state in that case: + * + * Speculative pi_state->owner read (we don't hold wait_lock); + * since we own the lock pi_state->owner == current is the + * stable state, anything else needs more attention. */ if (q->pi_state->owner != current) ret = fixup_pi_state_owner(uaddr, q, current); goto out; } + /* + * If we didn't get the lock; check if anybody stole it from us. In + * that case, we need to fix up the uval to point to them instead of + * us, otherwise bad things happen. [10] + * + * Another speculative read; pi_state->owner == current is unstable + * but needs our attention. + */ + if (q->pi_state->owner == current) { + ret = fixup_pi_state_owner(uaddr, q, NULL); + goto out; + } + /* * Paranoia check. If we did not take the lock, then we should not be * the owner of the rt_mutex. diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index de302c580d65d..d295821ed4cc8 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1314,6 +1314,19 @@ rt_mutex_slowlock(struct rt_mutex *lock, int state, return ret; } +static inline int __rt_mutex_slowtrylock(struct rt_mutex *lock) +{ + int ret = try_to_take_rt_mutex(lock, current, NULL); + + /* + * try_to_take_rt_mutex() sets the lock waiters bit + * unconditionally. Clean this up. + */ + fixup_rt_mutex_waiters(lock); + + return ret; +} + /* * Slow path try-lock function: */ @@ -1336,13 +1349,7 @@ static inline int rt_mutex_slowtrylock(struct rt_mutex *lock) */ raw_spin_lock_irqsave(&lock->wait_lock, flags); - ret = try_to_take_rt_mutex(lock, current, NULL); - - /* - * try_to_take_rt_mutex() sets the lock waiters bit - * unconditionally. Clean this up. - */ - fixup_rt_mutex_waiters(lock); + ret = __rt_mutex_slowtrylock(lock); raw_spin_unlock_irqrestore(&lock->wait_lock, flags); @@ -1530,6 +1537,11 @@ int __sched rt_mutex_futex_trylock(struct rt_mutex *lock) return rt_mutex_slowtrylock(lock); } +int __sched __rt_mutex_futex_trylock(struct rt_mutex *lock) +{ + return __rt_mutex_slowtrylock(lock); +} + /** * rt_mutex_timed_lock - lock a rt_mutex interruptible * the timeout structure is provided diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 882d84eda50aa..09991287491d1 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -114,6 +114,7 @@ extern bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter); extern int rt_mutex_timed_futex_lock(struct rt_mutex *l, struct hrtimer_sleeper *to); extern int rt_mutex_futex_trylock(struct rt_mutex *l); +extern int __rt_mutex_futex_trylock(struct rt_mutex *l); extern void rt_mutex_futex_unlock(struct rt_mutex *lock); extern bool __rt_mutex_futex_unlock(struct rt_mutex *lock, From patchwork Wed Feb 3 13:45:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375412 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp418302ejt; Wed, 3 Feb 2021 05:49:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJyLKK1d9EWpkzUPdk8KbXyQCdqeVqiPHH0eZ2ITkwKHfrO3WHz82V55VJ28/lS0mWOi9bzr X-Received: by 2002:a17:906:af86:: with SMTP id mj6mr1194934ejb.509.1612360160111; Wed, 03 Feb 2021 05:49:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360160; cv=none; d=google.com; s=arc-20160816; b=W3ZC15R+cHA4PMn2Bzh6aYvHv3mJ0tiVme8/PEBA0hRHcHeNjPpxXZANrDGK+9AETV 8nxutPZ4LCFEdFDoC3/jqnzjhnbMPWqehBEbl7BM3maDEMUtuQlxyjbw7X2RfE0PyUg1 rDsBueJe1RLenn9w4IXy1v66e4JV+r3NYk4H3Mu5Cr8w7yRTnQ87SBBRpa89uLLmDK3H BQk0DXPyw4WOgUnyCa50PMgx0qb4wgtMCdStI3HNJEqb7yoWTDmwys9gb5N/UNt1alWg xRmBR1ndI2recOAs8ZbwD5mcgI1rNQ9Qhl8T+uzmMKTuQNOEdfgdhsZUkcds4ClNj+FY ujoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=shpeaE1Az2Txha1mF0lLFgmVrxxkpcKGoNrqgxMpIXg=; b=f69Nv+NntprWQ1ihMnMEJg5Zb9fQ24iqu9Q6OGXPb0HFnjfyOKILbPAiWYBf8Icwn0 htFwwYvma7TqQhMMGfYV6XXwBS1feQa0lffdW3OWoP+t8FyaWcODdealIcJ/vxGHy3wK FFCxDU9djmAQ46tx3LsMcRSgHo5kl5OZK6+yevCkJx6Mh4iQJhPz3cu3yyX4QmPs/Mlu M8tEtxfddJ6M2IgTL3nAXM4gn+LJSDZS9WPkfz7jfhXYC2yQmKO8aCxJ/2KVQMJ1nMJl qqmDWHipkLIsp78qIse6dB3CMda9WIvi5Py/l13XuD0vwurbk/isOLcqV4bSsIYmdRVJ sVdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Qa3hzvqV; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id g19si1469620ejz.742.2021.02.03.05.49.19; Wed, 03 Feb 2021 05:49:20 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Qa3hzvqV; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231629AbhBCNrP (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232182AbhBCNqn (ORCPT ); Wed, 3 Feb 2021 08:46:43 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CE95C06178B for ; Wed, 3 Feb 2021 05:45:54 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id o10so3996600wmc.1 for ; Wed, 03 Feb 2021 05:45:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=shpeaE1Az2Txha1mF0lLFgmVrxxkpcKGoNrqgxMpIXg=; b=Qa3hzvqV8OBIHpAT4INGETBQtIOw7lrqD8+uUkzmqDN9yuM1J6fdHHbL+lGLyTNEKb MEd/TUFRyRhoPvolmxnAHLgR5cRZTRgIp8DXVhnLdbrd9P+Zue2cu+qcYW6LJsQlHHSn b5ceLNWqOQIay0GXUVSHBCPJSBUak4uH3dLaW1xnzgszVICQ73F2jzPBopB88Ir4pnIE DAD2m6V+2sBxnVs5M43Mh6URUsJTvhtK57w2XrcN7w4/4FVuV5MaXfPDMjGcKchjIjoZ Xntw4c0j4zNTNfzHM4di1q09hD+caDnaLDkCMO0KLeC518svncoadT3VKEi3HzIcoq7F SCZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=shpeaE1Az2Txha1mF0lLFgmVrxxkpcKGoNrqgxMpIXg=; b=NXkh8crWVCeosaqUsfEREltd2S0TxC5TIRkWtgcKCBe+nyWAI473A+cq9suVrdEtwS Hqk0DS1buJpGv7rotw8Vpsszgrzde5L2XgcFJ7wpTgPwsGW7ZZZG8tTBjSOeMHiQzhZJ Zxwd/a4cl1VChu4zzM7SuELXPf8fTFhc0oAenLB+rtdwTdhxjRkG2Y2RA2Rdgoykg/dE dYaL+EByXMQd1/NQ823nl+Q8kSeXWXw58nBTHD4SD2uaKkgm2LxDZvR8X4TUTLY4xhCP skAc65LCR/QoHNhAeC2q7QFw7A2Vs7jbXES/+av3sGLsJQ0H7pHfkK0Te+2NJcy+oHEM a/Wg== X-Gm-Message-State: AOAM530z1loKexfl+5iTapIBj2vI+Tus9CDlZHN+qtPFzORqwAxgpYFP Huz49K0SExGB1J7jedVypX5A6MIGdQvBfA== X-Received: by 2002:a1c:25c2:: with SMTP id l185mr2905518wml.62.1612359952877; Wed, 03 Feb 2021 05:45:52 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:52 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 05/10] futex: Replace pointless printk in fixup_owner() Date: Wed, 3 Feb 2021 13:45:34 +0000 Message-Id: <20210203134539.2583943-6-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 04b79c55201f02ffd675e1231d731365e335c307 ] If that unexpected case of inconsistent arguments ever happens then the futex state is left completely inconsistent and the printk is not really helpful. Replace it with a warning and make the state consistent. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 8b137505fb502..e44203956d54c 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2447,14 +2447,10 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) /* * Paranoia check. If we did not take the lock, then we should not be - * the owner of the rt_mutex. + * the owner of the rt_mutex. Warn and establish consistent state. */ - if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) { - printk(KERN_ERR "fixup_owner: ret = %d pi-mutex: %p " - "pi-state %p\n", ret, - q->pi_state->pi_mutex.owner, - q->pi_state->owner); - } + if (WARN_ON_ONCE(rt_mutex_owner(&q->pi_state->pi_mutex) == current)) + return fixup_pi_state_owner(uaddr, q, current); out: return ret ? ret : locked; From patchwork Wed Feb 3 13:45:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375409 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417319ejt; Wed, 3 Feb 2021 05:47:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJwSbPimXdzHwvSWL9P8jq/ICAECAoLQdrZHSUT29WuPLWewxbRnOYvjweEnHsP/GnXGNnT3 X-Received: by 2002:a05:6402:1bc7:: with SMTP id ch7mr3107136edb.124.1612360068584; Wed, 03 Feb 2021 05:47:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360068; cv=none; d=google.com; s=arc-20160816; b=OCpmwYKBsFZUKoCI15IVNmKQPG1+2hnAQu15K3Q61NJH2e8r0xq5GzxENhjZWI92xt otv8Qw0GdYaZDKjCRvK7dM1+soMYDEmDR/NACefhvf1eKw4T9p03+bmZAMvX764Z0wR5 q3F7+pov3rcT0djubaZzvqbg61STF4thddbaZTXHUKy4CxiFWIqu+NMu3hBPaG/FPaQK jEv0cyEvgCxLArfBX00+JKuji5dqG1HOaGbQAs1gRLs4wjifT+CbGguyN7TLn4Vs/ced /pO8g3wiZ4hHC8Lat4g0OXIyln31fq/CSd+Xci8hQO+MFbxfipzAwa1nOGNB61C6chSd +6qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3a5y1/WHdcXkbOPADuocB1pmZN0ZLdkklQNTVAL0OLE=; b=cNL4zgxy5+enNjTDOzwfUJcWtpMlx6NkulDZxdOWWXFkCbjoct0/jlptUlogJHSBpx XNI0coKe8c/pVB4Noac1UCxV0LbS7frb7svCrKLMk6H9HgZbMPnIpWzvlQraybR+0ML2 Pl1uFRPBB5TVo6SqYMI1sPgOsR9vZUF65bRR+LZ4I3j1Ie5VzM0Qfzn+lnESFo+nc4Xc wP9e4P8rXsLiDoAAnHwpEghei6u7f1vsCeJsYNRNN57xoGmpKLp3gjTzDJ5wItVsCT8B 1jxtnfMe3A34KwXGKd7hDPmt84ZaBX6dVoA7JWjCaE1hxUHNZbXRQH6baZEO5XmEtEAI hJQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Bq1gJ9ma; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.48; Wed, 03 Feb 2021 05:47:48 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Bq1gJ9ma; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232009AbhBCNrh (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231881AbhBCNrT (ORCPT ); Wed, 3 Feb 2021 08:47:19 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9C1FC06178C for ; Wed, 3 Feb 2021 05:45:55 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id u14so5276028wmq.4 for ; Wed, 03 Feb 2021 05:45:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3a5y1/WHdcXkbOPADuocB1pmZN0ZLdkklQNTVAL0OLE=; b=Bq1gJ9maLxf7KN7ZzMhJa34j/cJeCeZ/7HQZ/2lHf6gW0OzmQUYItx1zCaF5q1ION4 A3Ug1eKzXwyxwWvIhlMLmgAtBFZ8AE9fSVQ60SL3NsmUmYA4yZoFBPqBLBiCZVThTjva Ldzk4U5l1KWQPKDgvkCBgnUYDDcVc2yNyPnB1evGUYWaPIvccfUdJFe90wAJ9dv/B4hZ JS13ZJS0Vai9msnTRQM9877+8a1aOYrDjLVtbsF4DqcKYPLNl8E1KiOQcc46PA1e9BtA dbkPGVUkyIx1fTwcKGc6k8e7Ti1EiV2/5BrYqc0EGi8r5s03H7shHr1ASrYaK0F4Q/bm 0ybQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3a5y1/WHdcXkbOPADuocB1pmZN0ZLdkklQNTVAL0OLE=; b=ZzQmU52LG+F9OGSz9Oj5JW/mrxu17hECupL/igQNzJF1HNR8hj26H5MKkCNwhFwMPf /eA8lwlAsYhV2SUhvl2gvVKDyP2/rBBKOc3mUvzfTSS+H57puHLOSTn6QRHgM4HOYe15 6ld4OYGg2Os8KLO7SCNj2hrCkDbX7bCBb9WfAvRWF6BSj3kU2kjUDX7BWIP9anV94SVq 6tZUnHoxZo6fFJC+ZLm98pHxsQe0vDXkGzWhpWf3O+8pO7EwoefE9xq3hlAyhBRxh+o8 lgYWKSHBUUqOLiTnQr1OzN0KXaSIPFdjJKj2rWguLVHgnMDl13fu2AXOb9rm9A0qIudL klgQ== X-Gm-Message-State: AOAM5300bBD06NPWNoGuABTdXIQfX3TqLQwmirOP8Kl8xK6qHV9YYxMt FndKAE3RACyz2M8ucCqksJt4E70LG0AMKg== X-Received: by 2002:a7b:c3ce:: with SMTP id t14mr2685420wmj.175.1612359954087; Wed, 03 Feb 2021 05:45:54 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:53 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 06/10] futex: Provide and use pi_state_update_owner() Date: Wed, 3 Feb 2021 13:45:35 +0000 Message-Id: <20210203134539.2583943-7-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit c5cade200ab9a2a3be9e7f32a752c8d86b502ec7 ] Updating pi_state::owner is done at several places with the same code. Provide a function for it and use that at the obvious places. This is also a preparation for a bug fix to avoid yet another copy of the same code or alternatively introducing a completely unpenetratable mess of gotos. Originally-by: Peter Zijlstra Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 64 ++++++++++++++++++++++++++------------------------ 1 file changed, 33 insertions(+), 31 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index e44203956d54c..391a85dcd46c2 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -837,6 +837,29 @@ static struct futex_pi_state * alloc_pi_state(void) return pi_state; } +static void pi_state_update_owner(struct futex_pi_state *pi_state, + struct task_struct *new_owner) +{ + struct task_struct *old_owner = pi_state->owner; + + lockdep_assert_held(&pi_state->pi_mutex.wait_lock); + + if (old_owner) { + raw_spin_lock(&old_owner->pi_lock); + WARN_ON(list_empty(&pi_state->list)); + list_del_init(&pi_state->list); + raw_spin_unlock(&old_owner->pi_lock); + } + + if (new_owner) { + raw_spin_lock(&new_owner->pi_lock); + WARN_ON(!list_empty(&pi_state->list)); + list_add(&pi_state->list, &new_owner->pi_state_list); + pi_state->owner = new_owner; + raw_spin_unlock(&new_owner->pi_lock); + } +} + /* * Drops a reference to the pi_state object and frees or caches it * when the last reference is gone. @@ -1432,26 +1455,16 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, else ret = -EINVAL; } - if (ret) { - raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); - return ret; - } - - raw_spin_lock(&pi_state->owner->pi_lock); - WARN_ON(list_empty(&pi_state->list)); - list_del_init(&pi_state->list); - raw_spin_unlock(&pi_state->owner->pi_lock); - raw_spin_lock(&new_owner->pi_lock); - WARN_ON(!list_empty(&pi_state->list)); - list_add(&pi_state->list, &new_owner->pi_state_list); - pi_state->owner = new_owner; - raw_spin_unlock(&new_owner->pi_lock); - - /* - * We've updated the uservalue, this unlock cannot fail. - */ - deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + if (!ret) { + /* + * This is a point of no return; once we modified the uval + * there is no going back and subsequent operations must + * not fail. + */ + pi_state_update_owner(pi_state, new_owner); + deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + } raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); spin_unlock(&hb->lock); @@ -2353,19 +2366,8 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, * We fixed up user space. Now we need to fix the pi_state * itself. */ - if (pi_state->owner != NULL) { - raw_spin_lock_irq(&pi_state->owner->pi_lock); - WARN_ON(list_empty(&pi_state->list)); - list_del_init(&pi_state->list); - raw_spin_unlock_irq(&pi_state->owner->pi_lock); - } - - pi_state->owner = newowner; + pi_state_update_owner(pi_state, newowner); - raw_spin_lock_irq(&newowner->pi_lock); - WARN_ON(!list_empty(&pi_state->list)); - list_add(&pi_state->list, &newowner->pi_state_list); - raw_spin_unlock_irq(&newowner->pi_lock); return 0; /* From patchwork Wed Feb 3 13:45:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375410 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417338ejt; Wed, 3 Feb 2021 05:47:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJyk+APxBnr6N6Tk2IVaWj1Ky1QwVnshpqy88PtRBWTZWU58A0N+OOOaAofGpVLbgPLK5faw X-Received: by 2002:a17:907:2130:: with SMTP id qo16mr591597ejb.537.1612360070394; Wed, 03 Feb 2021 05:47:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360070; cv=none; d=google.com; s=arc-20160816; b=LNaqOrWsFgD/ctKseo8PzUo9dOK9xOLt47HUdxc21nZKe8RNd2ZUspl4xrTgwV4QM9 rqZ1CtulODRL+CXJ20uRcmUU7Q29DX7+RF64hrP1nJnHeUJBMfemrNLcU8RX/SQZmGpV YtHHyF4sTgGlWQBhgWqszQ7MaBQqWFV4EIVyQ9219jjfm/bTPcWbXG6Hq9pnjND5xkxk 1DNFA7b2Cm2vKY3pc0UFD3TGmTqALzRVLa1uQ4/13lUEu9TZvs+lw7XtdCiNpN7TwQx6 WO4adxb1CqGpWsy8AOcfn+wxvCuxb4PKWnZUuoqZTVUAhbVbLZBgB1FyKXL1sID+KkCC wodw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=glWvH/zooJaB8ZiuMmx31xOhEvfCDlYHDJ12kc3zNv8=; b=0L/UwuoXZaPlmw1R46RM38KLpNfg/pQ61tKXCr1SceUvUa36XEcs05pQ0Wa6UC107K Ra6YLkAzP+Dgz3kSkMQqT5i/A25Nsq3Zf7OtZ4pNztaVyHFE4139oNUcooKTn6Q14Ng+ 3w0AuJgfK6FYE05WzQWrvETUsboFF3ml9sp+KK0OzeedGy/ZwQEOOwYyjrKn6qSXGazJ zYzGZHFhZeB6GVy6NhDD3HBahRdpGHDbIozdlRTHJZH0FFBjFjaXOGipP2RYmg0lgzUg sKqYKugswSXVmpIT8Ca+GhXqOCgQ8ao6iw8LD+ZgOqPwh1j28/FLd+f2aXE8ByrwUjif Y1rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Byh7hCJ4; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.48; Wed, 03 Feb 2021 05:47:50 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Byh7hCJ4; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232195AbhBCNrf (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbhBCNrT (ORCPT ); Wed, 3 Feb 2021 08:47:19 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96B80C061793 for ; Wed, 3 Feb 2021 05:45:56 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id v15so24397205wrx.4 for ; Wed, 03 Feb 2021 05:45:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=glWvH/zooJaB8ZiuMmx31xOhEvfCDlYHDJ12kc3zNv8=; b=Byh7hCJ492o3cP67xLLIhIfoTJ2xYP/Dq7GPABZ/uil50mqHQgIHLztQZ8r3RqNSRL MHoU6/2hEU15UfexMQYPa46DVeftzD69Xt3kjIXMRGtO/32VQQOWJxTvXpIqrECOpxFL qNVvsmHrgjFhBBrGnOjDYvsdS3zX3HNQh+97jNAw9HRJ9kdhzKmGEpB/AqcCMZDecWEB 48qaQCUmRyIEsNhihg+BrG1Lt8fnHEbKl72SEiZzss/gy6Hm0aMklmcJU1+6GxTJKWrz 0BrIEMxh0ls5YEKBD1hvmykaVnlWJpE7matPG50KRzy5+DxLXdskSBjUQEW6DmwWsezP BzQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=glWvH/zooJaB8ZiuMmx31xOhEvfCDlYHDJ12kc3zNv8=; b=Rw+XVOu5i8Ctpc7r+AAYb8gvfD3ylKT4dnxEhheaj1wCrwG+QHJ/WuWUrhcSXXhylN BGWhIQPi1Dzk27PZCP/dDq9nqBEVA21rXiXWRJESI7+eMO6Mow1o8sFzGr8B5fHV6e8g 6AUfYoPh6CNr+I0eIDSG776zuuQn7Oev+I3gz3eiCzN51Hw1Ug9VvRuI22kdnh8LVX7Q +oJRTAE3VTYmhUHboaOx3zdnivOHS7fyOk8CD+/Wpgd2UVB3zZ4C56dZJHgfOVKlQpzj 8KMZKXSMSpr3XtuJx/pNhFHD9lMI9WAAyPWicMja5cL5X2rqDmVmcvQudquMXyncVbBk XI+A== X-Gm-Message-State: AOAM532xpXbiErYgCUcNwzUBoHrfQsotFXB/m8/6LwhvGUewW3zTjj5Q xFxM/QrfEImLcJKFMfrzDeugby6Zh1uHNw== X-Received: by 2002:adf:ff91:: with SMTP id j17mr3502622wrr.377.1612359955005; Wed, 03 Feb 2021 05:45:55 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:54 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 07/10] rtmutex: Remove unused argument from rt_mutex_proxy_unlock() Date: Wed, 3 Feb 2021 13:45:36 +0000 Message-Id: <20210203134539.2583943-8-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 2156ac1934166d6deb6cd0f6ffc4c1076ec63697 ] Nothing uses the argument. Remove it as preparation to use pi_state_update_owner(). Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 2 +- kernel/locking/rtmutex.c | 3 +-- kernel/locking/rtmutex_common.h | 3 +-- 3 files changed, 3 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 391a85dcd46c2..40b9ba24bd9a0 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -883,7 +883,7 @@ static void put_pi_state(struct futex_pi_state *pi_state) list_del_init(&pi_state->list); raw_spin_unlock_irq(&pi_state->owner->pi_lock); - rt_mutex_proxy_unlock(&pi_state->pi_mutex, pi_state->owner); + rt_mutex_proxy_unlock(&pi_state->pi_mutex); } if (current->pi_state_cache) diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index d295821ed4cc8..6ff4156b3929e 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1696,8 +1696,7 @@ void rt_mutex_init_proxy_locked(struct rt_mutex *lock, * No locking. Caller has to do serializing itself * Special API call for PI-futex support */ -void rt_mutex_proxy_unlock(struct rt_mutex *lock, - struct task_struct *proxy_owner) +void rt_mutex_proxy_unlock(struct rt_mutex *lock) { debug_rt_mutex_proxy_unlock(lock); rt_mutex_set_owner(lock, NULL); diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 09991287491d1..bea5d677fe343 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -102,8 +102,7 @@ enum rtmutex_chainwalk { extern struct task_struct *rt_mutex_next_owner(struct rt_mutex *lock); extern void rt_mutex_init_proxy_locked(struct rt_mutex *lock, struct task_struct *proxy_owner); -extern void rt_mutex_proxy_unlock(struct rt_mutex *lock, - struct task_struct *proxy_owner); +extern void rt_mutex_proxy_unlock(struct rt_mutex *lock); extern int rt_mutex_start_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter, struct task_struct *task); From patchwork Wed Feb 3 13:45:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375407 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417288ejt; Wed, 3 Feb 2021 05:47:46 -0800 (PST) X-Google-Smtp-Source: ABdhPJxa1HJG64dM2B0dJEeOgeh6VSy0iJ8MqpIPxkKHZXI0xzoEmrv/FnzbteaXwJDwj2twody0 X-Received: by 2002:a17:907:767c:: with SMTP id kk28mr3199048ejc.98.1612360066248; Wed, 03 Feb 2021 05:47:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360066; cv=none; d=google.com; s=arc-20160816; b=SwwPoOsbjs9i2VDQsHmtRnmq90FFQe7NORdbgl406zGNAwSm7Crjg0AcQCkUIoAGSt HqSVMCJEV2+e0anJTANU0gNO4gSqajijlYoHosvFNRBn/1Z3O5nsBYhgsratMS2Qx0KK WqT3tvAWG7bKz15MhKXn4zDH+YwlRH/xLY+Uqi+uz/DitBpOLY75D4KflzI2ozAPmVIQ oBayQPzEIe4k0op7Me/Nsc83IGXEzEO6gv7uDJtpR3sKPiuUyKHNdqIpDFKkA7IefHL9 nJMklhae9cs3UubAz4dsJzU9MPkHMaycX22vlLVVlbzTq38o+Lc/diXCIViZDgbqe+Rv 1LEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2Wjtls+pH/GZJVycHEPxwrPZB5Wdwg8gcit7enARmf4=; b=NPjbQtxpdzCzRUrIqH10rL6tOw3zVPdDDgOEr3R33p2lJon+7VUlpgQ40ul10DYEpg 1V8Bjcu8a4A8K01yPM6cWWSW4DYm1GOkTboWcY7linnIwAW+miDWd+D60f5810w35Uar kSK/XQW8DXTq5nVmMRGf6Eog4FOGpcJeh2EBIk58/TbX+4EHry2V/G10dHMk1HAxxXID JhRibhydQ32nigq3+mXFNVFmLnekuFwbuAQ40BGOh5sUMGUCEbSuLbX3DNpyABVrcJHr FvjhGG9ZGmIHqsdC8w7oJnV2rQZijxDLSzXltd/cJmXSWtYAhxL3FxodjqBsVg+lNIur 2DIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=f47c4ugf; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.41; Wed, 03 Feb 2021 05:47:46 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=f47c4ugf; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232161AbhBCNr0 (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232193AbhBCNrV (ORCPT ); Wed, 3 Feb 2021 08:47:21 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6A47C061794 for ; Wed, 3 Feb 2021 05:45:57 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id d16so24322919wro.11 for ; Wed, 03 Feb 2021 05:45:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2Wjtls+pH/GZJVycHEPxwrPZB5Wdwg8gcit7enARmf4=; b=f47c4ugfyeITPfmr8BwY4/nuEC3Hx/6feP2Z0MwgBmo5U5B3pGdIFg41B5aR85snq2 ocRFntQd53/bRbPfauI1keRtHKAgiYbyY5JEXuvxQ8ZlUKZMTMf2X4TW+kwAjizqQRCU DnyMk+xjCSyJwf58hiipHsQkzc0OtqRvD1Qo/3NyTtKFyvKijmhV+UDzKGe7cJZkJEpl LdwcpSCF0ui/Dqmuq9e1/ukfbH1zEErI4bSokDnS1WamBaUaqoWPKkFMTFXYdIzBaEvZ B7rBYFtWrbHqQ9s5ZleAcmBTfyGFZMpgWRXlwApV56RVS39Mjv+GwBBcBRroniRAgnrd 0jzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2Wjtls+pH/GZJVycHEPxwrPZB5Wdwg8gcit7enARmf4=; b=ldmlH2FiX6pjx+vX52jlpXEHzlLSF6Yr3Qt9qRl3cJrdpMuvZyaso4zvZ50dpqEtxq oloUqtZPaMFtKdSYrUI6o+LD6nKbbvUui4+nCRGAnrqKGTZsZxA+oxIIFcSUggum3EqQ O3yFRI3f2J6dKrQChVmF1MypfTu23V2F4t2qDE9RZHWW2/S4C83DcOuTem8FdlhmCZPV VKyI/rMQV099LBkmshMF4k4st3V3LWhMA7iQLwlHAXC/KUC/XEmmxreMeyDXeENgL7BZ ORn3gF0iyW8l8rqe65WD7oXMhQ5o/jen8sutOSwRgL+ei7Nqs60X0g8GIbqgfS8ZieEs oQAw== X-Gm-Message-State: AOAM530whqWo/k+EF+RMPT5t79kMHv9rG9QBMUGsr8s84rTdx8P120iK 7457slS8JgY11/qMvp+8BA5rG3i3RhwP2A== X-Received: by 2002:a5d:6a01:: with SMTP id m1mr3544022wru.318.1612359956182; Wed, 03 Feb 2021 05:45:56 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:55 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 08/10] futex: Use pi_state_update_owner() in put_pi_state() Date: Wed, 3 Feb 2021 13:45:37 +0000 Message-Id: <20210203134539.2583943-9-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 6ccc84f917d33312eb2846bd7b567639f585ad6d ] No point in open coding it. This way it gains the extra sanity checks. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 40b9ba24bd9a0..be5e3e927bffa 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -879,10 +879,7 @@ static void put_pi_state(struct futex_pi_state *pi_state) * and has cleaned up the pi_state already */ if (pi_state->owner) { - raw_spin_lock_irq(&pi_state->owner->pi_lock); - list_del_init(&pi_state->list); - raw_spin_unlock_irq(&pi_state->owner->pi_lock); - + pi_state_update_owner(pi_state, NULL); rt_mutex_proxy_unlock(&pi_state->pi_mutex); } From patchwork Wed Feb 3 13:45:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375408 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp417306ejt; Wed, 3 Feb 2021 05:47:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJxYe28hjJpKJ3HpVgAvKbMLxMLCAhogvicoBfnoukTQfXcozt7/aaxaB0878RjCHE8AaIph X-Received: by 2002:a17:906:15c7:: with SMTP id l7mr3338706ejd.226.1612360067042; Wed, 03 Feb 2021 05:47:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360067; cv=none; d=google.com; s=arc-20160816; b=FLxRjAgAYFW33FjOxmWAaCan2Juy8/fh/kLRXnZnhvytqVvNtGhpO7M9GpN0aWjMmg RabVIJxfhwEqhstyR5bwxMoO+EIQPfOVd3LyUHE+lweW5kwA++XgrY3Viw2R1lFLcx9H G/YDjAfVCG6j+3v5uMhPlhgIow9FNODueQBkcHcx5HMXRmCRp/fAhdUJ3nIbJx3c7VVS vCqg85MvTFda4gXHwZfyr+VkIUIBDT1pmtPVf0Jl1u79nQkOhYZBQxtXWiBCEr4E4VTm Nqp9XXmm22s0dORoFpESfYaBcUAlq6MV1o9FYzDOAjCZADAwvh7D8t0i4R9fNuAbWvK+ Cl0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SH5Nj25x7+v0gQvzlOdJud0ErPOJ5XP3gEjc5bfgYKY=; b=j0IzwJF7eE+LdHI1t46ibOAo8gKtNpofdBZRfPuRowjEFT/mISwvoRAoAF1ZUoKJRU TjWQ2iS2FI9V3951I1sZTDz4TE+3tSFS7u8IzGkjTz6otRi1J90IeGqKDuBZb3MyOjJv S51etotdvzf5XKii3S8HXFGFMlvsqg3TsVZMLb31k5eZ/qlty/g5I7XNeGsHxjiC4kh4 kGnexM7c08F0aDiwxTs/ibDhMLbI+7iEWiR6UPuDNu1ZwTzwzHVSRivYfpn5hg/7yLii n/s6qFbGNBQ2VmxzMgdiilfPKOqw0aUvWuJO8Qo0DMWDEL/9SQ3T1HJNH6bLyu2nkmWn srsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XfT445G4; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jz24si1404088ejc.430.2021.02.03.05.47.46; Wed, 03 Feb 2021 05:47:47 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XfT445G4; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232222AbhBCNrc (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232195AbhBCNrV (ORCPT ); Wed, 3 Feb 2021 08:47:21 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD490C061797 for ; Wed, 3 Feb 2021 05:45:58 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id l12so4506554wmq.2 for ; Wed, 03 Feb 2021 05:45:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SH5Nj25x7+v0gQvzlOdJud0ErPOJ5XP3gEjc5bfgYKY=; b=XfT445G4wZ4PKUnsgeT4VrLtHzCml1M0Oj6q+Hn/HN1tEDGfkWHIIPD0WvgsvcVOfp +pBW47XxIPkCRm/di75xUdPTOj/yGLjKCrcqLhYTBSaxqgTkpSOkNkctl/zE2gWhsHIH 80j9GQXlDSkOLtPt+Hl6uFrtnCDgsvjFBMDWLgncFPnBv7ciIu7fBdO1CXeAHDKoQ3/g zT89cFJjdtZikZ8g8MiOsJF/mqSq/JwZBeWw5EXoz3OiTylhVIi/HS6xV/hg9H1IAIlC xqVXwoh3ngTPuFMqIl8Uhs/TCtuiiTazF6GhjqW7mA8XIQFoiLinLwmEmfn+1Hz9OLeb Di3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SH5Nj25x7+v0gQvzlOdJud0ErPOJ5XP3gEjc5bfgYKY=; b=LPDugSbxsEASrNrmav5GuHy9JcOH++AdkGqsV26JhJLYfZYsdnHgJ9mLtvGk+c6e3O gDNfaSK8/VqBMQsfvDV6cM95gJCKgUe4yEIapqyuZud0Fnt9nPWAWBLPcPTgMTdBBi2a Is9jN4bL0ZcE6LrXru6q7Vfxno3mB7cDOK61UWoYzSfUIGvFpe5ZNMpqlMzjVxm9e83A oHKcYIreeF94ruplTyQ6/SdZNOyPK2+niKdZCvSiQL90D2f2ZgDmdtayQRZSrcCnFhq3 l+O7ymqv5crMNozUk4ZDWvDP5rQmycyC24YG3BUIU48GzwEaqndJL4JhUPVm9kuUyzvi LYNw== X-Gm-Message-State: AOAM530bikAtwDhL9HOSeS3UuEs9slA5IBxMw+GGmFk4+9/but6RILRf WjO32g1ZiQJBvEMM5lR+WrERsj5A81JKHA== X-Received: by 2002:a1c:21c6:: with SMTP id h189mr2774643wmh.173.1612359957109; Wed, 03 Feb 2021 05:45:57 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:56 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 09/10] futex: Simplify fixup_pi_state_owner() Date: Wed, 3 Feb 2021 13:45:38 +0000 Message-Id: <20210203134539.2583943-10-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit f2dac39d93987f7de1e20b3988c8685523247ae2 ] Too many gotos already and an upcoming fix would make it even more unreadable. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index be5e3e927bffa..c163f5d6efab3 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2272,18 +2272,16 @@ static void unqueue_me_pi(struct futex_q *q) spin_unlock(q->lock_ptr); } -static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, - struct task_struct *argowner) +static int __fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, + struct task_struct *argowner) { struct futex_pi_state *pi_state = q->pi_state; - u32 uval, uninitialized_var(curval), newval; struct task_struct *oldowner, *newowner; - u32 newtid; - int ret; - - lockdep_assert_held(q->lock_ptr); + u32 uval, curval, newval, newtid; + int err = 0; oldowner = pi_state->owner; + /* Owner died? */ if (!pi_state->owner) newtid |= FUTEX_OWNER_DIED; @@ -2324,7 +2322,7 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, if (__rt_mutex_futex_trylock(&pi_state->pi_mutex)) { /* We got the lock after all, nothing to fix. */ - return 0; + return 1; } /* @@ -2339,7 +2337,7 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, * We raced against a concurrent self; things are * already fixed up. Nothing to do. */ - return 0; + return 1; } newowner = argowner; } @@ -2380,7 +2378,7 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, handle_fault: spin_unlock(q->lock_ptr); - ret = fault_in_user_writeable(uaddr); + err = fault_in_user_writeable(uaddr); spin_lock(q->lock_ptr); @@ -2388,12 +2386,27 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, * Check if someone else fixed it for us: */ if (pi_state->owner != oldowner) - return 0; + return argowner == current; - if (ret) - return ret; + /* Retry if err was -EAGAIN or the fault in succeeded */ + if (!err) + goto retry; - goto retry; + return err; +} + +static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, + struct task_struct *argowner) +{ + struct futex_pi_state *pi_state = q->pi_state; + int ret; + + lockdep_assert_held(q->lock_ptr); + + raw_spin_lock_irq(&pi_state->pi_mutex.wait_lock); + ret = __fixup_pi_state_owner(uaddr, q, argowner); + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); + return ret; } static long futex_wait_restart(struct restart_block *restart); From patchwork Wed Feb 3 13:45:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 375411 Delivered-To: patch@linaro.org Received: by 2002:a17:906:48d2:0:0:0:0 with SMTP id d18csp418298ejt; Wed, 3 Feb 2021 05:49:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJzRls5jrEJt5PveHuwpJnyi7Z9PAGA0USea1ryr7HvA1ZSK5bUQIUePqcPaOOw3lbRY+OZ+ X-Received: by 2002:aa7:dd49:: with SMTP id o9mr3005008edw.14.1612360159158; Wed, 03 Feb 2021 05:49:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612360159; cv=none; d=google.com; s=arc-20160816; b=PpRcVVG6OsGtH1NGm3CgG58E16VWczh0lW61OsaC/VtVBbRtG1wEekcH/FQix81m83 8CgD0iydQHyK6AinIzAYeFn1EIcPlGNt3uEF16KHp4CgWj9bgjORzhMvSy47vXNfa+sr LCVEI6ZaD15En8D9P2P3WYQJJk5LzdNanOnkrfwbrMGk+cWTwlTkiMhdt0hgimau07qm GtsONkMy92CJMWRy62aoc5RgaBaARgGrv2XowFNV+RSFctD0LAkbTwfv+KaXIMk30feL R1YxVJCpTb3N+KuqDGghdHbSmFwF4RN2ITnN3+5lPdBrxBWu3tiu0BNSh4zOh3INrCPL ujLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=IefE4HShlaoESIdM7PNGNW8PHMR5MUG9Lc5XYbz4i4Y=; b=nCY4eU2BnUMpdwi1yXxHNLl2MqtWmgoGmYNzZF6+WnRbDXVYfXehiOH8oXplssuJ07 wRL06I20SZLYLwSpon31BaWwZTH5S2RpcDlFNSMHY+cAaRjP48GGaKofN8/tCSVCAL9L BjmcJqJN+yZ+ndrRQX/m7zL0siR/vxU4GLMm48884+TQQiG/xrPiCGu/PUFO8Rgvbg7H WsEJhAVN+lcARhE/5rTrlHJzJSlPGLCvLXK9RZZChALeox6CL2lvIGCFFAhvAUueC6yl a0ItiuEtEM+Et7kMQt4F/Q1Jpt4/M77cwHRkBHzwqq0BHarjipzR4EAfQiVhrK1yhxg2 mnXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WrgYDPQP; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id g19si1469620ejz.742.2021.02.03.05.49.18; Wed, 03 Feb 2021 05:49:19 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WrgYDPQP; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232207AbhBCNrr (ORCPT + 13 others); Wed, 3 Feb 2021 08:47:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232170AbhBCNrW (ORCPT ); Wed, 3 Feb 2021 08:47:22 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 023F0C0617A7 for ; Wed, 3 Feb 2021 05:46:00 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id q7so24334537wre.13 for ; Wed, 03 Feb 2021 05:45:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IefE4HShlaoESIdM7PNGNW8PHMR5MUG9Lc5XYbz4i4Y=; b=WrgYDPQPnfNK7+xWMP9tR8smGsyoEmixcR5cw0OSFJjhJKKGmmDkwZ2vTxVPfwT+lW wkI7gJ1NAoUiAhxreUbMeBfON7XMjR/Gxm8xCjt+fmhupIj9piTXn6tj8/L8Ow60h4P1 vAV6Bi7yNnigvzIDi9CFbtygT5Kaga0nUq91aZ96v4TkZqUuhIFgXavto0/Q/q4NkFL5 wKMU+BIzBIrzGyG6JBMiIiVGEF3CTNdMZe26XBMSPS/Ix0oD/xxvguqC0SYPhHqpEgA4 emLiYtaAA9bv/m7K8e7E4/IfTsjrElVKWdsRPX/aDlrprbbNmarsKeuQVp5axahba/Nt jLTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IefE4HShlaoESIdM7PNGNW8PHMR5MUG9Lc5XYbz4i4Y=; b=hPVC7lqx/6v7CjGmlelYqpk1AHRg/PtSli+O8oe6pmsh0LMRJL4UXBV15dkdT32ozR At68TeDBXgK2p9PSXPNmp3+GM54iIOaQdHGHvGvs82yV2zFpV30hdyVIrCj4L7Khg8DL T2cwED22aNWYRtVmoOoHRPVGHYcZd+paCgI4AovQN+vWElEcBu4fY+hDW1vrdOSpy6ni z/0Va5YvxXs9yBWiGKplwrQoXXBN7p8b3ISK8/wkA9vs7pNOmIQjCjNxI0zca24L463n 52Eoo0tzOZQRYPf/cN/H6TatRJg9pdjpxCsMBM+wo1IMcaT8nwMepbkDj2njagJBQVRc nCWw== X-Gm-Message-State: AOAM533MVv9N1FzXtQDB3/nPF0YyylXmRg39ftAQs/XCkNbZMJVaTdSX 56MCOXPkkzZ072+2Phr5eVeMHm674VgxaQ== X-Received: by 2002:a05:6000:1788:: with SMTP id e8mr3657643wrg.171.1612359958406; Wed, 03 Feb 2021 05:45:58 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id r124sm2867900wmr.16.2021.02.03.05.45.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Feb 2021 05:45:57 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , gzobqq@gmail.com, Peter Zijlstra , Lee Jones Subject: [PATCH 10/10] futex: Handle faults correctly for PI futexes Date: Wed, 3 Feb 2021 13:45:39 +0000 Message-Id: <20210203134539.2583943-11-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210203134539.2583943-1-lee.jones@linaro.org> References: <20210203134539.2583943-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner fixup_pi_state_owner() tries to ensure that the state of the rtmutex, pi_state and the user space value related to the PI futex are consistent before returning to user space. In case that the user space value update faults and the fault cannot be resolved by faulting the page in via fault_in_user_writeable() the function returns with -EFAULT and leaves the rtmutex and pi_state owner state inconsistent. A subsequent futex_unlock_pi() operates on the inconsistent pi_state and releases the rtmutex despite not owning it which can corrupt the RB tree of the rtmutex and cause a subsequent kernel stack use after free. It was suggested to loop forever in fixup_pi_state_owner() if the fault cannot be resolved, but that results in runaway tasks which is especially undesired when the problem happens due to a programming error and not due to malice. As the user space value cannot be fixed up, the proper solution is to make the rtmutex and the pi_state consistent so both have the same owner. This leaves the user space value out of sync. Any subsequent operation on the futex will fail because the 10th rule of PI futexes (pi_state owner and user space value are consistent) has been violated. As a consequence this removes the inept attempts of 'fixing' the situation in case that the current task owns the rtmutex when returning with an unresolvable fault by unlocking the rtmutex which left pi_state::owner and rtmutex::owner out of sync in a different and only slightly less dangerous way. Fixes: 1b7558e457ed ("futexes: fix fault handling in futex_lock_pi") Reported-by: gzobqq@gmail.com Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index c163f5d6efab3..83db5787c67ef 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1017,7 +1017,8 @@ static void exit_pi_state_list(struct task_struct *curr) * FUTEX_OWNER_DIED bit. See [4] * * [10] There is no transient state which leaves owner and user space - * TID out of sync. + * TID out of sync. Except one error case where the kernel is denied + * write access to the user address, see fixup_pi_state_owner(). */ /* @@ -2392,6 +2393,24 @@ static int __fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, if (!err) goto retry; + /* + * fault_in_user_writeable() failed so user state is immutable. At + * best we can make the kernel state consistent but user state will + * be most likely hosed and any subsequent unlock operation will be + * rejected due to PI futex rule [10]. + * + * Ensure that the rtmutex owner is also the pi_state owner despite + * the user space value claiming something different. There is no + * point in unlocking the rtmutex if current is the owner as it + * would need to wait until the next waiter has taken the rtmutex + * to guarantee consistent state. Keep it simple. Userspace asked + * for this wreckaged state. + * + * The rtmutex has an owner - either current or some other + * task. See the EAGAIN loop above. + */ + pi_state_update_owner(pi_state, rt_mutex_owner(&pi_state->pi_mutex)); + return err; } @@ -2777,13 +2796,6 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, if (res) ret = (res < 0) ? res : 0; - /* - * If fixup_owner() faulted and was unable to handle the fault, unlock - * it and return the fault to userspace. - */ - if (ret && (rt_mutex_owner(&q.pi_state->pi_mutex) == current)) - rt_mutex_futex_unlock(&q.pi_state->pi_mutex); - /* Unqueue and drop the lock */ unqueue_me_pi(&q); @@ -3088,8 +3100,6 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, if (q.pi_state && (q.pi_state->owner != current)) { spin_lock(q.lock_ptr); ret = fixup_pi_state_owner(uaddr2, &q, current); - if (ret && rt_mutex_owner(&q.pi_state->pi_mutex) == current) - rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* * Drop the reference to the pi state which * the requeue_pi() code acquired for us. @@ -3126,14 +3136,6 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, if (res) ret = (res < 0) ? res : 0; - /* - * If fixup_pi_state_owner() faulted and was unable to handle - * the fault, unlock the rt_mutex and return the fault to - * userspace. - */ - if (ret && rt_mutex_owner(pi_mutex) == current) - rt_mutex_futex_unlock(pi_mutex); - /* Unqueue and drop the lock. */ unqueue_me_pi(&q); }