From patchwork Thu Feb 4 17:28:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376307 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1481953jah; Thu, 4 Feb 2021 09:32:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJyJaichy09Mx1AzjtGzz5kXdlZJiHmumrOpVGguGHHqykNOpvosxfXkCbkCv3h7GaUBtpai X-Received: by 2002:aa7:c6d9:: with SMTP id b25mr86995eds.84.1612459957532; Thu, 04 Feb 2021 09:32:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612459957; cv=none; d=google.com; s=arc-20160816; b=FOVknpnGpTp0RShdgzHSs/WQAmrvEjdtLguxzR8eDXRr1Lzg6vNnRtVOXqjn+JtQAM jHTkayug7VfFj44gq1c7sGRyTA/wrB1s1lKS2Ris9sr8ZVjvl8YhqWm3Ti4NsD+sZr3i OO/Lx5gst9VjlXDsu4Yr0q3xf6IpgyhvW5AMlT2CsBBCUuh7MnEJeeRnI1Fc3ilgSqWZ Lvf18RrpxRCBtzPi/hETIplxN0gTw47GHuqIbY8wAnDrw5oMg/gDc3+2XSe7txbmkECY JBbufgYBCcHU2Df9yTvG4n3HWaJRiQeI/Bs8/kZVL7O1BoLuTSGB69P3pF6pLoP558I7 h/4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9G32MvUQnMihp2dXE+SFBFZc9Ph7JPBNbbGYxik49LE=; b=rVKNxmCvYRniYP9HsEq5MHPsp+PXKq/ldg8pYR2MQBhLA91XKFa/MMi3KT/gQFd5Z+ llQxLqr5bzZXwITVSOllBM7HW9rZks/qPvounix2wt8AQsIEhi8FbZHOSJalLw5Eim2G cbb2Xsb8zBr9teHJseUu00sGZ9a8lF3Ycp+qLCljcKTMHHclRBnGC1ZiwiqCNy+ySR0D jJAHdf4AmB2rvJ/jqEgHsCB8J2S/HhgjTKNhKDO9C2E6CpeIkLZkFG+xUh6JuIITmH3q t0ivelYE9sV3cDv3b53zTWZgiB3sUXIsgg2Zzc6rrZoR39RBY2jt7+ZkEEjyPTKP7GWk InaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lsk7RF6w; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l23si3310431ejx.701.2021.02.04.09.32.36; Thu, 04 Feb 2021 09:32:37 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lsk7RF6w; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238362AbhBDRbg (ORCPT + 13 others); Thu, 4 Feb 2021 12:31:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238518AbhBDR3u (ORCPT ); Thu, 4 Feb 2021 12:29:50 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D909FC06178A for ; Thu, 4 Feb 2021 09:29:09 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id m13so4447198wro.12 for ; Thu, 04 Feb 2021 09:29:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9G32MvUQnMihp2dXE+SFBFZc9Ph7JPBNbbGYxik49LE=; b=lsk7RF6wOITHSRBg7Mzg4E+e4uDvAVoApQDk+AC872daBAx5hySzKh7ogKt7UUSDR3 K9JHNlQ8YvMw1YIpXDNskGwmszMbExLzbGZkau3Se3zxgK3jhH9LrAhXRqbmW5HY/vvI 7ZK7KSD1KHQgebbtQ8laY+qIsDiYsp/EAl3QSzdLGBuAiRSrNnj07zPasCH5mQeX15IR iBIpWuC9WyBTuJHF4BLoIgxX0iddGf+3tIKjpughl4qEvimmffhbkZokH6sSnfaDtqhl eMAkX0oRi+G4goUGXhzB7g3ZFma2qa8ixf92ngv4W1dwepHObbVREFaDbajcUFE5mukN l1nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9G32MvUQnMihp2dXE+SFBFZc9Ph7JPBNbbGYxik49LE=; b=DXx/E2SOgcZVhz3Gsm5FPeyNYlL+4OMuX1FCwg/AJ+BDYtyX+yvHlyTzbHHtpb4eXA D8yHf5oXHoxTnH6ARGPUJPNwDuD6/quaWJku+XDgdwYQSCs5EiyNMXl+BYUA+ct63uHl CQ6F2jjeovk/9dTMDqDKH7Wy9i1KfycSkMeMNwX7VppdsC6z3oOeW72KXKqj9hrZD/X/ 4iIvnSuAG5EEL4g40xXs17iRr9jt3s62a/n05lONcjJNaK+aCbO8HGJT0XZA/C6EqKAw eN8W88Lua5p+Nb5Us2H3BmBsPVWms6thw/84XzvJg4r8Rmu7iAf4B0BBvHWWuS1F5GOf BqCQ== X-Gm-Message-State: AOAM530q5VahRgHOJKM6BBHhVvIkxiXkYdlSF36wSj4VqC5idmysBDFq aLMrgnIVx0FCqzWm3RpIyaZmOyR+r6mSAg== X-Received: by 2002:a05:6000:1189:: with SMTP id g9mr475090wrx.230.1612459748318; Thu, 04 Feb 2021 09:29:08 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:07 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 01/10] futex,rt_mutex: Provide futex specific rt_mutex API Date: Thu, 4 Feb 2021 17:28:54 +0000 Message-Id: <20210204172903.2860981-2-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra [ Upstream commit 5293c2efda37775346885c7e924d4ef7018ea60b ] Part of what makes futex_unlock_pi() intricate is that rt_mutex_futex_unlock() -> rt_mutex_slowunlock() can drop rt_mutex::wait_lock. This means it cannot rely on the atomicy of wait_lock, which would be preferred in order to not rely on hb->lock so much. The reason rt_mutex_slowunlock() needs to drop wait_lock is because it can race with the rt_mutex fastpath, however futexes have their own fast path. Since futexes already have a bunch of separate rt_mutex accessors, complete that set and implement a rt_mutex variant without fastpath for them. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.702962446@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 30 +++++++++--------- kernel/locking/rtmutex.c | 56 ++++++++++++++++++++++++--------- kernel/locking/rtmutex_common.h | 8 +++-- 3 files changed, 61 insertions(+), 33 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index f1990e2a51e5a..00b474b4b54e0 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -936,7 +936,7 @@ static void exit_pi_state_list(struct task_struct *curr) pi_state->owner = NULL; raw_spin_unlock_irq(&curr->pi_lock); - rt_mutex_unlock(&pi_state->pi_mutex); + rt_mutex_futex_unlock(&pi_state->pi_mutex); spin_unlock(&hb->lock); @@ -1436,20 +1436,18 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, pi_state->owner = new_owner; raw_spin_unlock_irq(&new_owner->pi_lock); - raw_spin_unlock(&pi_state->pi_mutex.wait_lock); - - deboost = rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); - /* - * First unlock HB so the waiter does not spin on it once he got woken - * up. Second wake up the waiter before the priority is adjusted. If we - * deboost first (and lose our higher priority), then the task might get - * scheduled away before the wake up can take place. + * We've updated the uservalue, this unlock cannot fail. */ + deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); spin_unlock(&hb->lock); - wake_up_q(&wake_q); - if (deboost) + + if (deboost) { + wake_up_q(&wake_q); rt_mutex_adjust_prio(current); + } return 0; } @@ -2362,7 +2360,7 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) * task acquired the rt_mutex after we removed ourself from the * rt_mutex waiters list. */ - if (rt_mutex_trylock(&q->pi_state->pi_mutex)) { + if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { locked = 1; goto out; } @@ -2686,7 +2684,7 @@ retry_private: if (!trylock) { ret = rt_mutex_timed_futex_lock(&q.pi_state->pi_mutex, to); } else { - ret = rt_mutex_trylock(&q.pi_state->pi_mutex); + ret = rt_mutex_futex_trylock(&q.pi_state->pi_mutex); /* Fixup the trylock return value: */ ret = ret ? 0 : -EWOULDBLOCK; } @@ -2709,7 +2707,7 @@ retry_private: * it and return the fault to userspace. */ if (ret && (rt_mutex_owner(&q.pi_state->pi_mutex) == current)) - rt_mutex_unlock(&q.pi_state->pi_mutex); + rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* Unqueue and drop the lock */ unqueue_me_pi(&q); @@ -3016,7 +3014,7 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, spin_lock(q.lock_ptr); ret = fixup_pi_state_owner(uaddr2, &q, current); if (ret && rt_mutex_owner(&q.pi_state->pi_mutex) == current) - rt_mutex_unlock(&q.pi_state->pi_mutex); + rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* * Drop the reference to the pi state which * the requeue_pi() code acquired for us. @@ -3059,7 +3057,7 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, * userspace. */ if (ret && rt_mutex_owner(pi_mutex) == current) - rt_mutex_unlock(pi_mutex); + rt_mutex_futex_unlock(pi_mutex); /* Unqueue and drop the lock. */ unqueue_me_pi(&q); diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index dd173df9ee5e5..3323ef935372f 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1485,15 +1485,23 @@ EXPORT_SYMBOL_GPL(rt_mutex_lock_interruptible); /* * Futex variant with full deadlock detection. + * Futex variants must not use the fast-path, see __rt_mutex_futex_unlock(). */ -int rt_mutex_timed_futex_lock(struct rt_mutex *lock, +int __sched rt_mutex_timed_futex_lock(struct rt_mutex *lock, struct hrtimer_sleeper *timeout) { might_sleep(); - return rt_mutex_timed_fastlock(lock, TASK_INTERRUPTIBLE, timeout, - RT_MUTEX_FULL_CHAINWALK, - rt_mutex_slowlock); + return rt_mutex_slowlock(lock, TASK_INTERRUPTIBLE, + timeout, RT_MUTEX_FULL_CHAINWALK); +} + +/* + * Futex variant, must not use fastpath. + */ +int __sched rt_mutex_futex_trylock(struct rt_mutex *lock) +{ + return rt_mutex_slowtrylock(lock); } /** @@ -1552,20 +1560,38 @@ void __sched rt_mutex_unlock(struct rt_mutex *lock) EXPORT_SYMBOL_GPL(rt_mutex_unlock); /** - * rt_mutex_futex_unlock - Futex variant of rt_mutex_unlock - * @lock: the rt_mutex to be unlocked - * - * Returns: true/false indicating whether priority adjustment is - * required or not. + * Futex variant, that since futex variants do not use the fast-path, can be + * simple and will not need to retry. */ -bool __sched rt_mutex_futex_unlock(struct rt_mutex *lock, - struct wake_q_head *wqh) +bool __sched __rt_mutex_futex_unlock(struct rt_mutex *lock, + struct wake_q_head *wake_q) { - if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) { - rt_mutex_deadlock_account_unlock(current); - return false; + lockdep_assert_held(&lock->wait_lock); + + debug_rt_mutex_unlock(lock); + + if (!rt_mutex_has_waiters(lock)) { + lock->owner = NULL; + return false; /* done */ + } + + mark_wakeup_next_waiter(wake_q, lock); + return true; /* deboost and wakeups */ +} + +void __sched rt_mutex_futex_unlock(struct rt_mutex *lock) +{ + WAKE_Q(wake_q); + bool deboost; + + raw_spin_lock_irq(&lock->wait_lock); + deboost = __rt_mutex_futex_unlock(lock, &wake_q); + raw_spin_unlock_irq(&lock->wait_lock); + + if (deboost) { + wake_up_q(&wake_q); + rt_mutex_adjust_prio(current); } - return rt_mutex_slowunlock(lock, wqh); } /** diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 6f8f68edb700c..cdcaccfb74432 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -112,8 +112,12 @@ extern int rt_mutex_wait_proxy_lock(struct rt_mutex *lock, extern bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter); extern int rt_mutex_timed_futex_lock(struct rt_mutex *l, struct hrtimer_sleeper *to); -extern bool rt_mutex_futex_unlock(struct rt_mutex *lock, - struct wake_q_head *wqh); +extern int rt_mutex_futex_trylock(struct rt_mutex *l); + +extern void rt_mutex_futex_unlock(struct rt_mutex *lock); +extern bool __rt_mutex_futex_unlock(struct rt_mutex *lock, + struct wake_q_head *wqh); + extern void rt_mutex_adjust_prio(struct task_struct *task); #ifdef CONFIG_DEBUG_RT_MUTEXES From patchwork Thu Feb 4 17:28:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376306 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1481909jah; Thu, 4 Feb 2021 09:32:34 -0800 (PST) X-Google-Smtp-Source: ABdhPJzD0KFLPHwKK+hsK4COzixK9BmjOXl0E46+VPFjAVJa7OfhpZcsvCgnjIu3ZXVfqi1zS0Al X-Received: by 2002:a50:e80d:: with SMTP id e13mr21165edn.251.1612459954661; Thu, 04 Feb 2021 09:32:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612459954; cv=none; d=google.com; s=arc-20160816; b=WvLaerEMc5KfNoPbeKxIYoeJ0ch/F84mG38e+cI5866K/xD5es/kAYOsh6HSWExZ1Z VZ4r/AsI8yAv4IBMWlzA8Bwb3+TxYpFO1zN4/VrtrwgxfEfjsf0Q7mTOn5GB5NCKw+q+ 7KPUuRS7AIm0RFXcQenERTitaqzY6kmlky5C4+jlyxH32FXYH7BqdsdxyIGn3KXhLWAJ cLT3jGh1TK02CA4e22wM5PGRL6yF2YyZbisTpUw1A+PNvTUZw8zedlh3DJXxnQ390vAo Vj7cnVgkX2o4k/RwvKsTzkZHEqetLc+eyM1L+sqmIsmfvxQ/Gz6wSeE7IALb8JjFaKNy fZIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wf5g1tBsiAuXWWCi8IkrUKPbvvUqMLHqfwWU02yQBFw=; b=v3OMaUYPPagkpuMPSHkK9Y1YnPbIRhRc/viPVapLhlI+yJVgIJf43HOhC/rjZU7ET+ c4pnANz6ZN4ojsN8jGx4t1xGLrWpVQUAGaSnNBRq56OAJu1ZVTE2H3pXXpCYCOHO8Tq+ Iz+jGPdI8XQPxhU0IfIR2IWiHBR1B0Tw8mEAKuK0t88p2EYXOQYwJyeTU/SnZAQcDd4B gqM4YGdEUNPSik3XzRGTNTqlIL5JbcT9QpyjDuOfMuT4gUkz6Iz4pPWavnd1oIE5GqEd i81OQMMY74nG+lEZhXwE54o8wzJsTdtaqW/X+BKdFu24uNmSXKjvne/WFQuPqJamjRX+ U0+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yt9iTqx9; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l23si3310431ejx.701.2021.02.04.09.32.33; Thu, 04 Feb 2021 09:32:34 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yt9iTqx9; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238579AbhBDRbC (ORCPT + 13 others); Thu, 4 Feb 2021 12:31:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238519AbhBDR3v (ORCPT ); Thu, 4 Feb 2021 12:29:51 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 297ADC06178B for ; Thu, 4 Feb 2021 09:29:11 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id c127so3682961wmf.5 for ; Thu, 04 Feb 2021 09:29:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Wf5g1tBsiAuXWWCi8IkrUKPbvvUqMLHqfwWU02yQBFw=; b=yt9iTqx9dfQKksFWrw/IwVn2S2Xa2eBB9TNBrZchb49oUDjjQI6oBGzYH8WF3FOetV UOXsm395cG+TyTqbvV1xY8eCalGeMEUZ934tkbBUXODTAIz3zCK6tds6E+l1fz9piswG AVV/M186yrTJjgdoK2XkHS0dHV+8fVdgEI4CdJUocSzQzaRZz6yuvEdPWpI50wNQmcOM zI8CpNLxHdAnb4TyhGAa7wsypcVWrbfyY1uUfRhSLnbu3l1ZrbPwAmZMKcdP21JPa195 k+BPZ5u1f32Qrod23nYlFDnJV2wHnyUe0jNM5M/H2TNhfBNhWrByFDsg1JToHGmTXoho FYDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wf5g1tBsiAuXWWCi8IkrUKPbvvUqMLHqfwWU02yQBFw=; b=C+HufJkyfM/cPhkBxIwASlLGZ5X9gfEqliItmWA+f7uEeXGd8C21p68wTiqOY1Vpno U3mUjXD4wIC16lsCZGsbJlsBz2m62DXPhcVBylAVJxYqNUrGHDzldTiU6RGNH6eQ1Kzx WKIHgFWPfjd/31iQABLg1bWDi7JQPe1+G+RsYFRUky5CvVwKvb/VCT6mISXuS7cKnsxE 3iitPqsN1TSaxneRZvN+jLF1smW8jlR+IVR9D5UM4FawUvjZOzgbHT5fBj6vp/eOND6h o/KYutuS5kJ9tqYUKCcdcG3fVWnXsnTqIuq7DLm+hUQkc2tpcERsU5FkdSM8ItlgMPX1 j4kQ== X-Gm-Message-State: AOAM5323M2NKCV48DLa9tBzX5F0pHcxQqGGPwVGlHGPlwI7P97Ts2uKn KjCMhpDIrnznMPBcME8gyic8dG4/Ps5qrA== X-Received: by 2002:a7b:c5cc:: with SMTP id n12mr206107wmk.123.1612459749560; Thu, 04 Feb 2021 09:29:09 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:08 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 02/10] futex: Remove rt_mutex_deadlock_account_*() Date: Thu, 4 Feb 2021 17:28:55 +0000 Message-Id: <20210204172903.2860981-3-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra These are unused and clutter up the code. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.652692478@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/locking/rtmutex-debug.c | 9 -------- kernel/locking/rtmutex-debug.h | 3 --- kernel/locking/rtmutex.c | 42 +++++++++++++--------------------- kernel/locking/rtmutex.h | 2 -- 4 files changed, 16 insertions(+), 40 deletions(-) -- 2.25.1 diff --git a/kernel/locking/rtmutex-debug.c b/kernel/locking/rtmutex-debug.c index 62b6cee8ea7f9..0613c4b1d0596 100644 --- a/kernel/locking/rtmutex-debug.c +++ b/kernel/locking/rtmutex-debug.c @@ -173,12 +173,3 @@ void debug_rt_mutex_init(struct rt_mutex *lock, const char *name) lock->name = name; } -void -rt_mutex_deadlock_account_lock(struct rt_mutex *lock, struct task_struct *task) -{ -} - -void rt_mutex_deadlock_account_unlock(struct task_struct *task) -{ -} - diff --git a/kernel/locking/rtmutex-debug.h b/kernel/locking/rtmutex-debug.h index d0519c3432b67..b585af9a1b508 100644 --- a/kernel/locking/rtmutex-debug.h +++ b/kernel/locking/rtmutex-debug.h @@ -9,9 +9,6 @@ * This file contains macros used solely by rtmutex.c. Debug version. */ -extern void -rt_mutex_deadlock_account_lock(struct rt_mutex *lock, struct task_struct *task); -extern void rt_mutex_deadlock_account_unlock(struct task_struct *task); extern void debug_rt_mutex_init_waiter(struct rt_mutex_waiter *waiter); extern void debug_rt_mutex_free_waiter(struct rt_mutex_waiter *waiter); extern void debug_rt_mutex_init(struct rt_mutex *lock, const char *name); diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 3323ef935372f..e3dd1642423f8 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -937,8 +937,6 @@ takeit: */ rt_mutex_set_owner(lock, task); - rt_mutex_deadlock_account_lock(lock, task); - return 1; } @@ -1331,8 +1329,6 @@ static bool __sched rt_mutex_slowunlock(struct rt_mutex *lock, debug_rt_mutex_unlock(lock); - rt_mutex_deadlock_account_unlock(current); - /* * We must be careful here if the fast path is enabled. If we * have no waiters queued we cannot set owner to NULL here @@ -1398,11 +1394,10 @@ rt_mutex_fastlock(struct rt_mutex *lock, int state, struct hrtimer_sleeper *timeout, enum rtmutex_chainwalk chwalk)) { - if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 0; - } else - return slowfn(lock, state, NULL, RT_MUTEX_MIN_CHAINWALK); + + return slowfn(lock, state, NULL, RT_MUTEX_MIN_CHAINWALK); } static inline int @@ -1414,21 +1409,19 @@ rt_mutex_timed_fastlock(struct rt_mutex *lock, int state, enum rtmutex_chainwalk chwalk)) { if (chwalk == RT_MUTEX_MIN_CHAINWALK && - likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 0; - } else - return slowfn(lock, state, timeout, chwalk); + + return slowfn(lock, state, timeout, chwalk); } static inline int rt_mutex_fasttrylock(struct rt_mutex *lock, int (*slowfn)(struct rt_mutex *lock)) { - if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) { - rt_mutex_deadlock_account_lock(lock, current); + if (likely(rt_mutex_cmpxchg_acquire(lock, NULL, current))) return 1; - } + return slowfn(lock); } @@ -1438,19 +1431,18 @@ rt_mutex_fastunlock(struct rt_mutex *lock, struct wake_q_head *wqh)) { WAKE_Q(wake_q); + bool deboost; - if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) { - rt_mutex_deadlock_account_unlock(current); + if (likely(rt_mutex_cmpxchg_release(lock, current, NULL))) + return; - } else { - bool deboost = slowfn(lock, &wake_q); + deboost = slowfn(lock, &wake_q); - wake_up_q(&wake_q); + wake_up_q(&wake_q); - /* Undo pi boosting if necessary: */ - if (deboost) - rt_mutex_adjust_prio(current); - } + /* Undo pi boosting if necessary: */ + if (deboost) + rt_mutex_adjust_prio(current); } /** @@ -1648,7 +1640,6 @@ void rt_mutex_init_proxy_locked(struct rt_mutex *lock, __rt_mutex_init(lock, NULL); debug_rt_mutex_proxy_lock(lock, proxy_owner); rt_mutex_set_owner(lock, proxy_owner); - rt_mutex_deadlock_account_lock(lock, proxy_owner); } /** @@ -1664,7 +1655,6 @@ void rt_mutex_proxy_unlock(struct rt_mutex *lock, { debug_rt_mutex_proxy_unlock(lock); rt_mutex_set_owner(lock, NULL); - rt_mutex_deadlock_account_unlock(proxy_owner); } /** diff --git a/kernel/locking/rtmutex.h b/kernel/locking/rtmutex.h index c4060584c4076..6607802efa8bd 100644 --- a/kernel/locking/rtmutex.h +++ b/kernel/locking/rtmutex.h @@ -11,8 +11,6 @@ */ #define rt_mutex_deadlock_check(l) (0) -#define rt_mutex_deadlock_account_lock(m, t) do { } while (0) -#define rt_mutex_deadlock_account_unlock(l) do { } while (0) #define debug_rt_mutex_init_waiter(w) do { } while (0) #define debug_rt_mutex_free_waiter(w) do { } while (0) #define debug_rt_mutex_lock(l) do { } while (0) From patchwork Thu Feb 4 17:28:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376315 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1492379jah; Thu, 4 Feb 2021 09:47:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJx9aZJeksimLiFVLK6b1wRX8kBD5kvIKRIdIyvFhjqyNwfApTrZ+6zWyAwZUdoCj3n7RfZM X-Received: by 2002:a05:6402:1152:: with SMTP id g18mr154122edw.18.1612460858490; Thu, 04 Feb 2021 09:47:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460858; cv=none; d=google.com; s=arc-20160816; b=hHDfnPK6zEz1atOnoRH/xERJXrHeeviiVW3oQ98NPqeYzNIuQi0dMiuridciLad3yC 6HGuhfeM1NDBAFQq5usWbqPmH/GiT2gaI1bX7Tm7e4mcfNFY9HB92htbJTVgLsP9S+9M VZ85SxMvRzrs3AfNDfSL6IhAoCBKyqzSfFAl+XsHz/1Rv3y3dTouaNKXrxaNh1I92wIr pZnw5wueyzVnLcLjuFNA9YAiSRmuWJnbF2HKr9lDNC2Us3rVT+lDNFIopRrFR7Jt8wGQ NMAIEl+31WjQlmiY3V7IaCiUTUO65qRRTz0fd79GDOXSQILfCT9C2UHvTfYb5xfsOkHk VM8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=t1Tdhu9sPTY73dg7/Bf+eqje/Zu+kzB0VXGmoviIFF4=; b=wWphBLkDHVbmn67TRfavwakU8uzTVlsfoJWmFkZ9oWNE0F5VoT/SQOGi7cgECV/wSG VjeDgCyoKn/2cuKJOn5WWRPcSyOjLzU9mJcK6/E6u5H6eApGAj13V8wi3sC7xSpfXgvh ce9OBFS/zXuxwsJ0odcUOyX/EN6LLT4/YzsFULSCUCtyXq1+W7pGCkE+nVSE3Yzqf+1e V5tS39CQNxbzDTa+SAXxCbJTQyelL2wutps9vWlI1Hd17/pm3dl2UFqVBoG7M0/hxplW 5IhURuRYBQrvVJYFLAmpsipvm+UblesIoY9cMuS7qe68prqjspbHO0soX5PS8yowKEdv gS0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EKiI2lzv; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id z17si3584606ejc.207.2021.02.04.09.47.37; Thu, 04 Feb 2021 09:47:38 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EKiI2lzv; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238456AbhBDRqb (ORCPT + 13 others); Thu, 4 Feb 2021 12:46:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238522AbhBDRaK (ORCPT ); Thu, 4 Feb 2021 12:30:10 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFA5AC06178C for ; Thu, 4 Feb 2021 09:29:12 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id l12so4525179wry.2 for ; Thu, 04 Feb 2021 09:29:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t1Tdhu9sPTY73dg7/Bf+eqje/Zu+kzB0VXGmoviIFF4=; b=EKiI2lzv+CAJcOpp0AeAe/WMXEeawAjECzkvWSdJ6WsxFSTiBHWsOUvaZv7FlEUivd SzA0rypFDIcCo/Gdn+fr35Y6spj/1R+wCumlFLM07BQtBgRdP884Qu7XKDsv4Cj5TgnX GxrdUH5vATme9cks1F/CfDZUUy5N5i4PftM6uboozqMG5jxNwOO2uzRwxGHKSGnoJlNK npeFVWKCVLjdsQNgJIzpvxMUmZAG+tN4KdrSrBv5zv4rC+wH63lmxxUtEapYi7d1eXLc oPC4e9eSZmIoRraOTwH+oV2QrLK3hIs2/jktxZFjLF+KWJ0JRFZOnTpdiKo0sbFY64oM nddg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t1Tdhu9sPTY73dg7/Bf+eqje/Zu+kzB0VXGmoviIFF4=; b=IQfMcsuR2F9l2WXd0jHN/nsryAx/jrOHSuoSraohF2SidK5qem11JB0Y3mlLypQJdy dsXdP6+cHjnEHR/gFEyf1QELU8e8IhoPhckF6+n8SWTYou094y/18euF6l4QTdYTy2UK JMUvrmynTi3RAl4pGkZYp2zh9rl4d3n6W1CstjvHLEg4n6OBZ412kjuU6tNmEHSV4lQB eBmFwOmM0gUxjZOEPwv20bmXfLQ8kT7kp83wjp2FFaxc9cekbU0zAb0YIGmF/ToA5yyV JhbtRjlqna52Zek0T89Q2implXAiSI+fQbygq7HWraoVfHc+a2y8NBWTPPWQAqbvzoZm 9KBA== X-Gm-Message-State: AOAM533RPquJ0iiKUzis7rXzdPwcT7UJVFp2mst4lEwR7d5/prLmq08q byqlFYyO9sngBZ/0/nwPHjeftYwqx5oiyw== X-Received: by 2002:adf:f606:: with SMTP id t6mr456334wrp.360.1612459751182; Thu, 04 Feb 2021 09:29:11 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:10 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , juri.lelli@arm.com, bigeasy@linutronix.de, xlpang@redhat.com, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jdesfossez@efficios.com, dvhart@infradead.org, bristot@redhat.com, Thomas Gleixner , Lee Jones Subject: [PATCH 03/10] futex: Rework inconsistent rt_mutex/futex_q state Date: Thu, 4 Feb 2021 17:28:56 +0000 Message-Id: <20210204172903.2860981-4-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra [Upstream commit 73d786bd043ebc855f349c81ea805f6b11cbf2aa ] There is a weird state in the futex_unlock_pi() path when it interleaves with a concurrent futex_lock_pi() at the point where it drops hb->lock. In this case, it can happen that the rt_mutex wait_list and the futex_q disagree on pending waiters, in particular rt_mutex will find no pending waiters where futex_q thinks there are. In this case the rt_mutex unlock code cannot assign an owner. The futex side fixup code has to cleanup the inconsistencies with quite a bunch of interesting corner cases. Simplify all this by changing wake_futex_pi() to return -EAGAIN when this situation occurs. This then gives the futex_lock_pi() code the opportunity to continue and the retried futex_unlock_pi() will now observe a coherent state. The only problem is that this breaks RT timeliness guarantees. That is, consider the following scenario: T1 and T2 are both pinned to CPU0. prio(T2) > prio(T1) CPU0 T1 lock_pi() queue_me() <- Waiter is visible preemption T2 unlock_pi() loops with -EAGAIN forever Which is undesirable for PI primitives. Future patches will rectify this. Signed-off-by: Peter Zijlstra (Intel) Cc: juri.lelli@arm.com Cc: bigeasy@linutronix.de Cc: xlpang@redhat.com Cc: rostedt@goodmis.org Cc: mathieu.desnoyers@efficios.com Cc: jdesfossez@efficios.com Cc: dvhart@infradead.org Cc: bristot@redhat.com Link: http://lkml.kernel.org/r/20170322104151.850383690@infradead.org Signed-off-by: Thomas Gleixner [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 50 ++++++++++++++------------------------------------ 1 file changed, 14 insertions(+), 36 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 00b474b4b54e0..a5a91a55c451f 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1389,12 +1389,19 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, new_owner = rt_mutex_next_owner(&pi_state->pi_mutex); /* - * It is possible that the next waiter (the one that brought - * this owner to the kernel) timed out and is no longer - * waiting on the lock. + * When we interleave with futex_lock_pi() where it does + * rt_mutex_timed_futex_lock(), we might observe @this futex_q waiter, + * but the rt_mutex's wait_list can be empty (either still, or again, + * depending on which side we land). + * + * When this happens, give up our locks and try again, giving the + * futex_lock_pi() instance time to complete, either by waiting on the + * rtmutex or removing itself from the futex queue. */ - if (!new_owner) - new_owner = this->task; + if (!new_owner) { + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); + return -EAGAIN; + } /* * We pass it to the next owner. The WAITERS bit is always @@ -2337,7 +2344,6 @@ static long futex_wait_restart(struct restart_block *restart); */ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) { - struct task_struct *owner; int ret = 0; if (locked) { @@ -2350,44 +2356,16 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) goto out; } - /* - * Catch the rare case, where the lock was released when we were on the - * way back before we locked the hash bucket. - */ - if (q->pi_state->owner == current) { - /* - * Try to get the rt_mutex now. This might fail as some other - * task acquired the rt_mutex after we removed ourself from the - * rt_mutex waiters list. - */ - if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { - locked = 1; - goto out; - } - - /* - * pi_state is incorrect, some other task did a lock steal and - * we returned due to timeout or signal without taking the - * rt_mutex. Too late. - */ - raw_spin_lock(&q->pi_state->pi_mutex.wait_lock); - owner = rt_mutex_owner(&q->pi_state->pi_mutex); - if (!owner) - owner = rt_mutex_next_owner(&q->pi_state->pi_mutex); - raw_spin_unlock(&q->pi_state->pi_mutex.wait_lock); - ret = fixup_pi_state_owner(uaddr, q, owner); - goto out; - } - /* * Paranoia check. If we did not take the lock, then we should not be * the owner of the rt_mutex. */ - if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) + if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) { printk(KERN_ERR "fixup_owner: ret = %d pi-mutex: %p " "pi-state %p\n", ret, q->pi_state->pi_mutex.owner, q->pi_state->owner); + } out: return ret ? ret : locked; From patchwork Thu Feb 4 17:28:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376308 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1481977jah; Thu, 4 Feb 2021 09:32:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJxjPicVsL59eps2ftcEi6nvgUMtM0X+kZj84Y79C87AcpGNbA46C8Mfq9x+NU5i8wnS++N8 X-Received: by 2002:aa7:c88a:: with SMTP id p10mr72920eds.204.1612459959641; Thu, 04 Feb 2021 09:32:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612459959; cv=none; d=google.com; s=arc-20160816; b=xbawCBFuE0GvoT3oed8hwpAMB1UQSs57qSgUADrXyltlnlg3QT3We7zeroXG3AHsve 35dLtB7emNEVZAAZTk9wOMZ6L6KRvwcJ6RZekbtbTds3VzeDKU5tA/HufboMHSRXEuFU IxX/sEHK5VwwIIypejTne4bTV2PwwAJOFpnSA7DMzwpeT3tsPtcH09px5kF/Ofzb2PkH nOWckSUQhzA8xab3xuL2mJ+kypx0uHH6b6Lx487BJwmU9N3hA9cR6H3FMFYtgXBSOhzw TsvsnCJ3/74pOAD3jbf7gOoKL76Fu+yoMMWj4abHyqrBCBbWhkDtXuZRb1PwU01aSTh8 JCmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OeZnmFAMRqXu+BLMGhYtdJauLwns5kAZmT2Z5KbH+AM=; b=OE1LtJSbf+ybMji5DJU01u0wU3flg0pdmuYoBm7A5cFvRj9Au9LLio4qQffcH9W3eO DB/StRviEcYg+1P2oFsaSFWveaC7CQ7+LjzUBh17AQqh7usazET29nFl3mA5QWAxV4Y/ 04KAHXuxwKx0Hh05CtY9wv2iqQI6Trcprd7tPlmEih+YErJ7hAXLj9xVykmQ01Y95atL EYl9uwbyLAJCJYwvaY4HK5G3L2LX5EfoEU2MZ8KSvVs0HcS9L/cVldC5D5jqSc04Z6Fz dA4n0/wygrtKZpk7nO6FBeas/QLZrkQdbAv8N4PtAraagSHntTgsWoUdRVFQxyvj+qNn kjXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PfJ2wW3e; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l23si3310431ejx.701.2021.02.04.09.32.38; Thu, 04 Feb 2021 09:32:39 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PfJ2wW3e; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238422AbhBDRbl (ORCPT + 13 others); Thu, 4 Feb 2021 12:31:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238526AbhBDRaK (ORCPT ); Thu, 4 Feb 2021 12:30:10 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14F2CC061793 for ; Thu, 4 Feb 2021 09:29:14 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id q7so4426286wre.13 for ; Thu, 04 Feb 2021 09:29:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OeZnmFAMRqXu+BLMGhYtdJauLwns5kAZmT2Z5KbH+AM=; b=PfJ2wW3ed357ljBROw+x5Hu1pIRKkKx0J1vu70rtYA7thBW0zQJjx71IP0jVT64sP1 dcgx30riEl6tKl+CkqytHQ27qMpkNEspv0okSYRiTLwaiSPPQTlNwPgHHl6GADDbWEil wLP0VeRjM7PktbLDOvbr39G4wp1Qw/Oci91LrfvjtrZdByYmUQ/TYbWcjSMIF1eqmwxC 9+ECHd8VUC/eiKUompBV25w0XaS5JDeXd1jz1Nso/JBAwxyXmfvZ+Y/AOq22LXwt8rKn 8yLgUKe3PHYXtH2MkK+UkzvMyMaLHNAxJhybSghQwnK7zD+d3bWMjjBiEn5u4TCV+MUo Long== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OeZnmFAMRqXu+BLMGhYtdJauLwns5kAZmT2Z5KbH+AM=; b=Y/eVb1CZpdGaxiDuooxdFzT7puRPphrx0FBSl4Chu/7HpqBav1Js0qqZLUSESE6iDj Rl7LGajAn/NbyTl56+sMd4mXslC+XcgwVH/qXlNx0qN2oIkB2x3/VX6WAcYWBCiSfCrS mFav0Re4klIdUHVEG1eH3znNzpPtGs0khPh7eS4upYeGy7zqVTA8LC4QwGIlMqyMXzUK 4HcFFVa1pIBs1ovZFysufG7y1VqIifx6TMedNo2fqpGIWe+tED9rkzXDMGnY3K9OiXfR rViT+9G2vcWb44LsYkrH44QAwgsDjzFW6BL//JZceqVp0mfqI4IjmBpkAuCDys/Asuzx RJpg== X-Gm-Message-State: AOAM532sZBpImE368/iqaLzQVOC+F4wMxaTK7EjU56Iet6CsbKYaRp7Y S1fCUcZ5im470zz7WQHRS5kuJ/ZTDuN2KQ== X-Received: by 2002:adf:f18a:: with SMTP id h10mr399573wro.299.1612459752354; Thu, 04 Feb 2021 09:29:12 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:11 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Peter Zijlstra , Julia Cartwright , Gratian Crisan , Thomas Gleixner , Darren Hart , Greg Kroah-Hartman , Lee Jones Subject: [PATCH 04/10] futex: Avoid violating the 10th rule of futex Date: Thu, 4 Feb 2021 17:28:57 +0000 Message-Id: <20210204172903.2860981-5-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra commit c1e2f0eaf015fb7076d51a339011f2383e6dd389 upstream. Julia reported futex state corruption in the following scenario: waiter waker stealer (prio > waiter) futex(WAIT_REQUEUE_PI, uaddr, uaddr2, timeout=[N ms]) futex_wait_requeue_pi() futex_wait_queue_me() freezable_schedule() futex(LOCK_PI, uaddr2) futex(CMP_REQUEUE_PI, uaddr, uaddr2, 1, 0) /* requeues waiter to uaddr2 */ futex(UNLOCK_PI, uaddr2) wake_futex_pi() cmp_futex_value_locked(uaddr2, waiter) wake_up_q() task> futex(LOCK_PI, uaddr2) __rt_mutex_start_proxy_lock() try_to_take_rt_mutex() /* steals lock */ rt_mutex_set_owner(lock, stealer) rt_mutex_wait_proxy_lock() __rt_mutex_slowlock() try_to_take_rt_mutex() /* fails, lock held by stealer */ if (timeout && !timeout->task) return -ETIMEDOUT; fixup_owner() /* lock wasn't acquired, so, fixup_pi_state_owner skipped */ return -ETIMEDOUT; /* At this point, we've returned -ETIMEDOUT to userspace, but the * futex word shows waiter to be the owner, and the pi_mutex has * stealer as the owner */ futex_lock(LOCK_PI, uaddr2) -> bails with EDEADLK, futex word says we're owner. And suggested that what commit: 73d786bd043e ("futex: Rework inconsistent rt_mutex/futex_q state") removes from fixup_owner() looks to be just what is needed. And indeed it is -- I completely missed that requeue_pi could also result in this case. So we need to restore that, except that subsequent patches, like commit: 16ffa12d7425 ("futex: Pull rt_mutex_futex_unlock() out from under hb->lock") changed all the locking rules. Even without that, the sequence: - if (rt_mutex_futex_trylock(&q->pi_state->pi_mutex)) { - locked = 1; - goto out; - } - raw_spin_lock_irq(&q->pi_state->pi_mutex.wait_lock); - owner = rt_mutex_owner(&q->pi_state->pi_mutex); - if (!owner) - owner = rt_mutex_next_owner(&q->pi_state->pi_mutex); - raw_spin_unlock_irq(&q->pi_state->pi_mutex.wait_lock); - ret = fixup_pi_state_owner(uaddr, q, owner); already suggests there were races; otherwise we'd never have to look at next_owner. So instead of doing 3 consecutive wait_lock sections with who knows what races, we do it all in a single section. Additionally, the usage of pi_state->owner in fixup_owner() was only safe because only the rt_mutex owner would modify it, which this additional case wrecks. Luckily the values can only change away and not to the value we're testing, this means we can do a speculative test and double check once we have the wait_lock. Fixes: 73d786bd043e ("futex: Rework inconsistent rt_mutex/futex_q state") Reported-by: Julia Cartwright Reported-by: Gratian Crisan Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Thomas Gleixner Tested-by: Julia Cartwright Tested-by: Gratian Crisan Cc: Darren Hart Link: https://lkml.kernel.org/r/20171208124939.7livp7no2ov65rrc@hirez.programming.kicks-ass.net Signed-off-by: Greg Kroah-Hartman [Lee: Back-ported to solve a dependency] Signed-off-by: Lee Jones --- kernel/futex.c | 80 +++++++++++++++++++++++++++------ kernel/locking/rtmutex.c | 26 ++++++++--- kernel/locking/rtmutex_common.h | 1 + 3 files changed, 87 insertions(+), 20 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index a5a91a55c451f..780872ac7d675 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2227,30 +2227,34 @@ static void unqueue_me_pi(struct futex_q *q) spin_unlock(q->lock_ptr); } -/* - * Fixup the pi_state owner with the new owner. - * - * Must be called with hash bucket lock held and mm->sem held for non - * private futexes. - */ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, - struct task_struct *newowner) + struct task_struct *argowner) { - u32 newtid = task_pid_vnr(newowner) | FUTEX_WAITERS; struct futex_pi_state *pi_state = q->pi_state; - struct task_struct *oldowner = pi_state->owner; u32 uval, uninitialized_var(curval), newval; + struct task_struct *oldowner, *newowner; + u32 newtid; int ret; + lockdep_assert_held(q->lock_ptr); + + oldowner = pi_state->owner; /* Owner died? */ if (!pi_state->owner) newtid |= FUTEX_OWNER_DIED; /* - * We are here either because we stole the rtmutex from the - * previous highest priority waiter or we are the highest priority - * waiter but failed to get the rtmutex the first time. - * We have to replace the newowner TID in the user space variable. + * We are here because either: + * + * - we stole the lock and pi_state->owner needs updating to reflect + * that (@argowner == current), + * + * or: + * + * - someone stole our lock and we need to fix things to point to the + * new owner (@argowner == NULL). + * + * Either way, we have to replace the TID in the user space variable. * This must be atomic as we have to preserve the owner died bit here. * * Note: We write the user space value _before_ changing the pi_state @@ -2264,6 +2268,39 @@ static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, * in lookup_pi_state. */ retry: + if (!argowner) { + if (oldowner != current) { + /* + * We raced against a concurrent self; things are + * already fixed up. Nothing to do. + */ + return 0; + } + + if (__rt_mutex_futex_trylock(&pi_state->pi_mutex)) { + /* We got the lock after all, nothing to fix. */ + return 0; + } + + /* + * Since we just failed the trylock; there must be an owner. + */ + newowner = rt_mutex_owner(&pi_state->pi_mutex); + BUG_ON(!newowner); + } else { + WARN_ON_ONCE(argowner != current); + if (oldowner == current) { + /* + * We raced against a concurrent self; things are + * already fixed up. Nothing to do. + */ + return 0; + } + newowner = argowner; + } + + newtid = task_pid_vnr(newowner) | FUTEX_WAITERS; + if (get_futex_value_locked(&uval, uaddr)) goto handle_fault; @@ -2350,12 +2387,29 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) /* * Got the lock. We might not be the anticipated owner if we * did a lock-steal - fix up the PI-state in that case: + * + * Speculative pi_state->owner read (we don't hold wait_lock); + * since we own the lock pi_state->owner == current is the + * stable state, anything else needs more attention. */ if (q->pi_state->owner != current) ret = fixup_pi_state_owner(uaddr, q, current); goto out; } + /* + * If we didn't get the lock; check if anybody stole it from us. In + * that case, we need to fix up the uval to point to them instead of + * us, otherwise bad things happen. [10] + * + * Another speculative read; pi_state->owner == current is unstable + * but needs our attention. + */ + if (q->pi_state->owner == current) { + ret = fixup_pi_state_owner(uaddr, q, NULL); + goto out; + } + /* * Paranoia check. If we did not take the lock, then we should not be * the owner of the rt_mutex. diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index e3dd1642423f8..45d3c9aec8533 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1284,6 +1284,19 @@ rt_mutex_slowlock(struct rt_mutex *lock, int state, return ret; } +static inline int __rt_mutex_slowtrylock(struct rt_mutex *lock) +{ + int ret = try_to_take_rt_mutex(lock, current, NULL); + + /* + * try_to_take_rt_mutex() sets the lock waiters bit + * unconditionally. Clean this up. + */ + fixup_rt_mutex_waiters(lock); + + return ret; +} + /* * Slow path try-lock function: */ @@ -1305,13 +1318,7 @@ static inline int rt_mutex_slowtrylock(struct rt_mutex *lock) */ raw_spin_lock(&lock->wait_lock); - ret = try_to_take_rt_mutex(lock, current, NULL); - - /* - * try_to_take_rt_mutex() sets the lock waiters bit - * unconditionally. Clean this up. - */ - fixup_rt_mutex_waiters(lock); + ret = __rt_mutex_slowtrylock(lock); raw_spin_unlock(&lock->wait_lock); @@ -1496,6 +1503,11 @@ int __sched rt_mutex_futex_trylock(struct rt_mutex *lock) return rt_mutex_slowtrylock(lock); } +int __sched __rt_mutex_futex_trylock(struct rt_mutex *lock) +{ + return __rt_mutex_slowtrylock(lock); +} + /** * rt_mutex_timed_lock - lock a rt_mutex interruptible * the timeout structure is provided diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index cdcaccfb74432..ea7310b9ce83a 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -113,6 +113,7 @@ extern bool rt_mutex_cleanup_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter); extern int rt_mutex_timed_futex_lock(struct rt_mutex *l, struct hrtimer_sleeper *to); extern int rt_mutex_futex_trylock(struct rt_mutex *l); +extern int __rt_mutex_futex_trylock(struct rt_mutex *l); extern void rt_mutex_futex_unlock(struct rt_mutex *lock); extern bool __rt_mutex_futex_unlock(struct rt_mutex *lock, From patchwork Thu Feb 4 17:28:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376309 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1481986jah; Thu, 4 Feb 2021 09:32:40 -0800 (PST) X-Google-Smtp-Source: ABdhPJy8Y+0BhLvIlWfHBksYAzxH6rj0v0N2o+4g3CynpZnE29b+pjSkgJDAA2koInCfhmjt9sf+ X-Received: by 2002:a05:6402:424a:: with SMTP id g10mr66908edb.236.1612459960313; Thu, 04 Feb 2021 09:32:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612459960; cv=none; d=google.com; s=arc-20160816; b=yOaTKB2wwiKboDwDe3McxqNPEeJ4QniGKm/BDVk63TiutGTR+MzXC1fSjHu8ITVzvM +xHhSERwnxNJOWwIEt3tnCp69H6UmZ9CdyUxBZ67Ai937Pj3UbQ9010sTDNPiK6i7fUQ CWf0MBCT5Z3aOCkekU1kjY6qk8fYzRKu/Aho8os+9ao0QorPxwMnafSXHVRZhbznzMvL bkSyCxvRkhO29LmXKrGKx5DtlhGrG27AE18F8maD5omR9ROFC8ZG/Z+a9FB8VmGaAyde L8acap4c69N8i52HhzR4PWK3Zyk7pZO94Qtj+vXpwcRPHBgJe25I6PCZFA3UAgmwYdD1 kGXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lPYBVdllGLJ3bXLXYmMIkSoN54YSFIG3csr7TETPut8=; b=t5tUcab1FTebai2sZZb5tjZ4Hb2/b3tkd7nMu0OkyhxUwGurImiR4TGym00V6XasGA wEMyfdsBRGpEhYEYGifhM9KZF9etaaihaVps9mbvWa5JrIzDBx0nYZ8AXob7QLRtcObb k3JpdAGYWimITwsk/KUiG/W1pKTR83INL5mmhghlnHRGrF9brxiSpm/dGCii1NuzrM8z R3GtUX36r7irNvJ38IbR5ucejnp22eU+zfgRs9qr27A1HYZoV3dDcgU6kFRhYNkZsZm7 myuk9Fkt8Gut4P26N02Pu3FSSH9FSVbd0WeWnntfMp8nIrugV6GfwnZb+5Q957Wve3Cm KBYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="BIOQPa/4"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l23si3310431ejx.701.2021.02.04.09.32.40; Thu, 04 Feb 2021 09:32:40 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="BIOQPa/4"; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238518AbhBDRbs (ORCPT + 13 others); Thu, 4 Feb 2021 12:31:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238529AbhBDRaP (ORCPT ); Thu, 4 Feb 2021 12:30:15 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F0C1C061794 for ; Thu, 4 Feb 2021 09:29:15 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id m13so4447528wro.12 for ; Thu, 04 Feb 2021 09:29:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lPYBVdllGLJ3bXLXYmMIkSoN54YSFIG3csr7TETPut8=; b=BIOQPa/49drpGC8qLqHVmeB5plHAsLeshzqlg+CctgCv3a6baDU0vfUQt24nzdeRnc XcEYqLikaPsc58jUDBwGvYrL1tq/j9FX9nlfnGeuDKfs81+ZrQR+Zl/5HTjAfOL34i/n wdp9wZ1PXZ/l0I0N7VpcWeBfM+j4sohyca9r+eujo62RwDt0Rc8ZlrBM3Faj/AdsHFi+ AZvG/F5tTc/mZHf+lKfr5aAz8owmcE9OeMN4/0SYFtU/iZd06A/60qiR8FAoBm38L3zo LTiTgKzCsLF/xMX6mC9JNvW3KmNV1ABoEE8bmDtyGjM7v3lLk+obSq4jfgEN57k+p4CD d4DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lPYBVdllGLJ3bXLXYmMIkSoN54YSFIG3csr7TETPut8=; b=dym6AvR43lvbC5XNoYeZhV9kPQnfA/6ipY3NMMSv2O28Pv79dsHGjvhj9pnaE4TnV1 Tyws+DZQgrfCCbanJJ0vCyUcjFjxsIgEmeB4KEZdxNV6Pp0RIsGu1gaMMaCbeZxtpSsm uSgrrX/abSTAk8EVJmEdkU/GgfgfNU2rRPU92HK5pdRiDAxjh4gr/S3DpSatxVawKZBo o0QNCE0SizYz3L9CfmF0amA6najOvtonUa+zJ3LPmt2bcpT3NZ+uhbX/QOm0AMsV3nbi GRIPdRsN1AcD2uf+0mr5JLk2K6h5fGEdPGOeYzCYOyTTuhr3MPGCMeV2mOhe7w3FyvSB WhMg== X-Gm-Message-State: AOAM532WtPgiWW134qSU74SBRGSgsIrFsgCPEVdFUiqf3zRYD2/+qTj3 8LCx1O47lgi1WP7ofvgPYpnBtZqUZsuWhA== X-Received: by 2002:a5d:5502:: with SMTP id b2mr413960wrv.245.1612459753662; Thu, 04 Feb 2021 09:29:13 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:12 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 05/10] futex: Replace pointless printk in fixup_owner() Date: Thu, 4 Feb 2021 17:28:58 +0000 Message-Id: <20210204172903.2860981-6-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 04b79c55201f02ffd675e1231d731365e335c307 ] If that unexpected case of inconsistent arguments ever happens then the futex state is left completely inconsistent and the printk is not really helpful. Replace it with a warning and make the state consistent. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 780872ac7d675..a247942d69799 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2412,14 +2412,10 @@ static int fixup_owner(u32 __user *uaddr, struct futex_q *q, int locked) /* * Paranoia check. If we did not take the lock, then we should not be - * the owner of the rt_mutex. + * the owner of the rt_mutex. Warn and establish consistent state. */ - if (rt_mutex_owner(&q->pi_state->pi_mutex) == current) { - printk(KERN_ERR "fixup_owner: ret = %d pi-mutex: %p " - "pi-state %p\n", ret, - q->pi_state->pi_mutex.owner, - q->pi_state->owner); - } + if (WARN_ON_ONCE(rt_mutex_owner(&q->pi_state->pi_mutex) == current)) + return fixup_pi_state_owner(uaddr, q, current); out: return ret ? ret : locked; From patchwork Thu Feb 4 17:28:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376314 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1492367jah; Thu, 4 Feb 2021 09:47:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJxqetnXl8haw92kKl3gF0kCmY53WJOU8j/Koihkj+oXpFOjjYjUz7SIhMcdEN6ETKI6GJYP X-Received: by 2002:a05:6402:1bc7:: with SMTP id ch7mr144916edb.124.1612460857616; Thu, 04 Feb 2021 09:47:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460857; cv=none; d=google.com; s=arc-20160816; b=CVKm9NwXf1L9SNtYRLRnPfyjOz1xF3tDeA5DgP1ydNWembiXVa8W3k6Vul+mSZEhlk EVHhIEls8X3Ym5xTT4NKvezHe1QIr/CVwYGGEedMlBfC6k0L+VXm4/AQM6HbzrI6m8e3 bXAbRUsHOMFuxC0mYfS9ntYM/DXXQM3epVgkKZHVdWST3BWl4VQl12E0ciEkHTiX8Qgk IKPa62+iHW0uP028FhShnmjLK0nY44ktke4fg0uovlZ72qngYY07oM0KOHBMKJW6xRq0 8u8U79nkdHOKjj/ffjPGgglC0E5MXu2zXdoV1ooBmmRMVZ6bd4PTaw3jpD7xEpuS7sbe JtPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=L9WDlXzldPwLcNIVpjw64K4i0YIfrHdhJn+ngLH0dQE=; b=G60HecH6PEP5AZR/afYJxsqLqKKQY3lqS59Fno4/M1AbWvdvauh3uJ1QqPzWQMuJe9 D8vVzudrL/j5pNo12LrfBcnJh4Uzn1oXjvDQDIkdpN4rLna3uIOAwckYYr3mFTC1Nlmb Y2zoYOebeht0r2IvTq5FX5/aVPWpI1XxARtU5y0bfc75tpiG6CTALkmtU7YUkV6kM6Ta jXCQKZHhtjp9/ulYIgRMDOw9SRNUc4rM25xZeK+q0qXkGVJPFoRj/hVIt+cF+jWjoZmn jHRdV9PxyGtdfo2vfG6PhjXUvtbWJAvnLVVAv67W/huVpLb3rFVJ/npZAj2Z//+ErgI/ ToEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NTLTlxqX; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id z17si3584606ejc.207.2021.02.04.09.47.35; Thu, 04 Feb 2021 09:47:37 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NTLTlxqX; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237290AbhBDRqY (ORCPT + 13 others); Thu, 4 Feb 2021 12:46:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238530AbhBDRaP (ORCPT ); Thu, 4 Feb 2021 12:30:15 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29E48C061797 for ; Thu, 4 Feb 2021 09:29:16 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id 7so4551333wrz.0 for ; Thu, 04 Feb 2021 09:29:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L9WDlXzldPwLcNIVpjw64K4i0YIfrHdhJn+ngLH0dQE=; b=NTLTlxqXUs9SwMrPjNG0ZyDO1lAKLCgMYooX9qcFgzVmPo3prVb0uDJfMjfEPYaDKf eICcyeIfhn1SxbnsPc1KoqwxPsJ7LtyDgF/CXrcobUa7U48X/CA+c6ztlB30K/95J/aF FIzbfbLlRlV5sBJtt+M/KHkhPm3LWeGSm9ibbFnPIWvNcO3hqGAfMj96/ddmWtyz+wmi 6gClkTxXEdLakFp8MoKH4rHXNB39SCj0BPgO7OfoQZlh+T7oo3bZa10H+1Ya7zFEAiOG 0Qs/outahWmrt6sW48Gg/9lehbF3NbsHfNUC2MVmG7yuflHTvPlwJthaIp+Y9wicMBfT iD2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L9WDlXzldPwLcNIVpjw64K4i0YIfrHdhJn+ngLH0dQE=; b=KB76HkRCIke0xA9oR4bfPlLnzI5lBGKlt9dJkutYSrSj08527Ksr7hMrSR4B6lxAbs yTwExnNBbcSxBltPVUfURYeaTif0wIdwvyrPPIAk+O8DdQ5L4/4STnfXt6myJohZ2w9c hm2gHM8njPmWitPqSEaFlUqo/2Wkp3WfIM5K4pzpG+n/zZBNGFf9dqCbjHPt1Dq2Bhur 3F0nCZqbDAN8K8Py1TwNI/jN0KisuB3cI++kRwbRosAskRo6szEDEpSc3BDHm+A347YS TA2xr1DtnaQzxotSlPowEZ6d4sS1NzAPguOaRSxIlh5rC36yQOnklUplSPjxb+/INPh6 /gCg== X-Gm-Message-State: AOAM532KTMB2lI4wLId29ACaLHD7rtwgWoiGNqnuXCVbhMYUeQEKY8jN +HWDRJXXHZy84u94zEcActX78XBJiy+F1Q== X-Received: by 2002:adf:a2ca:: with SMTP id t10mr454572wra.370.1612459754633; Thu, 04 Feb 2021 09:29:14 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:14 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 06/10] futex: Provide and use pi_state_update_owner() Date: Thu, 4 Feb 2021 17:28:59 +0000 Message-Id: <20210204172903.2860981-7-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit c5cade200ab9a2a3be9e7f32a752c8d86b502ec7 ] Updating pi_state::owner is done at several places with the same code. Provide a function for it and use that at the obvious places. This is also a preparation for a bug fix to avoid yet another copy of the same code or alternatively introducing a completely unpenetratable mess of gotos. Originally-by: Peter Zijlstra Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 64 ++++++++++++++++++++++++++------------------------ 1 file changed, 33 insertions(+), 31 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index a247942d69799..1390ffa874a6b 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -835,6 +835,29 @@ static struct futex_pi_state * alloc_pi_state(void) return pi_state; } +static void pi_state_update_owner(struct futex_pi_state *pi_state, + struct task_struct *new_owner) +{ + struct task_struct *old_owner = pi_state->owner; + + lockdep_assert_held(&pi_state->pi_mutex.wait_lock); + + if (old_owner) { + raw_spin_lock(&old_owner->pi_lock); + WARN_ON(list_empty(&pi_state->list)); + list_del_init(&pi_state->list); + raw_spin_unlock(&old_owner->pi_lock); + } + + if (new_owner) { + raw_spin_lock(&new_owner->pi_lock); + WARN_ON(!list_empty(&pi_state->list)); + list_add(&pi_state->list, &new_owner->pi_state_list); + pi_state->owner = new_owner; + raw_spin_unlock(&new_owner->pi_lock); + } +} + /* * Must be called with the hb lock held. */ @@ -1427,26 +1450,16 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, else ret = -EINVAL; } - if (ret) { - raw_spin_unlock(&pi_state->pi_mutex.wait_lock); - return ret; - } - - raw_spin_lock_irq(&pi_state->owner->pi_lock); - WARN_ON(list_empty(&pi_state->list)); - list_del_init(&pi_state->list); - raw_spin_unlock_irq(&pi_state->owner->pi_lock); - raw_spin_lock_irq(&new_owner->pi_lock); - WARN_ON(!list_empty(&pi_state->list)); - list_add(&pi_state->list, &new_owner->pi_state_list); - pi_state->owner = new_owner; - raw_spin_unlock_irq(&new_owner->pi_lock); - - /* - * We've updated the uservalue, this unlock cannot fail. - */ - deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + if (!ret) { + /* + * This is a point of no return; once we modified the uval + * there is no going back and subsequent operations must + * not fail. + */ + pi_state_update_owner(pi_state, new_owner); + deboost = __rt_mutex_futex_unlock(&pi_state->pi_mutex, &wake_q); + } raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); spin_unlock(&hb->lock); @@ -2318,19 +2331,8 @@ retry: * We fixed up user space. Now we need to fix the pi_state * itself. */ - if (pi_state->owner != NULL) { - raw_spin_lock_irq(&pi_state->owner->pi_lock); - WARN_ON(list_empty(&pi_state->list)); - list_del_init(&pi_state->list); - raw_spin_unlock_irq(&pi_state->owner->pi_lock); - } - - pi_state->owner = newowner; + pi_state_update_owner(pi_state, newowner); - raw_spin_lock_irq(&newowner->pi_lock); - WARN_ON(!list_empty(&pi_state->list)); - list_add(&pi_state->list, &newowner->pi_state_list); - raw_spin_unlock_irq(&newowner->pi_lock); return 0; /* From patchwork Thu Feb 4 17:29:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376313 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1491315jah; Thu, 4 Feb 2021 09:46:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJyrWUOh+nzsBdsYulRpYndFnis11N82fEVgqVttQzkVfF8bjl4qqJrC3tASCgBthxRuB4jG X-Received: by 2002:a17:907:1b06:: with SMTP id mp6mr227712ejc.408.1612460763308; Thu, 04 Feb 2021 09:46:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460763; cv=none; d=google.com; s=arc-20160816; b=o0uBD1Coz/kN4Slo1zfQrN+aAsoWM7Dnd0uctCMU6jakB1qjGHNrd4a0zvqZ/QvJu/ mzUSsP6vvPi0lr1nTFSwqSoZB806OBqQEGNayqKeg49tD1wPGs3nL4JlvrhXxgt8bxxB RTre4pc2BE7pQtoAiH7++XayukbrGndXM94JHXX4FWM01e22fqFsn90KqI7+kAb1cExT IJ1YoycZTTYQ8zN35yTWe/6hubacy+occb+uBKbM85kyP8lC+HqaoG7lFDaTbdE0hYya tJUPfQltW4yw+tJ4NKrFUSc16GFpM8ookgs5uIXbt4+y3q1ZZTh9xFJAgeuuz3/NJTB4 LYaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=i0z1K40DidNKxQN3Eytjf2ywiefk/Nm8nOBl77/Mwk0=; b=U2+uR6mVtq7bAQOdgNtu4gexUOj3zyPESVIa9LWvvX4DdgWfQZCc96cOq0HvTLtq8Y uta3ox5kwkH0xxSlV/Kq8uDWjHy45s6nhJODycf9srcezaGG3IzBzuaTwaEA2hw004eN AkIBgKJShCX1++iQOrCbdjXa0ObMx7SJb8hfAzVX7XL16x1UeSGljUGtIXtACsB+uz/i hsQX0Y+lTm0PkFWevEK/deLwWww9ZFfxEtBLoCHFOPMLV3wGqeYRnFMbfnjNLxr6oFty clVxDa+3pWPaSN4iSG1AQV31dJedbGqswULyz/tM23mZc1tg3UZ6fhBj3HRa/3tvFQGj K9AA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lV28H1mu; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id 17si3691053ejv.562.2021.02.04.09.45.55; Thu, 04 Feb 2021 09:46:03 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lV28H1mu; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238463AbhBDRpU (ORCPT + 13 others); Thu, 4 Feb 2021 12:45:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238370AbhBDRbE (ORCPT ); Thu, 4 Feb 2021 12:31:04 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CF5BC0617A7 for ; Thu, 4 Feb 2021 09:29:17 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id b3so4492357wrj.5 for ; Thu, 04 Feb 2021 09:29:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i0z1K40DidNKxQN3Eytjf2ywiefk/Nm8nOBl77/Mwk0=; b=lV28H1muNLxLI1Z7Z6duhKI24U/TSQc60I/6Md+9844vfJekkAqWxEc8EGCF2ddcbb 0RPxFgj68JmQ3IfsazeH/vs+59gyOe79QmSO6pEVf+DmXy14s3p6k56FVyL/L+k81Kok ADvbsLfWOvCN1XS4ZDr4b8jS4gSnXdooIz7kLpeJaeyv38VZo939Grm5zhcnosTzzgs6 /K0A5wScWOXuOoVBlS2i58xr1kmUKmRIUjfBCwTEy/7Zg/G7uTXU/nrNKL/DW5NIeTLr rjoodrCOuavdQh5q6bHn6hiadYVGD8M2oxy3MfngURF60+3sDycAxufoZRDpgL6XuLnR +OPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i0z1K40DidNKxQN3Eytjf2ywiefk/Nm8nOBl77/Mwk0=; b=NhM+hSRzGasLS/uEJljqpVxN9fN+qtzvRJ0cE5rLO6+T6/9Hk6bzFNF4nzN6Iyu4xF AFXIUu35SCVn6Unf1dB51C7jCoc2SziIczIM8gKyTW0oCn3jBZpqTwWGQNLtC9IJF2qd ofRfUXtdgY0KwIAqtlW/XhI7YTGMp7AZTW8oqEg/622zifzD0iPohZrmwjmy4mFiu1bB UQHD6jRcwuPA0sLTmaCi4opa5JTZZhb08MDGKp6jxD77zpvP1CB62cqnsKAmj8M4mWHj 52gVOQKb5lBhbeOPIik9fGWmwAKg+4BRjb+2Gponx0pqmUoNdbrOjAzn83IZ1nmjvxun yH8g== X-Gm-Message-State: AOAM531l6PLrFcbSPjg1yoAvKRFQblHotpznCPdi9qacEd+3en1t2etG jr7myTveTjmpskeG1e4rw6j5AUW7veULkQ== X-Received: by 2002:a5d:6a85:: with SMTP id s5mr394962wru.283.1612459755657; Thu, 04 Feb 2021 09:29:15 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:14 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 07/10] rtmutex: Remove unused argument from rt_mutex_proxy_unlock() Date: Thu, 4 Feb 2021 17:29:00 +0000 Message-Id: <20210204172903.2860981-8-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 2156ac1934166d6deb6cd0f6ffc4c1076ec63697 ] Nothing uses the argument. Remove it as preparation to use pi_state_update_owner(). Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 2 +- kernel/locking/rtmutex.c | 3 +-- kernel/locking/rtmutex_common.h | 3 +-- 3 files changed, 3 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 1390ffa874a6b..bf40921ef1200 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -878,7 +878,7 @@ static void free_pi_state(struct futex_pi_state *pi_state) list_del_init(&pi_state->list); raw_spin_unlock_irq(&pi_state->owner->pi_lock); - rt_mutex_proxy_unlock(&pi_state->pi_mutex, pi_state->owner); + rt_mutex_proxy_unlock(&pi_state->pi_mutex); } if (current->pi_state_cache) diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 45d3c9aec8533..1c0cb5c3c6ad6 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1662,8 +1662,7 @@ void rt_mutex_init_proxy_locked(struct rt_mutex *lock, * No locking. Caller has to do serializing itself * Special API call for PI-futex support */ -void rt_mutex_proxy_unlock(struct rt_mutex *lock, - struct task_struct *proxy_owner) +void rt_mutex_proxy_unlock(struct rt_mutex *lock) { debug_rt_mutex_proxy_unlock(lock); rt_mutex_set_owner(lock, NULL); diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index ea7310b9ce83a..4584db96265d4 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -101,8 +101,7 @@ enum rtmutex_chainwalk { extern struct task_struct *rt_mutex_next_owner(struct rt_mutex *lock); extern void rt_mutex_init_proxy_locked(struct rt_mutex *lock, struct task_struct *proxy_owner); -extern void rt_mutex_proxy_unlock(struct rt_mutex *lock, - struct task_struct *proxy_owner); +extern void rt_mutex_proxy_unlock(struct rt_mutex *lock); extern int rt_mutex_start_proxy_lock(struct rt_mutex *lock, struct rt_mutex_waiter *waiter, struct task_struct *task); From patchwork Thu Feb 4 17:29:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376312 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1491167jah; Thu, 4 Feb 2021 09:45:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJzsLwBZaNCud87H3x/km92Lwru3sEF3Gdrg1nqLytrjnZ1rvxZ7+x/VGn50lZ3DbwjacCqW X-Received: by 2002:a05:6402:318e:: with SMTP id di14mr77603edb.223.1612460755022; Thu, 04 Feb 2021 09:45:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460755; cv=none; d=google.com; s=arc-20160816; b=gmZI+Lacfwh6FQkveMJR0pEhxF95MwedB+qK8H1yBs09MSEDLkdUMnAq+wbLWQL9V0 Td1hR7nzF2mZps4/EvHfG+QhV5h1z/ub4D0u8W0hymL1iaFOpLTGqRHeXoKi/cCogLib tHOHLdWS6+vxUTenKGXYrxlpElLMoXuq5QGRExYmmoI338KluGpG94GO7pQvz2NLMpmO EJzn4AC57Fi9J0D3wgIn59COPi4rc01kIfFDeCDiUJjleBKlVy6fAl7GD1zpMuWhm0kY 93LSlgQDiVLeEe0QN8X/+dVsSgm/iaiVtVwf4x/f3kF6YT9Npi7iUob3Um+Kl4oJ19Jo aUYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Jt6ppsBFHDM6FBIc6eMl8p62XKsDCQzpsx3kuToeRvA=; b=eVBFqsZ2/Wcw+kzf4DDjgLJPp5OvO2eUZX5CjzIzj48PfR/feTpcDfsU4MJ00qN4yW dBmAhUtWEFLHW2SmLU11TpnG6ksjdKoeA5LwAIAT3a5ulYy8eMTiGHEa5WoF1W4D8L5y hFYv0lVIA7zVu1FtHNiXePg9ycFcgM5NgbkkAu9lC85NQdwKR98HPrIidFwWETZxePfU v6TJEKz7brXfQjOLkZVlkTPhVL7kN/AuZ2jchmkqKB0I7rn33olITGPSRSZj2xpRHM5K VDD3nUNsl0iOGpSKs3pq7T/Pd0Wyq+afxD4P7Jmsy4yFjtjHmTuY/QcbcE0pxhkXWoAn oPlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C8kl80gO; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id 17si3691053ejv.562.2021.02.04.09.45.52; Thu, 04 Feb 2021 09:45:55 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C8kl80gO; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238113AbhBDRpS (ORCPT + 13 others); Thu, 4 Feb 2021 12:45:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238505AbhBDRbE (ORCPT ); Thu, 4 Feb 2021 12:31:04 -0500 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4181C0617A9 for ; Thu, 4 Feb 2021 09:29:18 -0800 (PST) Received: by mail-wm1-x32a.google.com with SMTP id o10so6360631wmc.1 for ; Thu, 04 Feb 2021 09:29:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jt6ppsBFHDM6FBIc6eMl8p62XKsDCQzpsx3kuToeRvA=; b=C8kl80gOrI7VtnIXjE6O1j2+YqM/p5AY+wSB0/y2Zm6L4PXwqtc5uCcba4NzXR6ewy y7yebs2smS8wL+54kG0bwlRcSwajvoL/9yhlDgSxGvKrXBtCca+L501bYVVqROJoey1d WaMqLVZhDs67W5/ZFl7kB3Z7QnUQ31PMn2VTN1tbJYMElXQp/vciUUAkuWtdn8+KAW0t KR5Xg2cVQUfqs0Cdz+H3t9MxHzW48bSC9Dmz78sybyfCaEwDdrE5Mdu7mOY6lod2OGy3 ZDdSrlu0hD6Dga7i0f4iNfV2wMxnZ+84xJilL111lNp6BR7TkseZ4Olfb9HTSUbLvaBE m5UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jt6ppsBFHDM6FBIc6eMl8p62XKsDCQzpsx3kuToeRvA=; b=mNkNEFNeaPgYyauTPPKiRrHltklxVAW5nBi8aMgSZQxrQVUnCLkPQlrGMGEJFeolb5 oL/lxfRlpa/Yi8Wm2uOaGxx8l+HqTtag4X5U2brzM2+RHD9sqiJcEXK3feJs5YLSK8FW O6xb3ybaG51JUlmyrfsUa26EcEZyGiTof7edAr7juBNu2eterKjZGJ4pAHIOmIr4X1qi WWCNIKvnKwgZHC37Qnm1jvohvD4UIV6/b6E/fKShVuSW+XA2dD8JhaHMdLA5B0kw74Fp y2sKhHcckkk5n4g6WN3bRkqI34Q26XK4qHDn0x/d092wssXAueKMx+jOJ87yim/RWpad Rkww== X-Gm-Message-State: AOAM532yoKWdL5PrP8Rq8wpbcD05uwizhjZFZHwZtqGbcAiGA6w7iTXV lzKqYiYwMqJhHnd3VocFPBdojjXr9A5i2A== X-Received: by 2002:a1c:e90d:: with SMTP id q13mr238175wmc.136.1612459757185; Thu, 04 Feb 2021 09:29:17 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:16 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 08/10] futex: Use pi_state_update_owner() in put_pi_state() Date: Thu, 4 Feb 2021 17:29:01 +0000 Message-Id: <20210204172903.2860981-9-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit 6ccc84f917d33312eb2846bd7b567639f585ad6d ] No point in open coding it. This way it gains the extra sanity checks. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index bf40921ef1200..d9bec8eb60969 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -874,10 +874,7 @@ static void free_pi_state(struct futex_pi_state *pi_state) * and has cleaned up the pi_state already */ if (pi_state->owner) { - raw_spin_lock_irq(&pi_state->owner->pi_lock); - list_del_init(&pi_state->list); - raw_spin_unlock_irq(&pi_state->owner->pi_lock); - + pi_state_update_owner(pi_state, NULL); rt_mutex_proxy_unlock(&pi_state->pi_mutex); } From patchwork Thu Feb 4 17:29:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376311 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1490072jah; Thu, 4 Feb 2021 09:44:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJxyI9npBr2MwBN4e+a/cGmSaBJYd2Wqd5bCLJ8JiLBQ7dQp/KJFqp9AWO9COpoie8xMuWed X-Received: by 2002:aa7:c901:: with SMTP id b1mr61191edt.329.1612460660236; Thu, 04 Feb 2021 09:44:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460660; cv=none; d=google.com; s=arc-20160816; b=yviygiyiEFkdADcwBqucjYkdZ3NelsMOaU/L9Pn9kjIF+8gBOIMKKU+2H/5WUOkznc fq7edXwmvIb8JbSC//yZ9iLCsXS14wi0hnZ8QrGwEmFHBlAf9uqWF98PwUMDa0tV3vrd gpBV3X46yrB05fDKGaHvHPL7ftSAaAM79n/KBYq5+xbHXKd0n11gCM3SQZD6mC9xoG+M P1Lx6ACp5HYR5IWTLFwgJd0Yd4pgO4l636FJof7pqNUes0/sI/5JP1M2iym6mXPeZxri iRfUxWT5MoXPUm/nBjkm9cLl/bnpM7gWVEToBpp2baJJEe48lMsWPtVsIOMfwXckvzHD WGFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kpLUGBUL3ePc9LcRSMK136epmeMCkjKyot5g5+ExSZs=; b=QAF6OrhnlQsnAP4WaRJgBo4LznCiUDGDs66Wnfs2Z29EXD05ix9j9tuoa21qDQvdEL OgZSkNxLLA8pudzoIL0lVkBCEvGQt6Mf9+RxxxyeSIRtOUMrFUgv8lIqfMiFBAWAVLGY Kq08w5Neg4lcHQjv0h7Nl1f4YMcPRME3x577N3cOOHWok6Nfka1gZRszzbRaHdK4MDDi asbUDRBzEvlbk91PJ4n81fwGkP2u54FYptpp/HojorexMmh14pIImvoISBbfkTtnx5mG vXWTuEQrmmhGK5nGl8LNyyKJo7pkqFXMdXIt8wdnbvesJPxksfqrDMlJQxmKv0FJD2T0 fMGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lrkDfKrm; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l8si3611592edt.265.2021.02.04.09.44.12; Thu, 04 Feb 2021 09:44:20 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lrkDfKrm; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238200AbhBDRnw (ORCPT + 13 others); Thu, 4 Feb 2021 12:43:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238586AbhBDRbP (ORCPT ); Thu, 4 Feb 2021 12:31:15 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DEEFFC0617AA for ; Thu, 4 Feb 2021 09:29:19 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id c12so4466684wrc.7 for ; Thu, 04 Feb 2021 09:29:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kpLUGBUL3ePc9LcRSMK136epmeMCkjKyot5g5+ExSZs=; b=lrkDfKrmqKhCtD/jgvCllWo5ju50m/wkQJTuvx7E0r9oQ6VMUclDsay9h080wssrrX CBmHhQyOiLZlowk/io5iyiBY5KK1VsIDm53Mrq0QvlydCrC6sQW/uhEIT0PltlQ6W+iL tozxel+p89Lo6FQ11NuWWTqFJ9k/rSQy5i9S0vixQQwE7M+yS17hKgJwC01LZteMFVJ2 yELnfvwNpwxmwmvMvMRFIoRTAhgXSV1srXz/EjC098m7FFrSWt7Q4CQXBde5R/uhsrz1 rxK1MZwNPUhKyWe6MtUsJIDReMLNZRBeRXb91Kqum2LaGJJFJ5fY0okjnprC7vygahs6 gWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kpLUGBUL3ePc9LcRSMK136epmeMCkjKyot5g5+ExSZs=; b=qw+koHEQvOJ/5/N7d/6W37USPGKXj3fwwcxKuTk21wror6/App5LHF46JSiD4rewk2 +kKulR7J3gl0w5GIY+OWsVbHmBPi17oTVnRdzDql93GPkngR9OwXhDbl0/5Kl4mNN+42 UTvHYxjTE9jntKAT1lYyntzIv8yORLY84lRHSScsImbiNVPN4nZkn9AwwXdLGQOFf3eA 8oS/TbQD4hIEaDtJHfrIr1aRG7MvjhTq+pQL6CtFO2jFfxA1ZUHLuTw9xm2wLScKY96e 8e8j4bISRkgc9SBaYDgmnauup2HEy4xK8EkQ+PFY+StPiTa0+JmH62uZe8DdMSolOITu m1zg== X-Gm-Message-State: AOAM5302FJX6HVPXAgFpVm/0otPjqQgnWLilfRkDIUhlzAMmGzIDnhlX ifF0Tjylhh8J0+vvlzm23OC8d/QBrVdeBA== X-Received: by 2002:adf:e50e:: with SMTP id j14mr488496wrm.162.1612459758389; Thu, 04 Feb 2021 09:29:18 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:17 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , Peter Zijlstra , Lee Jones Subject: [PATCH 09/10] futex: Simplify fixup_pi_state_owner() Date: Thu, 4 Feb 2021 17:29:02 +0000 Message-Id: <20210204172903.2860981-10-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner [ Upstream commit f2dac39d93987f7de1e20b3988c8685523247ae2 ] Too many gotos already and an upcoming fix would make it even more unreadable. Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index d9bec8eb60969..8300870666638 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -2237,18 +2237,16 @@ static void unqueue_me_pi(struct futex_q *q) spin_unlock(q->lock_ptr); } -static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, - struct task_struct *argowner) +static int __fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, + struct task_struct *argowner) { struct futex_pi_state *pi_state = q->pi_state; - u32 uval, uninitialized_var(curval), newval; struct task_struct *oldowner, *newowner; - u32 newtid; - int ret; - - lockdep_assert_held(q->lock_ptr); + u32 uval, curval, newval, newtid; + int err = 0; oldowner = pi_state->owner; + /* Owner died? */ if (!pi_state->owner) newtid |= FUTEX_OWNER_DIED; @@ -2289,7 +2287,7 @@ retry: if (__rt_mutex_futex_trylock(&pi_state->pi_mutex)) { /* We got the lock after all, nothing to fix. */ - return 0; + return 1; } /* @@ -2304,7 +2302,7 @@ retry: * We raced against a concurrent self; things are * already fixed up. Nothing to do. */ - return 0; + return 1; } newowner = argowner; } @@ -2345,7 +2343,7 @@ retry: handle_fault: spin_unlock(q->lock_ptr); - ret = fault_in_user_writeable(uaddr); + err = fault_in_user_writeable(uaddr); spin_lock(q->lock_ptr); @@ -2353,12 +2351,27 @@ handle_fault: * Check if someone else fixed it for us: */ if (pi_state->owner != oldowner) - return 0; + return argowner == current; - if (ret) - return ret; + /* Retry if err was -EAGAIN or the fault in succeeded */ + if (!err) + goto retry; - goto retry; + return err; +} + +static int fixup_pi_state_owner(u32 __user *uaddr, struct futex_q *q, + struct task_struct *argowner) +{ + struct futex_pi_state *pi_state = q->pi_state; + int ret; + + lockdep_assert_held(q->lock_ptr); + + raw_spin_lock_irq(&pi_state->pi_mutex.wait_lock); + ret = __fixup_pi_state_owner(uaddr, q, argowner); + raw_spin_unlock_irq(&pi_state->pi_mutex.wait_lock); + return ret; } static long futex_wait_restart(struct restart_block *restart); From patchwork Thu Feb 4 17:29:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lee Jones X-Patchwork-Id: 376310 Delivered-To: patch@linaro.org Received: by 2002:a02:b18a:0:0:0:0:0 with SMTP id t10csp1489974jah; Thu, 4 Feb 2021 09:44:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJxJ75rxxRb2J4edyZQUwYLL+OT4CHqcy+8ilceZW8I3kP4ud/PMNfQRGqP+KpfoOSqG1MVq X-Received: by 2002:a17:906:2ac1:: with SMTP id m1mr218272eje.149.1612460652067; Thu, 04 Feb 2021 09:44:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612460652; cv=none; d=google.com; s=arc-20160816; b=WVjHT2ylHkzlpwehddg8oj0Syd1xkijN0VPK8YylxOWzKN2QbJl8fmFxtyF4if8tNi 8gEx5aP7G389YitAf/VkTMyJxkwlakywGBs010McpmxfVxQWINGimxWXGSiM0LUDi9kV 5CsXBMxMAngPZYr4JflOGMrHeBzNqJFsi7zwVRKuReLnA1C0GTnBfL5prZuK6Rz8oNRg /csiD0JFXwvjG7KikEDs0KWRah2VK/n/cXhoAtAzehGI3lFlMZe7JmD/DyjWPCRDu4Fa 9d0tpWDsdyCjX2EOU1u3hpySQjwaK4XEEYdj2FA5hJTsfOpXFWcQVPdk6z+DLO29XjrA QC0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4ltiq3mCpsTPc6kiuj14O3FHIVaCpAzgI/dn2agPl+o=; b=NsUbmuk7/mXdeAK94YyEmdmHoCvSQNWl0hp33hwU9NNLdpAU3puXEaVYoty04Oi2hW pRRCOdHx/ngSw6m+t4zexqILjsCZe8JK4W3CBIdGYjt8nPEKgwvF1UWmI8yq4lS+/iA5 keEx4lSDRScjEbkR61uRLD22gWjgLrXAJ7/3R6NRkJtkNRGSiMqFSyhF+HWIGZZRs8SS BeQYi9mLQZ1EhnYPjpKrHiv4bdeaQmxinSf4b7th7M+oLT+Q9hRQDdoGqpkkSz8Jl52W HobbZ4TuufGtFKN+z3iotInE9ek7PNkoTYEZnKJhk00yyBMdAC1odA5L1Qb/LoV3FuOm 8BWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fn89MqmC; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l8si3611592edt.265.2021.02.04.09.44.10; Thu, 04 Feb 2021 09:44:12 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Fn89MqmC; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238567AbhBDRnu (ORCPT + 13 others); Thu, 4 Feb 2021 12:43:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238587AbhBDRbP (ORCPT ); Thu, 4 Feb 2021 12:31:15 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1ADB3C0617AB for ; Thu, 4 Feb 2021 09:29:21 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id u14so3690676wmq.4 for ; Thu, 04 Feb 2021 09:29:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4ltiq3mCpsTPc6kiuj14O3FHIVaCpAzgI/dn2agPl+o=; b=Fn89MqmC90zf8D7DVeJwrNOhXSjtY2uZm9Y3mgGEBtOK9DKxSDTkmi9QwNbuIxsgzh Mmm1k3MFzhlbOhz1WSy3JAtms5IlDer9uhwEyWCFTt7CTDjU5WPBFD+whR75roMsvDHk +xPBUyTGJ3P2jZ1tdWZAegJBO2KFfDvz6E3h4Ga0VzQzEMO9Yb7KB4yEpbkuEYBSmZ3v 4UcWzhbfLRgJzCbMjMmsJ2dgoM+okD5TeicEiVNNGWTfZEgPNsg4+DWvtIbM+bK1OJyL uBUoaZTbDffqxRQ3hOy2DgOfO3Cr7yMnzADtagY9svTJNobnlHlUmu7RuJ3NMGFjI6OG pW1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4ltiq3mCpsTPc6kiuj14O3FHIVaCpAzgI/dn2agPl+o=; b=f4Ms8DOtLBTlY0EBfeXSHYU0HeV6wilGIkVlsnuzOrHYrb+dWI/2Hsg1CtoD9cBA7Q 2F//BCoL4uKV+bhrS4IJLgAQowkr5Qo3oVi1qO+Cp6+9ZFtLQwdo3omB93/FQvlhL67c AZeWlOHoWeU3s4J47wB6mgTdwtovMJW2I+3+BMEfjOvkvMES8M4Lf/Zn/O8ySeOSP+wS 1qG5LOAsK6+sZnSyWIpXY3noygl60YIefWk+qV5sbSr3hS7iZIA4jDv35CONth3zf7gl s5jax/5VSQcyvgQWnsUBIBW6eLifAtTWM7b+DmbDzNl5rsMXBaV/kZ4r28X8bUOoMg4m Wq8Q== X-Gm-Message-State: AOAM5328uvavldTT58La7GDGJ5ltRoPaaR8xpRss+rtWEdiKitv9Dad9 N8BiKkaa0+TO4M2OIU9TPpKYjzazBsNUbg== X-Received: by 2002:a05:600c:35d6:: with SMTP id r22mr244546wmq.44.1612459759426; Thu, 04 Feb 2021 09:29:19 -0800 (PST) Received: from dell.default ([91.110.221.188]) by smtp.gmail.com with ESMTPSA id j7sm9641334wrp.72.2021.02.04.09.29.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Feb 2021 09:29:18 -0800 (PST) From: Lee Jones To: stable@vger.kernel.org Cc: Thomas Gleixner , gzobqq@gmail.com, Peter Zijlstra , Lee Jones Subject: [PATCH 10/10] futex: Handle faults correctly for PI futexes Date: Thu, 4 Feb 2021 17:29:03 +0000 Message-Id: <20210204172903.2860981-11-lee.jones@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210204172903.2860981-1-lee.jones@linaro.org> References: <20210204172903.2860981-1-lee.jones@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Thomas Gleixner fixup_pi_state_owner() tries to ensure that the state of the rtmutex, pi_state and the user space value related to the PI futex are consistent before returning to user space. In case that the user space value update faults and the fault cannot be resolved by faulting the page in via fault_in_user_writeable() the function returns with -EFAULT and leaves the rtmutex and pi_state owner state inconsistent. A subsequent futex_unlock_pi() operates on the inconsistent pi_state and releases the rtmutex despite not owning it which can corrupt the RB tree of the rtmutex and cause a subsequent kernel stack use after free. It was suggested to loop forever in fixup_pi_state_owner() if the fault cannot be resolved, but that results in runaway tasks which is especially undesired when the problem happens due to a programming error and not due to malice. As the user space value cannot be fixed up, the proper solution is to make the rtmutex and the pi_state consistent so both have the same owner. This leaves the user space value out of sync. Any subsequent operation on the futex will fail because the 10th rule of PI futexes (pi_state owner and user space value are consistent) has been violated. As a consequence this removes the inept attempts of 'fixing' the situation in case that the current task owns the rtmutex when returning with an unresolvable fault by unlocking the rtmutex which left pi_state::owner and rtmutex::owner out of sync in a different and only slightly less dangerous way. Fixes: 1b7558e457ed ("futexes: fix fault handling in futex_lock_pi") Reported-by: gzobqq@gmail.com Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Cc: stable@vger.kernel.org Signed-off-by: Lee Jones --- kernel/futex.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) -- 2.25.1 diff --git a/kernel/futex.c b/kernel/futex.c index 8300870666638..199e63c5b6120 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -1012,7 +1012,8 @@ static void exit_pi_state_list(struct task_struct *curr) * FUTEX_OWNER_DIED bit. See [4] * * [10] There is no transient state which leaves owner and user space - * TID out of sync. + * TID out of sync. Except one error case where the kernel is denied + * write access to the user address, see fixup_pi_state_owner(). */ /* @@ -2357,6 +2358,24 @@ handle_fault: if (!err) goto retry; + /* + * fault_in_user_writeable() failed so user state is immutable. At + * best we can make the kernel state consistent but user state will + * be most likely hosed and any subsequent unlock operation will be + * rejected due to PI futex rule [10]. + * + * Ensure that the rtmutex owner is also the pi_state owner despite + * the user space value claiming something different. There is no + * point in unlocking the rtmutex if current is the owner as it + * would need to wait until the next waiter has taken the rtmutex + * to guarantee consistent state. Keep it simple. Userspace asked + * for this wreckaged state. + * + * The rtmutex has an owner - either current or some other + * task. See the EAGAIN loop above. + */ + pi_state_update_owner(pi_state, rt_mutex_owner(&pi_state->pi_mutex)); + return err; } @@ -2742,13 +2761,6 @@ retry_private: if (res) ret = (res < 0) ? res : 0; - /* - * If fixup_owner() faulted and was unable to handle the fault, unlock - * it and return the fault to userspace. - */ - if (ret && (rt_mutex_owner(&q.pi_state->pi_mutex) == current)) - rt_mutex_futex_unlock(&q.pi_state->pi_mutex); - /* Unqueue and drop the lock */ unqueue_me_pi(&q); @@ -3053,8 +3065,6 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, if (q.pi_state && (q.pi_state->owner != current)) { spin_lock(q.lock_ptr); ret = fixup_pi_state_owner(uaddr2, &q, current); - if (ret && rt_mutex_owner(&q.pi_state->pi_mutex) == current) - rt_mutex_futex_unlock(&q.pi_state->pi_mutex); /* * Drop the reference to the pi state which * the requeue_pi() code acquired for us. @@ -3091,14 +3101,6 @@ static int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, if (res) ret = (res < 0) ? res : 0; - /* - * If fixup_pi_state_owner() faulted and was unable to handle - * the fault, unlock the rt_mutex and return the fault to - * userspace. - */ - if (ret && rt_mutex_owner(pi_mutex) == current) - rt_mutex_futex_unlock(pi_mutex); - /* Unqueue and drop the lock. */ unqueue_me_pi(&q); }