From patchwork Mon Feb 19 14:55:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128816 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp3594533ljc; Mon, 19 Feb 2018 06:56:08 -0800 (PST) X-Google-Smtp-Source: AH8x224gLN4+1nkUHm0A+eHgUXPE7OfKqJmHec1fKkqfbXfhue0WRdZxLbNZupc04rfHQY8bPhBW X-Received: by 10.99.182.76 with SMTP id v12mr12309241pgt.158.1519052168722; Mon, 19 Feb 2018 06:56:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519052168; cv=none; d=google.com; s=arc-20160816; b=dxgLwZ6/j6IyzXyHDKaL341qc6cplEAfw1qPfeEBH1kLE5+EoYe8wUbXq9eeYgBqAO UyHVbAcLEKvXj2YvouXUF6Oh6YDz3XfxjxiVdaxCbR9CpmtiWs5Wi5ZJdzCwfiM/bCbl GBBnZeCXbKxg9Ru1fDmfclOFUDGsXxM7xv7EHnMBVfojKBpnlnCvIbMA8OVnYXpmX+Zn fdnhmA6DobNEXt/vUoiTMoXa8tqMIgjMleIt9xm3wbjVVCrK+gI9gQrIHSFf9c2WeOlI KT1Y73VP+JBLH4gT0lRdiYj1fvTX5MMi8OvdqsjfxvY+Q7MIvOhiSBS1CrcdRi40/JUb nn/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Xts7ebPL04mtUoh4JMHxdtFpRuJhoVp3OIau7gyvM5Y=; b=P8JP3U2vT7yLkkTk2CDDXPckRz1mdvEn1Uc6xLHVxBkiefnTSH9WFbnrCwdyE3e/P9 WxQx4M0iuq5dfggGn9WipsHI0lZNTSUrvkKCTlXwt6wMwHvJgXTs0fMG2502bgS/fxFf TRjokUeshRG4gAMfHG7HqffdnfWnDQFUVlihIxxmX2Uca4Ccxvf5MUd6I0rp9EgFPfXH n4QQvg8Ji4naJbvMblrNnGzi3CzMm/HaNz/vDJZVLN/RmR60kBPkJjpCAkWq+lEGBxxg W8RLTA1RKHmTCozEIdX1U4hBC00lSgnIRjCsxU+CP74KHoL2F8EtiQucppIQs9+yriuc mOKQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l1si1019651pga.408.2018.02.19.06.56.08; Mon, 19 Feb 2018 06:56:08 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753064AbeBSOzu (ORCPT + 28 others); Mon, 19 Feb 2018 09:55:50 -0500 Received: from foss.arm.com ([217.140.101.70]:60256 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753010AbeBSOzr (ORCPT ); Mon, 19 Feb 2018 09:55:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8669D1684; Mon, 19 Feb 2018 06:55:47 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5845F3F590; Mon, 19 Feb 2018 06:55:47 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 413B31AE4D02; Mon, 19 Feb 2018 14:55:56 +0000 (GMT) From: Will Deacon To: linux-fsdevel@vger.kernel.org Cc: peterz@infradead.org, viro@zeniv.linux.org.uk, willy@infradead.org, linux-kernel@vger.kernel.org, Will Deacon Subject: [PATCH v2 1/2] fs: dcache: Avoid livelock between d_alloc_parallel and __d_add Date: Mon, 19 Feb 2018 14:55:54 +0000 Message-Id: <1519052155-6361-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1519052155-6361-1-git-send-email-will.deacon@arm.com> References: <1519052155-6361-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If d_alloc_parallel runs concurrently with __d_add, it is possible for d_alloc_parallel to continuously retry whilst i_dir_seq has been incremented to an odd value by __d_add: CPU0: __d_add n = start_dir_add(dir); cmpxchg(&dir->i_dir_seq, n, n + 1) == n CPU1: d_alloc_parallel retry: seq = smp_load_acquire(&parent->d_inode->i_dir_seq) & ~1; hlist_bl_lock(b); bit_spin_lock(0, (unsigned long *)b); // Always succeeds CPU0: __d_lookup_done(dentry) hlist_bl_lock bit_spin_lock(0, (unsigned long *)b); // Never succeeds CPU1: if (unlikely(parent->d_inode->i_dir_seq != seq)) { hlist_bl_unlock(b); goto retry; } Since the simple bit_spin_lock used to implement hlist_bl_lock does not provide any fairness guarantees, then CPU1 can starve CPU0 of the lock and prevent it from reaching end_dir_add(dir), therefore CPU1 cannot exit its retry loop because the sequence number always has the bottom bit set. This patch resolves the livelock by not taking hlist_bl_lock in d_alloc_parallel if the sequence counter is odd, since any subsequent masked comparison with i_dir_seq will fail anyway. Cc: Peter Zijlstra Cc: Al Viro Reported-by: Naresh Madhusudana Acked-by: Peter Zijlstra (Intel) Reviewed-by: Matthew Wilcox Signed-off-by: Will Deacon --- fs/dcache.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) -- 2.1.4 diff --git a/fs/dcache.c b/fs/dcache.c index 7c38f39958bc..b243deec298c 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -2474,7 +2474,7 @@ struct dentry *d_alloc_parallel(struct dentry *parent, retry: rcu_read_lock(); - seq = smp_load_acquire(&parent->d_inode->i_dir_seq) & ~1; + seq = smp_load_acquire(&parent->d_inode->i_dir_seq); r_seq = read_seqbegin(&rename_lock); dentry = __d_lookup_rcu(parent, name, &d_seq); if (unlikely(dentry)) { @@ -2495,6 +2495,12 @@ struct dentry *d_alloc_parallel(struct dentry *parent, rcu_read_unlock(); goto retry; } + + if (unlikely(seq & 1)) { + rcu_read_unlock(); + goto retry; + } + hlist_bl_lock(b); if (unlikely(parent->d_inode->i_dir_seq != seq)) { hlist_bl_unlock(b); From patchwork Mon Feb 19 14:55:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128815 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp3594355ljc; Mon, 19 Feb 2018 06:55:55 -0800 (PST) X-Google-Smtp-Source: AH8x224XJsIva+ZW4XNxoPYbvV8KNDHPD8ayOx8hMxEOc6nL9WkCudDYIEGzsLX5isNs/9RV1XfH X-Received: by 10.98.108.65 with SMTP id h62mr6750421pfc.32.1519052155270; Mon, 19 Feb 2018 06:55:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519052155; cv=none; d=google.com; s=arc-20160816; b=B7WujDpQfEco38+xJrbIjCRZMgQeXPGoQrrF7JcRkAvzUbesVb31VvTNORgIYpz6PL NFyGeNkpigniWdNo0iA1sXecVqvSSQxM4jjpj6LSFBpFzvOO90mJKj2GJuAAS4goCpDo GTE4BKy/plun2C/CB9ghEScMq/U+mUCFffQI7pBPNu5E76beg5EYJmKAH0dDrY0xIZSt UVj78R4LpjsvdT41AbMABJADnpUi0tzopxbK4uc3/JscFvdFTtF4ZPxGWtZTYjomRRUW tp+dJPS4diJjrT1+PLaGPgGGFv2XloJYz9AYOLlOzcRIwpDVoNfhX6Jt6zfJktyO8d94 5d2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=a1V6BwQO4moREB4V8DdZsoaaYJ3c4dnM7bhD6UB+FKQ=; b=eGijF59kY55M7TVGtQhAfvMCfXlYkTN+g9o+pknIb49GX3Kv9qbcP8JyvtCfUjd8aU ZWVYiYZFTxGS+m6HqI/2Ls25XcKRYCFBBlZJYQ3rUpo5BcB2qoAwGSAnDvcevTkBFwGU u4SAvdEi7snaxz1bW+x6lEIagMhNYPiylby0jYEgT29sBWMZzoclufhQNJyb3dMBJLCH iZ2W7rVHrWM+CrvSA6WJ5v/QS8GC+NzUZr9bQcHntbycMchNqQPhOaisr3OEd8pdJwLW AaPEshNT6hNNaTnGwIMsuub77gp8XWRBkbMcB+lqqEXyt9qcS5ADi6Bjna2fOa4DW6Jc 7KOA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t5si2133699pgf.93.2018.02.19.06.55.55; Mon, 19 Feb 2018 06:55:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753091AbeBSOzw (ORCPT + 28 others); Mon, 19 Feb 2018 09:55:52 -0500 Received: from foss.arm.com ([217.140.101.70]:60258 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753011AbeBSOzr (ORCPT ); Mon, 19 Feb 2018 09:55:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 935511688; Mon, 19 Feb 2018 06:55:47 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 64FD13F7B8; Mon, 19 Feb 2018 06:55:47 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 4FBDC1AE5044; Mon, 19 Feb 2018 14:55:56 +0000 (GMT) From: Will Deacon To: linux-fsdevel@vger.kernel.org Cc: peterz@infradead.org, viro@zeniv.linux.org.uk, willy@infradead.org, linux-kernel@vger.kernel.org, Will Deacon Subject: [PATCH v2 2/2] fs: dcache: Use READ_ONCE when accessing i_dir_seq Date: Mon, 19 Feb 2018 14:55:55 +0000 Message-Id: <1519052155-6361-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1519052155-6361-1-git-send-email-will.deacon@arm.com> References: <1519052155-6361-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org i_dir_seq is subject to concurrent modification by a cmpxchg or store-release operation, so ensure that the relaxed access in d_alloc_parallel uses READ_ONCE. Reported-by: Peter Zijlstra Signed-off-by: Will Deacon --- fs/dcache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.1.4 diff --git a/fs/dcache.c b/fs/dcache.c index b243deec298c..5716ab04e1db 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -2502,7 +2502,7 @@ struct dentry *d_alloc_parallel(struct dentry *parent, } hlist_bl_lock(b); - if (unlikely(parent->d_inode->i_dir_seq != seq)) { + if (unlikely(READ_ONCE(parent->d_inode->i_dir_seq) != seq)) { hlist_bl_unlock(b); rcu_read_unlock(); goto retry;