From patchwork Tue Jan 26 11:17:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C972C433E9 for ; Tue, 26 Jan 2021 11:23:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E26312310E for ; Tue, 26 Jan 2021 11:23:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392170AbhAZLWt (ORCPT ); Tue, 26 Jan 2021 06:22:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392297AbhAZLVl (ORCPT ); Tue, 26 Jan 2021 06:21:41 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CA96C06174A for ; Tue, 26 Jan 2021 03:21:01 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id dj23so19267830edb.13 for ; Tue, 26 Jan 2021 03:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M5TPvRyw0cGrL3K0zvfB1Iff9KwIVNzXAQTKSicxMQI=; b=D0tYRZ0t+Rj7bh7z0RNcCEqd1cQi7FnPY/vUqGJVq9sr0a6NwIymffvKgfGGJ0Wdr4 zp4tzcF16cC10FJKuZBYC9+seQEuyuSisfIsM6a5qT8IpNyS8LvrMt3EiNID/tzrG8Bc 08OayJY+A1y7U1LUBGD+iFAIyAtzjQw6UTEmMEHDO0YNHgHuxdIkh5XtO5lOkUD4CSOR /oqQQQG7vaHLA8VfRVZR0+QZqCz4e5qkvcjIgdmnPY7PuufTOSEb5bnX/c8x75TGlgCh tUgF7RknCP6OAp08bescpUCDg7sWZky24zO1FuCQ80T3vAnDZFMPUhtbMhkNptBaG+0h JofQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M5TPvRyw0cGrL3K0zvfB1Iff9KwIVNzXAQTKSicxMQI=; b=kG0FQQw8Hndke+X4HG6fKkoxpNwRnTxdKcSl5ppkXyMKL0eVLSkBIileePB7QcY7Bu 8Lf8VQpKQyr8CRzCep1Rv9EBNu7Qwiw5ivkIHX8DRwsXDy1tkQ+DOmQsVxE8YDow+y7R NB/nVOac8QUTrta1Zug1QnlBcpxpGalpAhi45GPL8SS9x6r8REI+ttON3odCEiT/dh6T W1T7y6KgBNxUkSb0+hBEzGNlnFisvH6csIrxgNfm0dExcravyXT3S+J+iYKEp9YncqXC HHaOD6b9Z43231G8Xi+Dr2P8Kg7gcm+vLzjAg4O8p2JJJxhot95QLy46qWMERe6Buan1 mMhA== X-Gm-Message-State: AOAM530fDjH0OjPMOfe5uQ4HCLP9OK6LOW2FEb0WqdX+mjgUFr+PAOi3 E7oWwHtpKXuUyjSfHj/2CiwMtH2KdGfGtw== X-Google-Smtp-Source: ABdhPJyRn11bhk3W/k3x195QcHM5YPHZ1dF1qXDJ+KCmLUBsn76an91kmxXzL5S5ewNNIfWUhsapng== X-Received: by 2002:a05:6402:1a2a:: with SMTP id be10mr4144669edb.185.1611660059554; Tue, 26 Jan 2021 03:20:59 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.20.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:20:59 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe Subject: [PATCH stable 01/11] kernel/io_uring: cancel io_uring before task works Date: Tue, 26 Jan 2021 11:17:00 +0000 Message-Id: <96a68f8f062a7bc6e267fef65e01a665ab232a29.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit b1b6b5a30dce872f500dc43f067cba8e7f86fc7d ] For cancelling io_uring requests it needs either to be able to run currently enqueued task_works or having it shut down by that moment. Otherwise io_uring_cancel_files() may be waiting for requests that won't ever complete. Go with the first way and do cancellations before setting PF_EXITING and so before putting the task_work infrastructure into a transition state where task_work_run() would better not be called. Cc: stable@vger.kernel.org # 5.5+ Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/file.c | 2 -- kernel/exit.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/file.c b/fs/file.c index 4559b5fec3bd..21c0893f2f1d 100644 --- a/fs/file.c +++ b/fs/file.c @@ -21,7 +21,6 @@ #include #include #include -#include unsigned int sysctl_nr_open __read_mostly = 1024*1024; unsigned int sysctl_nr_open_min = BITS_PER_LONG; @@ -453,7 +452,6 @@ void exit_files(struct task_struct *tsk) struct files_struct * files = tsk->files; if (files) { - io_uring_files_cancel(files); task_lock(tsk); tsk->files = NULL; task_unlock(tsk); diff --git a/kernel/exit.c b/kernel/exit.c index 1f236ed375f8..d13d67fc5f4e 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -762,6 +763,7 @@ void __noreturn do_exit(long code) schedule(); } + io_uring_files_cancel(tsk->files); exit_signals(tsk); /* sets PF_EXITING */ /* sync mm's RSS info before statistics gathering */ From patchwork Tue Jan 26 11:17:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 372444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B772C433DB for ; Tue, 26 Jan 2021 11:22:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FE0523108 for ; Tue, 26 Jan 2021 11:22:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392213AbhAZLW0 (ORCPT ); Tue, 26 Jan 2021 06:22:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392300AbhAZLVm (ORCPT ); Tue, 26 Jan 2021 06:21:42 -0500 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2717C061756 for ; Tue, 26 Jan 2021 03:21:01 -0800 (PST) Received: by mail-ej1-x62d.google.com with SMTP id hs11so22442687ejc.1 for ; Tue, 26 Jan 2021 03:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Bo2KvLyJ3CoRAV90XXAzC6MvE67j5C0xtYYmI4wQQFc=; b=J924yGnWRNUC6Q++owCw2rGPr8JWM0Zn+bH/ktGEjr0VOPu4707+29prxhGar/haJE A7MaAJPpyhD/2UvA73DTA7WL7VOwnScMQ0zDuRzYghYJ2XC5e/stGDA30Y7/6V1INmsK RM8gaDaRLDMph+cvZPxXVaMxk+E0PCEcX8DG4t0qjOrw+pJWzVNvatvlRVyuCE0jGBIZ 6Gz69UgLBJlD01Dca3X2NbZLv3djD7kXNrK1ZJrWsVUJIpTvECpvjD5NsVY5vdGZTwsr Scw5QA6iN0N3/rsrgP8tSSVSONKeKVn3Jid/KVqJW6D+z54fI/etV9CyoxoIScldHPje eQ7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Bo2KvLyJ3CoRAV90XXAzC6MvE67j5C0xtYYmI4wQQFc=; b=Hx32hsg2tMD7+SLi4GzbVrv4biN6Ny9PMP5+VFjrux4KjiTz5nKv5HwaMQUtCefPbg IXnYuEMhC8vxs0wiR0hZX+LPJCW9ZywN5Ubp9hzcC9dVw9Y7wm5nOA2Z+GbC9HxpyIY5 ynIQZjB+/pVwc4yNpvfo5sJBVBshKAk7SGsw35ZVm8cbQ+0DrigTzkUM5pV8PDbv5/Fq b4n2yKOdL4CHtLscjAaUFd14oaLWuW6fJuCN/46ikbZWHFIxdwbkeFOK9o2YxE1CpL7T 5o0A0Iya593AxcybSXVtVvmG+emNecOjsLa74pAOq0drYggS9t9ha5Z+zPLbkdBs/f8+ d72A== X-Gm-Message-State: AOAM5339TNnTdTMKaqJdraXFodeetdpLqnisgvtdVXsQcVlF22S2HKbg TUBaijS3oU4MumpXWO+XmhpHisJmdms/5w== X-Google-Smtp-Source: ABdhPJwS8KrazKTQ7mhoeL+GjbnPsuXuz4xXmgVhRzCTJuh9Pl/zWLNy2FbfdcPUoAKbbXYT5ozdOg== X-Received: by 2002:a17:906:2a42:: with SMTP id k2mr3101669eje.118.1611660060415; Tue, 26 Jan 2021 03:21:00 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:00 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe Subject: [PATCH stable 02/11] io_uring: inline io_uring_attempt_task_drop() Date: Tue, 26 Jan 2021 11:17:01 +0000 Message-Id: <706c41d54e42eff8de3f2d1741cead614c9b454b.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 4f793dc40bc605b97624fd36baf085b3c35e8bfd ] A simple preparation change inlining io_uring_attempt_task_drop() into io_uring_flush(). Cc: stable@vger.kernel.org # 5.5+ Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 265aea2cd7bc..6c89d38076d0 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8804,23 +8804,6 @@ static void io_uring_del_task_file(struct file *file) fput(file); } -/* - * Drop task note for this file if we're the only ones that hold it after - * pending fput() - */ -static void io_uring_attempt_task_drop(struct file *file) -{ - if (!current->io_uring) - return; - /* - * fput() is pending, will be 2 if the only other ref is our potential - * task file note. If the task is exiting, drop regardless of count. - */ - if (fatal_signal_pending(current) || (current->flags & PF_EXITING) || - atomic_long_read(&file->f_count) == 2) - io_uring_del_task_file(file); -} - static void io_uring_remove_task_files(struct io_uring_task *tctx) { struct file *file; @@ -8912,7 +8895,17 @@ void __io_uring_task_cancel(void) static int io_uring_flush(struct file *file, void *data) { - io_uring_attempt_task_drop(file); + if (!current->io_uring) + return 0; + + /* + * fput() is pending, will be 2 if the only other ref is our potential + * task file note. If the task is exiting, drop regardless of count. + */ + if (fatal_signal_pending(current) || (current->flags & PF_EXITING) || + atomic_long_read(&file->f_count) == 2) + io_uring_del_task_file(file); + return 0; } From patchwork Tue Jan 26 11:17:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 372443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07287C433DB for ; Tue, 26 Jan 2021 11:23:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9B332310C for ; Tue, 26 Jan 2021 11:23:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392304AbhAZLWp (ORCPT ); Tue, 26 Jan 2021 06:22:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392303AbhAZLVn (ORCPT ); Tue, 26 Jan 2021 06:21:43 -0500 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4612C0613D6 for ; Tue, 26 Jan 2021 03:21:02 -0800 (PST) Received: by mail-ed1-x52f.google.com with SMTP id d2so15725162edz.3 for ; Tue, 26 Jan 2021 03:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=830NVUoeZpi1YUm0xNN/ayO3qApi6qcIRadh5RBk/94=; b=uagH8EqE+e5flLeWpMxEnhMLqgd7P3Zlf6YTIi2AOCTdZxwWfQKQ4deZBwSFZ/tsRh RmDtWSptHSMmTXf8K/M1e9oAew+XotppTtmYMPAWFSbB2dhzE12aj9wyTeUsClzuVeoN mgtBt1ILOERbh1UwSoJiP8Ka20cJFvDsrheT9wLzwzEWlBG3YriX1rlHoP2z4y9K0e2+ L8699M0Sny2F8wWXHe0W6GYWnKOFjpQaFekDbX0a184nUumWyKDpFVfLdPAWPtRueHfV /mbuUzBmkbWjg+RdexmqfNhxST95Z/yPRTj3a5nz8hT9l15l0a7+eabJcwNFydP+TYWb 3/Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=830NVUoeZpi1YUm0xNN/ayO3qApi6qcIRadh5RBk/94=; b=qSNByOYM36UWuR25PtRpOtAtaqSk5AoEoA8DCbzdyoJ2keuTBILOrj7Iy6NvLLIxg7 6dGxOxbCC9IN4Z2Dg4rY+U4zroYhc8ZnKR0JwFOT1QHJwwq3gmAkYFQSyZfpu15imB4p c7XoUzlTs0ogORkTquRjHjKMdHIQ20yCPsLVzvaCYwDwxIl5YrBahvxCrX5RQEzc+qah zpaUdjRKaco2a7ryqSFkeMK8tt5OVcjgqd9FVe+MSjIMK74oTwHPDdf47TyKjG/34lOh zpq/oaaCIZKBoSGPxLWnkJ8QC63QJ03GX3J5vTw4jcQmujwQWz/or+XmXmvMTzoaRqqZ jd1g== X-Gm-Message-State: AOAM530Az9sq+SGl4oDxJQ07G29uxaBmzejj08phgHhKebOfkXraiE47 mAd4eGRvF2eDbG4WYOrUIX+C/3/ZXxlS4A== X-Google-Smtp-Source: ABdhPJzVBvUnQc8Wwm0h35+A/WgT7l2T7u6PKD1qchyKs7MzIuvsXq40xKD/PT9xz/WWyWjOnED6wg== X-Received: by 2002:a05:6402:34c3:: with SMTP id w3mr4255432edc.3.1611660061275; Tue, 26 Jan 2021 03:21:01 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:00 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe Subject: [PATCH stable 03/11] io_uring: add warn_once for io_uring_flush() Date: Tue, 26 Jan 2021 11:17:02 +0000 Message-Id: <1abdd0e576ae991c6ab04bebd20360ea2b3a175b.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 6b5733eb638b7068ab7cb34e663b55a1d1892d85] files_cancel() should cancel all relevant requests and drop file notes, so we should never have file notes after that, including on-exit fput and flush. Add a WARN_ONCE to be sure. Cc: stable@vger.kernel.org # 5.5+ Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 6c89d38076d0..4dfba3d44a3c 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8895,17 +8895,23 @@ void __io_uring_task_cancel(void) static int io_uring_flush(struct file *file, void *data) { - if (!current->io_uring) + struct io_uring_task *tctx = current->io_uring; + + if (!tctx) return 0; + /* we should have cancelled and erased it before PF_EXITING */ + WARN_ON_ONCE((current->flags & PF_EXITING) && + xa_load(&tctx->xa, (unsigned long)file)); + /* * fput() is pending, will be 2 if the only other ref is our potential * task file note. If the task is exiting, drop regardless of count. */ - if (fatal_signal_pending(current) || (current->flags & PF_EXITING) || - atomic_long_read(&file->f_count) == 2) - io_uring_del_task_file(file); + if (atomic_long_read(&file->f_count) != 2) + return 0; + io_uring_del_task_file(file); return 0; } From patchwork Tue Jan 26 11:17:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E05B4C433E6 for ; Tue, 26 Jan 2021 11:23:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEBC823104 for ; Tue, 26 Jan 2021 11:23:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392330AbhAZLWn (ORCPT ); Tue, 26 Jan 2021 06:22:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392304AbhAZLVo (ORCPT ); Tue, 26 Jan 2021 06:21:44 -0500 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B66E5C0613ED for ; Tue, 26 Jan 2021 03:21:03 -0800 (PST) Received: by mail-ed1-x52b.google.com with SMTP id bx12so19260145edb.8 for ; Tue, 26 Jan 2021 03:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7/1fbzJNHjtwlf7qCbLHZQFGUrVOA3+SGTkQr7SgJH0=; b=TQCbK9KRzVURYJdyiTHTOFPuk5nC51Bph7wARDKZMo85/kp0ZV64O2hxZG6cxRs7Z5 ucMTjVbsbH15fsDbjQzsui76ExJN94UU6PsQIaVwV2NeoR1fKz5k4rXNEhATnTliNFk4 JkFeu5dgUU0jG03kRNKv4t1gwmPnZ6fV9kPLg+UkfXXVlBMWIbmp7TF7e4w8e2N8Drya Q4s1G8TztRaTTjr49x1asb7bcEtO+xw/6dDiA0qSr5urbnrL993l5Om87ziBLaH2f1uv AnIWfNVn9wYcO35puvlbItY+3oFtXPWA1zPQsrbQQT2zC550Zma9c2mXrhg9gRuMHaBr dOlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7/1fbzJNHjtwlf7qCbLHZQFGUrVOA3+SGTkQr7SgJH0=; b=jkhlZtbSyiVasz9KdNCgEIUPK7u0TycLi7DHozjuu42GCt90pJIDQTQLNA5SOUj0zF KBn1kXruOzSCsCJohwtwODyRuTCTpL6pYAkhm9xjh0JpP/JJsLKdAQk7F5GUM11zQyMw GSGxmTX6BFCzAIPJQRIcZQwp9GD9ep74QtsnJ0dOgC/jT9lQFP3kovWoDi1IybVNP4if lUUW5ODKuniTSyRvbL+2DEEaSfhv+EEwZGlqQaS/FHKAb37UrpPOfEIhZk5iU+zS70J1 +xShu/fChwZa0gOR8fhc26xCHNnF5wSqpmRmXp4hUiTCI3Om4fz4kuJwU4U1VPLDAzsf ++Hg== X-Gm-Message-State: AOAM533tj5ECDu7kIaW0yT11Qf5ul1Swcjg/Z//REAJcXEmuW9YFyYSa yxejoyRQIDCOpoEjFhgl/YS28LMpHsIEEw== X-Google-Smtp-Source: ABdhPJyUozs0/b4Sj/4sFeYd3Y9W23s6exl9AGUNx1pN5BwzrZjroLLtV/SOcOiArW8y5u0tKsiRJA== X-Received: by 2002:a05:6402:1398:: with SMTP id b24mr4005952edv.108.1611660062183; Tue, 26 Jan 2021 03:21:02 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:01 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe Subject: [PATCH stable 04/11] io_uring: stop SQPOLL submit on creator's death Date: Tue, 26 Jan 2021 11:17:03 +0000 Message-Id: X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c ] When the creator of SQPOLL io_uring dies (i.e. sqo_task), we don't want its internals like ->files and ->mm to be poked by the SQPOLL task, it have never been nice and recently got racy. That can happen when the owner undergoes destruction and SQPOLL tasks tries to submit new requests in parallel, and so calls io_sq_thread_acquire*(). That patch halts SQPOLL submissions when sqo_task dies by introducing sqo_dead flag. Once set, the SQPOLL task must not do any submission, which is synchronised by uring_lock as well as the new flag. The tricky part is to make sure that disabling always happens, that means either the ring is discovered by creator's do_exit() -> cancel, or if the final close() happens before it's done by the creator. The last is guaranteed by the fact that for SQPOLL the creator task and only it holds exactly one file note, so either it pins up to do_exit() or removed by the creator on the final put in flush. (see comments in uring_flush() around file->f_count == 2). One more place that can trigger io_sq_thread_acquire_*() is __io_req_task_submit(). Shoot off requests on sqo_dead there, even though actually we don't need to. That's because cancellation of sqo_task should wait for the request before going any further. note 1: io_disable_sqo_submit() does io_ring_set_wakeup_flag() so the caller would enter the ring to get an error, but it still doesn't guarantee that the flag won't be cleared. note 2: if final __userspace__ close happens not from the creator task, the file note will pin the ring until the task dies. Cc: stable@vger.kernel.org # 5.5+ Fixed: b1b6b5a30dce8 ("kernel/io_uring: cancel io_uring before task works") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 58 ++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 50 insertions(+), 8 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 4dfba3d44a3c..723e1eb5349a 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -260,6 +260,7 @@ struct io_ring_ctx { unsigned int drain_next: 1; unsigned int eventfd_async: 1; unsigned int restricted: 1; + unsigned int sqo_dead: 1; /* * Ring buffer of indices into array of io_uring_sqe, which is @@ -2063,11 +2064,9 @@ static void io_req_task_cancel(struct callback_head *cb) static void __io_req_task_submit(struct io_kiocb *req) { struct io_ring_ctx *ctx = req->ctx; - bool fail; - fail = __io_sq_thread_acquire_mm(ctx); mutex_lock(&ctx->uring_lock); - if (!fail) + if (!ctx->sqo_dead && !__io_sq_thread_acquire_mm(ctx)) __io_queue_sqe(req, NULL); else __io_req_task_cancel(req, -EFAULT); @@ -6765,7 +6764,7 @@ static enum sq_ret __io_sq_thread(struct io_ring_ctx *ctx, to_submit = 8; mutex_lock(&ctx->uring_lock); - if (likely(!percpu_ref_is_dying(&ctx->refs))) + if (likely(!percpu_ref_is_dying(&ctx->refs) && !ctx->sqo_dead)) ret = io_submit_sqes(ctx, to_submit); mutex_unlock(&ctx->uring_lock); @@ -8456,6 +8455,10 @@ static void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx) mutex_lock(&ctx->uring_lock); percpu_ref_kill(&ctx->refs); /* if force is set, the ring is going away. always drop after that */ + + if (WARN_ON_ONCE((ctx->flags & IORING_SETUP_SQPOLL) && !ctx->sqo_dead)) + ctx->sqo_dead = 1; + ctx->cq_overflow_flushed = 1; if (ctx->rings) __io_cqring_overflow_flush(ctx, true, NULL, NULL); @@ -8714,6 +8717,18 @@ static bool __io_uring_cancel_task_requests(struct io_ring_ctx *ctx, return ret; } +static void io_disable_sqo_submit(struct io_ring_ctx *ctx) +{ + WARN_ON_ONCE(ctx->sqo_task != current); + + mutex_lock(&ctx->uring_lock); + ctx->sqo_dead = 1; + mutex_unlock(&ctx->uring_lock); + + /* make sure callers enter the ring to get error */ + io_ring_set_wakeup_flag(ctx); +} + /* * We need to iteratively cancel requests, in case a request has dependent * hard links. These persist even for failure of cancelations, hence keep @@ -8725,6 +8740,8 @@ static void io_uring_cancel_task_requests(struct io_ring_ctx *ctx, struct task_struct *task = current; if ((ctx->flags & IORING_SETUP_SQPOLL) && ctx->sq_data) { + /* for SQPOLL only sqo_task has task notes */ + io_disable_sqo_submit(ctx); task = ctx->sq_data->thread; atomic_inc(&task->io_uring->in_idle); io_sq_thread_park(ctx->sq_data); @@ -8896,6 +8913,7 @@ void __io_uring_task_cancel(void) static int io_uring_flush(struct file *file, void *data) { struct io_uring_task *tctx = current->io_uring; + struct io_ring_ctx *ctx = file->private_data; if (!tctx) return 0; @@ -8911,7 +8929,16 @@ static int io_uring_flush(struct file *file, void *data) if (atomic_long_read(&file->f_count) != 2) return 0; - io_uring_del_task_file(file); + if (ctx->flags & IORING_SETUP_SQPOLL) { + /* there is only one file note, which is owned by sqo_task */ + WARN_ON_ONCE((ctx->sqo_task == current) == + !xa_load(&tctx->xa, (unsigned long)file)); + + io_disable_sqo_submit(ctx); + } + + if (!(ctx->flags & IORING_SETUP_SQPOLL) || ctx->sqo_task == current) + io_uring_del_task_file(file); return 0; } @@ -8985,8 +9012,9 @@ static unsigned long io_uring_nommu_get_unmapped_area(struct file *file, #endif /* !CONFIG_MMU */ -static void io_sqpoll_wait_sq(struct io_ring_ctx *ctx) +static int io_sqpoll_wait_sq(struct io_ring_ctx *ctx) { + int ret = 0; DEFINE_WAIT(wait); do { @@ -8995,6 +9023,11 @@ static void io_sqpoll_wait_sq(struct io_ring_ctx *ctx) prepare_to_wait(&ctx->sqo_sq_wait, &wait, TASK_INTERRUPTIBLE); + if (unlikely(ctx->sqo_dead)) { + ret = -EOWNERDEAD; + goto out; + } + if (!io_sqring_full(ctx)) break; @@ -9002,6 +9035,8 @@ static void io_sqpoll_wait_sq(struct io_ring_ctx *ctx) } while (!signal_pending(current)); finish_wait(&ctx->sqo_sq_wait, &wait); +out: + return ret; } SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, @@ -9045,10 +9080,16 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, if (ctx->flags & IORING_SETUP_SQPOLL) { io_cqring_overflow_flush(ctx, false, NULL, NULL); + ret = -EOWNERDEAD; + if (unlikely(ctx->sqo_dead)) + goto out; if (flags & IORING_ENTER_SQ_WAKEUP) wake_up(&ctx->sq_data->wait); - if (flags & IORING_ENTER_SQ_WAIT) - io_sqpoll_wait_sq(ctx); + if (flags & IORING_ENTER_SQ_WAIT) { + ret = io_sqpoll_wait_sq(ctx); + if (ret) + goto out; + } submitted = to_submit; } else if (to_submit) { ret = io_uring_add_task_file(ctx, f.file); @@ -9467,6 +9508,7 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, trace_io_uring_create(ret, ctx, p->sq_entries, p->cq_entries, p->flags); return ret; err: + io_disable_sqo_submit(ctx); io_ring_ctx_wait_and_kill(ctx); return ret; } From patchwork Tue Jan 26 11:17:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93FD1C433E6 for ; Tue, 26 Jan 2021 12:07:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B4DD2311F for ; Tue, 26 Jan 2021 12:07:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392220AbhAZMGR (ORCPT ); Tue, 26 Jan 2021 07:06:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392338AbhAZLW6 (ORCPT ); Tue, 26 Jan 2021 06:22:58 -0500 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86A1FC061786 for ; Tue, 26 Jan 2021 03:21:04 -0800 (PST) Received: by mail-ej1-x629.google.com with SMTP id a10so22414811ejg.10 for ; Tue, 26 Jan 2021 03:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=suQfLER1J7hWgKba7DAAjmsln0J3BgW+wjZ6YaOdEvs=; b=oc3pOW2vOm+5zQ3X6kVaTwwj1CZvObGy87BZf4qDIMnbq1/E4P1xG7/o/t/EClp8fQ N63ccgibgCoPUyqYMSvyY1dh2Tub1HSBwUZEJ6crZtL9jOKc21jSScQGOYcjpugqsunc k0DVSm4lpWii9H3sQNYa3xHwfjI5KTYChyb/MWk+TEaraEFXdYXniXNHPUlK/615MjAd AapDmDaRjNkZXB5oaYhM7hL181aDlPGr0Ovr3OkmjOrHZZk9Hmo8bE9h0LM+eF0AQd9M TbrHCHPMv1BKqVIom71nqkOixOjGf/ONXYIwZUrXtFa72V902/VikcapP1Hgo44iNnZ0 LKqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=suQfLER1J7hWgKba7DAAjmsln0J3BgW+wjZ6YaOdEvs=; b=CM/Y1B7eDuzDgsG2EUt1HHzqbAcz+9sK/NcFUp1aCUnH7a/6VeRAe2cFUR70cUYO4V Z3dFGBpkfVd0vI7YtSYSutmwKrJSBuYTV53zEUFi0BkbHZN4k4fKF7siOCAyVMUnItlO Ac2GiBqAUCVZ35n5r/lKHkKvaw4iyYQiJqfLVSxnCBy2SF6sWUkL5GkhtNeNF5bgd3+z +tI88pGUwsvexMJdnqQjK93Cb2ia+1RtSpVH+1udE2BImd/JSjTuWhvEw1UBbrcfBraB 7BJsd9NuqX/RNe1OAi9rc/WC+o4EZBU3ahNXa4QTO7vTx9SV+cxJ/gTOuNprI/8v4uD8 1bQw== X-Gm-Message-State: AOAM532JcCQqcspOMAvOHCx4PEHjRbHTiIAAeWJduVs5QhN7ADb9rrCB GkN0CU981j9J30SEELtTEEIsaPSnFTjM7g== X-Google-Smtp-Source: ABdhPJy5DbZqN3SYkdEdjL2bh9WRDbGn/yAddWJSzkheVJ1wW4+H1Sd4TjRayFmBJUU3jBmtguy7sw== X-Received: by 2002:a17:906:d0c1:: with SMTP id bq1mr3060941ejb.202.1611660063132; Tue, 26 Jan 2021 03:21:03 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:02 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , syzbot+ab412638aeb652ded540@syzkaller.appspotmail.com Subject: [PATCH stable 05/11] io_uring: fix null-deref in io_disable_sqo_submit Date: Tue, 26 Jan 2021 11:17:04 +0000 Message-Id: X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit b4411616c26f26c4017b8fa4d3538b1a02028733 ] general protection fault, probably for non-canonical address 0xdffffc0000000022: 0000 [#1] KASAN: null-ptr-deref in range [0x0000000000000110-0x0000000000000117] RIP: 0010:io_ring_set_wakeup_flag fs/io_uring.c:6929 [inline] RIP: 0010:io_disable_sqo_submit+0xdb/0x130 fs/io_uring.c:8891 Call Trace: io_uring_create fs/io_uring.c:9711 [inline] io_uring_setup+0x12b1/0x38e0 fs/io_uring.c:9739 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 io_disable_sqo_submit() might be called before user rings were allocated, don't do io_ring_set_wakeup_flag() in those cases. Cc: stable@vger.kernel.org # 5.5+ Reported-by: syzbot+ab412638aeb652ded540@syzkaller.appspotmail.com Fixes: d9d05217cb69 ("io_uring: stop SQPOLL submit on creator's death") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 723e1eb5349a..f1f1de815755 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8726,7 +8726,8 @@ static void io_disable_sqo_submit(struct io_ring_ctx *ctx) mutex_unlock(&ctx->uring_lock); /* make sure callers enter the ring to get error */ - io_ring_set_wakeup_flag(ctx); + if (ctx->rings) + io_ring_set_wakeup_flag(ctx); } /* From patchwork Tue Jan 26 11:17:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 372439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F770C433E9 for ; Tue, 26 Jan 2021 12:07:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA9B72311F for ; Tue, 26 Jan 2021 12:07:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392343AbhAZMHU (ORCPT ); Tue, 26 Jan 2021 07:07:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392337AbhAZLW4 (ORCPT ); Tue, 26 Jan 2021 06:22:56 -0500 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C842C061788 for ; Tue, 26 Jan 2021 03:21:05 -0800 (PST) Received: by mail-ej1-x62b.google.com with SMTP id w1so22361302ejf.11 for ; Tue, 26 Jan 2021 03:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zY7eyzui5lrsV2Pla8jrRefVhNic/1zSoUc/C9YVCgo=; b=H0ynR2cTeva2XzhMbn0gwM2WFfwJKAoWgrfEWVdjSBRNZqWtqgx8C0iVi0a4U1eC3t FAoTK/UJbWtzPEGofVExhqAFzB28vjg3vriGoWyf2bqe38b3EEKwnkW7nFZpiJbOsoxj /HwcxaXcG1IxqSZkkL4VPs5eLhd1Cp0Wv4CAE40z/p1iUxZ2O4L9lZyGs7+t5RB/RcYA aTeMLU/S0xG0a7OV5OuNIfjTIvMJz2hU6ht93HB01yWvzRkW7IAFQ2kDAT0wgW2d6ZeX 9i2I6IWynR5evunlMfCoKAC7xmUFPKuwAZv4D5Y/fPuveVcAqf+EoCOStALHfKEE3teN jqMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zY7eyzui5lrsV2Pla8jrRefVhNic/1zSoUc/C9YVCgo=; b=DMuPGDi9sWmgepcu+cly8jYmEMerIbABy9wUdk9wNqkuS6QoOxAPMpaeQtC+Wum+bc 7p6LNdypr+KsN+GyrwPLC5BlyDROH6226D+VPcFCqXixu2SAWqK6bftMYmQrPRDucAtV Rs31/1fNuTAEUIrcsk8KcC/JVkL97wZAKqVBhEE792PiTsQQt/3RcdfEPZQtUhnPe3Rf dgPDpti+vcu79WEhpC9muhEriPnJDJ54nXypapwuiNJBUCS/pjlz/771cB+1JSqsL7iU RjswVqcEcrGKvN1fAbNNRmigCE3Kkr/vUzFzsbtUvLbHvt/1ROQrNoS74xoDeRf6AU5y OW2Q== X-Gm-Message-State: AOAM530cH+x4jyktMfzAgXYbpQudZM4PpihsAb1GA9aT+gjT3dy8KtAW PwuHE/rc5/TQwiaO8nEvmVF1h6QWnBH0XQ== X-Google-Smtp-Source: ABdhPJymDFUozw5ybFdh9uTrQRXEHZKRzbxc9RhmlAKCDBt4A2x3uyT/gwIp1iIkMq77Um3LiclkxA== X-Received: by 2002:a17:906:1a0c:: with SMTP id i12mr3262512ejf.325.1611660064020; Tue, 26 Jan 2021 03:21:04 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:03 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , syzbot+9c9c35374c0ecac06516@syzkaller.appspotmail.com Subject: [PATCH stable 06/11] io_uring: do sqo disable on install_fd error Date: Tue, 26 Jan 2021 11:17:05 +0000 Message-Id: X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 06585c497b55045ec21aa8128e340f6a6587351c ] WARNING: CPU: 0 PID: 8494 at fs/io_uring.c:8717 io_ring_ctx_wait_and_kill+0x4f2/0x600 fs/io_uring.c:8717 Call Trace: io_uring_release+0x3e/0x50 fs/io_uring.c:8759 __fput+0x283/0x920 fs/file_table.c:280 task_work_run+0xdd/0x190 kernel/task_work.c:140 tracehook_notify_resume include/linux/tracehook.h:189 [inline] exit_to_user_mode_loop kernel/entry/common.c:174 [inline] exit_to_user_mode_prepare+0x249/0x250 kernel/entry/common.c:201 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline] syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302 entry_SYSCALL_64_after_hwframe+0x44/0xa9 failed io_uring_install_fd() is a special case, we don't do io_ring_ctx_wait_and_kill() directly but defer it to fput, though still need to io_disable_sqo_submit() before. note: it doesn't fix any real problem, just a warning. That's because sqring won't be available to the userspace in this case and so SQPOLL won't submit anything. Cc: stable@vger.kernel.org # 5.5+ Reported-by: syzbot+9c9c35374c0ecac06516@syzkaller.appspotmail.com Fixes: d9d05217cb69 ("io_uring: stop SQPOLL submit on creator's death") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index f1f1de815755..2acea64656f3 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -9501,6 +9501,7 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, */ ret = io_uring_install_fd(ctx, file); if (ret < 0) { + io_disable_sqo_submit(ctx); /* fput will clean it up */ fput(file); return ret; From patchwork Tue Jan 26 11:17:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 757E8C433E0 for ; Tue, 26 Jan 2021 12:07:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4566C2311F for ; Tue, 26 Jan 2021 12:07:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392337AbhAZMHV (ORCPT ); Tue, 26 Jan 2021 07:07:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392339AbhAZLW4 (ORCPT ); Tue, 26 Jan 2021 06:22:56 -0500 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E38BC06178A for ; Tue, 26 Jan 2021 03:21:06 -0800 (PST) Received: by mail-ej1-x636.google.com with SMTP id g12so22421174ejf.8 for ; Tue, 26 Jan 2021 03:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=S7JZ22JSG95h1zepz/8nX5FiPVxTaZFLuEXAwhmItbY=; b=t2z76xzqM47hgdXfHe+t3b7UlXrtKhfWJ1u5l5kn5+eO29Zp5BoG0TgV0ngPWtPLrG HEzWfHYcgLWRNkpwy8L6FBOedso/FbhqDvIlEuLI4paguzSz2Ey8MDf2dngXyhj4grlY pkA6vgdZmfiiO/FD4lHlh9ihSdIkiNA6MMVtiq7UU8eN4dofyVQKX8WTWPnwO3JoNbp0 cOTBWk+fJTHhCgApUN99zRMYWLxm6i9icXbJNlJpS7KlTl9vTLA16y5THNTXHsFOMJ7b lQ/F1pn6d4Bon4IM/VZoDXT0H3uBTJp7pq6Sm+qa2jXErizH4TINy43q+OnPNgYpj+cJ wcWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S7JZ22JSG95h1zepz/8nX5FiPVxTaZFLuEXAwhmItbY=; b=C6j5YwXBayHS9i7k8Bk7o0Rp0ZTSk0LTMdb1YgyuM0VT7JRwNtmIClkGvivOSjVkLz qe3wdBTmqmYFz+0a/udS//XO7hueCHNbtf4gcyOJmmUKD3XMnutdlH+Gi5X+md19lV0X KAMX+ZToE8fGWW4kd1IB07SkeikFSFHNEVMLxrkgVwV2w6E81hTP4gqiqRMUI1AToEMy MjI/B37JH7/tMu97MBuzjbPgr5NAU2CyIR0CFTvKHnyl4KWPPZOMEnNAOlmLrTfEDrPn +lV2LMnawuP5SzM8JpFo3s8jbxzjuopoPMppISdteqVVem3tt+rQcuacDm/yOHeft38Q UyRw== X-Gm-Message-State: AOAM531tBZkWIVIJAyydwIPINlppQ8zxPnXlKWZkSFNYU66W516RD9Ck YEVKvHyK38/8sX/5W2vWlOTBPMzMWEF5wg== X-Google-Smtp-Source: ABdhPJwBbFCp+ci/9p69H3Y2MoX8rRZHfqNSFKzpWKE2dhFv9imRpjdwPEHvkNI7P/66uHn3RcRrUg== X-Received: by 2002:a17:906:690:: with SMTP id u16mr3231399ejb.186.1611660065034; Tue, 26 Jan 2021 03:21:05 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:04 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , syzbot+2f5d1785dc624932da78@syzkaller.appspotmail.com Subject: [PATCH stable 07/11] io_uring: fix false positive sqo warning on flush Date: Tue, 26 Jan 2021 11:17:06 +0000 Message-Id: X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 6b393a1ff1746a1c91bd95cbb2d79b104d8f15ac ] WARNING: CPU: 1 PID: 9094 at fs/io_uring.c:8884 io_disable_sqo_submit+0x106/0x130 fs/io_uring.c:8884 Call Trace: io_uring_flush+0x28b/0x3a0 fs/io_uring.c:9099 filp_close+0xb4/0x170 fs/open.c:1280 close_fd+0x5c/0x80 fs/file.c:626 __do_sys_close fs/open.c:1299 [inline] __se_sys_close fs/open.c:1297 [inline] __x64_sys_close+0x2f/0xa0 fs/open.c:1297 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 io_uring's final close() may be triggered by any task not only the creator. It's well handled by io_uring_flush() including SQPOLL case, though a warning in io_disable_sqo_submit() will fallaciously fire by moving this warning out to the only call site that matters. Cc: stable@vger.kernel.org # 5.5+ Reported-by: syzbot+2f5d1785dc624932da78@syzkaller.appspotmail.com Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 2acea64656f3..e8d0bea702a3 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8719,8 +8719,6 @@ static bool __io_uring_cancel_task_requests(struct io_ring_ctx *ctx, static void io_disable_sqo_submit(struct io_ring_ctx *ctx) { - WARN_ON_ONCE(ctx->sqo_task != current); - mutex_lock(&ctx->uring_lock); ctx->sqo_dead = 1; mutex_unlock(&ctx->uring_lock); @@ -8742,6 +8740,7 @@ static void io_uring_cancel_task_requests(struct io_ring_ctx *ctx, if ((ctx->flags & IORING_SETUP_SQPOLL) && ctx->sq_data) { /* for SQPOLL only sqo_task has task notes */ + WARN_ON_ONCE(ctx->sqo_task != current); io_disable_sqo_submit(ctx); task = ctx->sq_data->thread; atomic_inc(&task->io_uring->in_idle); From patchwork Tue Jan 26 11:17:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3CD3C433E0 for ; Tue, 26 Jan 2021 11:23:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1A9723104 for ; Tue, 26 Jan 2021 11:23:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392364AbhAZLX3 (ORCPT ); Tue, 26 Jan 2021 06:23:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392340AbhAZLW4 (ORCPT ); Tue, 26 Jan 2021 06:22:56 -0500 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 658ADC06178B for ; Tue, 26 Jan 2021 03:21:07 -0800 (PST) Received: by mail-ej1-x62f.google.com with SMTP id w1so22361437ejf.11 for ; Tue, 26 Jan 2021 03:21:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b62UQnLTv6+nYld9hSZxRvsaLNtFsASEvgTrPG6d828=; b=O9guSWsKduHr7hx4bRqcF3hwxeAP+h7NFcoYeCVc+Cc+JM9ocJQ5Re+mSq4lYAKW6M NyL/tsMDb2aVpHGN37BWmWLFNNf/twb7lru8KlrHnDv9Yu8mGAWs6OchDJHdK/rLfNfP bNANIJ4AnxjG6ag7d89K4YRFQnM7Bfx+MKTUgdo0Itq9f+Wr4nXNx3qcARxNK6eVxrxv 5TNcN3lD5FEiqTDwLrVHEF3r31lsh90QlmTn3+N3H3mFaUWM57Pe8VJ5QL2zQfb9akCH 02hxP8jSO7L0oSdvwz/sL5AnnVzm/cfUKsKr93uE7IOFLUF+vLcsh+sFpuzPs6rZEAR7 WDPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b62UQnLTv6+nYld9hSZxRvsaLNtFsASEvgTrPG6d828=; b=V2YBvc8KeD+6+WM6fO21MIFCh2ilPUCWqk6iQfLwBrabTksjkkZDQX8SBIMvAWX6Sf SDZpBGXElah4WBlUbn1I6JcjVOrIIOxACo70xW+KUlr6YZIyJ/9F/oYOXzaUS3Re/YTy x2xRg4ZrwDb4qJNTOF89qLlTVZjnAYGcOJth856nVwy71yRgFfvBKfUMatdI1joIm9R1 CrYIiN3hxj/2ZgPvSLIti9BXo7r3QVX00AkVKGJ/vDe1z9yk6od2ldncpuNW8xI7MLX5 S5wCBaCQKRW1S57ObP0Sn3tsMsV3wf+tykwgw+F8rtXafvnaaJjs1XqRB8CqVVntijxH 5u1A== X-Gm-Message-State: AOAM530uNAke+NA91E9VWCXA9IC60sKJat+20TO+VlKPCo9vpFzsbnrs gUqh5Z+bFK8eCfqMGh2kdRVXYftAeAQF4A== X-Google-Smtp-Source: ABdhPJy92NyAiiWgaw12rKLE1fQXN7h99NqGBiohKryl6vCWqsdl/BUmJ8BpFsuZjI4rHYKi5VPZTA== X-Received: by 2002:a17:906:6407:: with SMTP id d7mr1950702ejm.133.1611660065951; Tue, 26 Jan 2021 03:21:05 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:05 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , syzbot+a32b546d58dde07875a1@syzkaller.appspotmail.com Subject: [PATCH stable 08/11] io_uring: fix uring_flush in exit_files() warning Date: Tue, 26 Jan 2021 11:17:07 +0000 Message-Id: <0e32dce528dd20f3539b624e52b2f60d47e067fa.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 4325cb498cb743dacaa3edbec398c5255f476ef6 ] WARNING: CPU: 1 PID: 11100 at fs/io_uring.c:9096 io_uring_flush+0x326/0x3a0 fs/io_uring.c:9096 RIP: 0010:io_uring_flush+0x326/0x3a0 fs/io_uring.c:9096 Call Trace: filp_close+0xb4/0x170 fs/open.c:1280 close_files fs/file.c:401 [inline] put_files_struct fs/file.c:416 [inline] put_files_struct+0x1cc/0x350 fs/file.c:413 exit_files+0x7e/0xa0 fs/file.c:433 do_exit+0xc22/0x2ae0 kernel/exit.c:820 do_group_exit+0x125/0x310 kernel/exit.c:922 get_signal+0x3e9/0x20a0 kernel/signal.c:2770 arch_do_signal_or_restart+0x2a8/0x1eb0 arch/x86/kernel/signal.c:811 handle_signal_work kernel/entry/common.c:147 [inline] exit_to_user_mode_loop kernel/entry/common.c:171 [inline] exit_to_user_mode_prepare+0x148/0x250 kernel/entry/common.c:201 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline] syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302 entry_SYSCALL_64_after_hwframe+0x44/0xa9 An SQPOLL ring creator task may have gotten rid of its file note during exit and called io_disable_sqo_submit(), but the io_uring is still left referenced through fdtable, which will be put during close_files() and cause a false positive warning. First split the warning into two for more clarity when is hit, and the add sqo_dead check to handle the described case. Cc: stable@vger.kernel.org # 5.5+ Reported-by: syzbot+a32b546d58dde07875a1@syzkaller.appspotmail.com Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index e8d0bea702a3..12fa5e09cefa 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8931,7 +8931,10 @@ static int io_uring_flush(struct file *file, void *data) if (ctx->flags & IORING_SETUP_SQPOLL) { /* there is only one file note, which is owned by sqo_task */ - WARN_ON_ONCE((ctx->sqo_task == current) == + WARN_ON_ONCE(ctx->sqo_task != current && + xa_load(&tctx->xa, (unsigned long)file)); + /* sqo_dead check is for when this happens after cancellation */ + WARN_ON_ONCE(ctx->sqo_task == current && !ctx->sqo_dead && !xa_load(&tctx->xa, (unsigned long)file)); io_disable_sqo_submit(ctx); From patchwork Tue Jan 26 11:17:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 372440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD402C433E0 for ; Tue, 26 Jan 2021 12:07:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AAD342311F for ; Tue, 26 Jan 2021 12:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391617AbhAZMHS (ORCPT ); Tue, 26 Jan 2021 07:07:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392342AbhAZLW4 (ORCPT ); Tue, 26 Jan 2021 06:22:56 -0500 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32537C06178C for ; Tue, 26 Jan 2021 03:21:08 -0800 (PST) Received: by mail-ej1-x629.google.com with SMTP id a10so22415053ejg.10 for ; Tue, 26 Jan 2021 03:21:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y7C3dEjSQf0vm136vVDcVcxL/La2ICtMhTEJduTPdyY=; b=KJNA8NZZcIqyDFyTvW23llcFINgS/obbkExltdjW82NBT1uNl66Vrtx75yq5fLNwXF ZatTxKXv7CU57F+0WtG6SICWiuV35jxc6qBxhxIpeuy73rCDxhS142Fr9ZDI3DL3DQkA RCgW+WWf6IM0jOu1KHPlU9ZEP9ipH8c23OOOWhjYxnV1FFCsRa57Piadtbg5hn3dkk2Z D7xylax4G4u1t+TaL3ztsq9r2LAXh0PK6MT/Voc5Ct45mjcmMtC7y705xWUr35yp9ptH YeP1Dhe8VH8GymFBIvMEZ2VECsYvzZ32h2hH3wTXeNOPaa8MshgstUBwl+JTTtc1QtY6 Npbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y7C3dEjSQf0vm136vVDcVcxL/La2ICtMhTEJduTPdyY=; b=HEuYfXrKSWCc7kVVT+a9RPAI0Skr8DjUM61XGxCZjpzJDA3kzH/8MlS4fpRtLg8KGL F/O2hhd4waYHuRm1xlbd9wwj6FoWIvbdFLGw1MQQvk7m9qtIVhgp0ohlLk/CRyYzY3CO B6airxUIPoN+2XfikmPHrsXQjxlDyHpRFnT5PF3RDaIpso0VuEjyVBZhveQKrTxy07z4 t7+8rKGi8ztNR6NUC81SkunZoxgWj/L+icDsaXLx8z/K4R3s+Qg6zaypjbh1oWwK4YET 033Npo1DsJkzJeQa4Gjc6dmUI6WulQ1aAwdTrCBxpzCCl283dr93nCC5We9/L+j7+ExW nCAQ== X-Gm-Message-State: AOAM5310qxHsbuCn5vzUZsHR6wfF/TvsVJ5nCXN0IWOANnqvTfo7mfpP DviSN6RCqr3SRzE/0McWFtH4/4Ci6WM25w== X-Google-Smtp-Source: ABdhPJzVgo46tNjT7KgYUTxCtiPl9MwsQbb65NlJhPAuRBux5T4ydeEyrWjgTM+nO5QVmXc4F4BchQ== X-Received: by 2002:a17:906:298b:: with SMTP id x11mr3251012eje.158.1611660066796; Tue, 26 Jan 2021 03:21:06 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:06 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe Subject: [PATCH stable 09/11] io_uring: fix skipping disabling sqo on exec Date: Tue, 26 Jan 2021 11:17:08 +0000 Message-Id: <4bb1c422df133f0e883fefe221ffc866bfce7aa9.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 0b5cd6c32b14413bf87e10ee62be3162588dcbe6 ] If there are no requests at the time __io_uring_task_cancel() is called, tctx_inflight() returns zero and and it terminates not getting a chance to go through __io_uring_files_cancel() and do io_disable_sqo_submit(). And we absolutely want them disabled by the time cancellation ends. Cc: stable@vger.kernel.org # 5.5+ Reported-by: Jens Axboe Signed-off-by: Pavel Begunkov Fixes: d9d05217cb69 ("io_uring: stop SQPOLL submit on creator's death") Signed-off-by: Jens Axboe --- fs/io_uring.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index 12fa5e09cefa..5ead8b6aeda2 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8886,6 +8886,10 @@ void __io_uring_task_cancel(void) /* make sure overflow events are dropped */ atomic_inc(&tctx->in_idle); + /* trigger io_disable_sqo_submit() */ + if (tctx->sqpoll) + __io_uring_files_cancel(NULL); + do { /* read completions before cancelations */ inflight = tctx_inflight(tctx); From patchwork Tue Jan 26 11:17:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 371357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62C75C433DB for ; Tue, 26 Jan 2021 12:07:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33ECD2311F for ; Tue, 26 Jan 2021 12:07:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392347AbhAZMHT (ORCPT ); Tue, 26 Jan 2021 07:07:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392343AbhAZLW6 (ORCPT ); Tue, 26 Jan 2021 06:22:58 -0500 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 095E7C061793 for ; Tue, 26 Jan 2021 03:21:09 -0800 (PST) Received: by mail-ej1-x62f.google.com with SMTP id r12so22373720ejb.9 for ; Tue, 26 Jan 2021 03:21:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=43tk6v3IVG2/MesRm+z+pQpDZTwX1/vxi63pQJvRipg=; b=CQYgS/cvzzkk16lLeYHNEzPzrTPh29AP2JhvgDOTWaR0aFt+yNBYM4uKPu7yusFhBO aEVodqe6CzD4LH646HbYc6SP9yqCxYKVmCQPOU+a/S+S91npBw2OVdAS3Jspt2Ggv6Zs NPuhDPXjwFFV6qxHG8NL2Guu0aY1l8DtoFpizDC0TUyt+VckqoZ4uaIYwWuvz84A4Idm OG2QOR5g2EMXBFrJuNssPdOa2zLtzQB/Q0xwKL4nZylkFEcvamRk66SL5WQ8mLBuiC7R sjKYLVLiQOWneCDVE/hwls+y9Be0WG4YTjleiHzzlQKxDvW9hwOvqb2gz9ex/9KZq90A 8CIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=43tk6v3IVG2/MesRm+z+pQpDZTwX1/vxi63pQJvRipg=; b=JTJ5GDO358b0In/KX84dim1gC/+AAJ0+aR0lNkVQglLccMq8pzp4bSC1ss8syOjeP5 Uz+sL2lIl6r+bB3zKiVFkmRECsBjdNhFHiNlkG2RCxaxiQ0G63lNlOuHQMH/t9JF2akO cX0wHLQob9s8TpNix8ABC7uyPJg3JuCj4Bg5piv8KGm33b+kBNrAFK+sOZRJbfbyiUNG fjt367bI5/jc6tVtTlLjdY1Pz6yFo3Qp1dtB67gPU79el+94052rBsnm2+TxgyXamvbw yjoKHPIhp+DUchvssEOffrT+3KyI3Hmd4Bk9ncHwD8ndeJhEvIGmoOh0Y7TdPFIcaNIp hLUw== X-Gm-Message-State: AOAM533Sn6ToAfdpc++M64Wyt6kGxRDUzrCOuhI+aRQCd/t2tDPFlB0q fwvrbJs5zhsHYcNCyRfd8X+gdJOz0JE65g== X-Google-Smtp-Source: ABdhPJwl3NOhtSsb7q0O6s0l9JbSsho+0RLToUKXR4ut5Y9mHVY288/kouyYyhqFIPeAyPCt/5N5Xg== X-Received: by 2002:a17:906:b2d5:: with SMTP id cf21mr3164578ejb.387.1611660067623; Tue, 26 Jan 2021 03:21:07 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:07 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , syzbot+91ca3f25bd7f795f019c@syzkaller.appspotmail.com Subject: [PATCH stable 10/11] io_uring: dont kill fasync under completion_lock Date: Tue, 26 Jan 2021 11:17:09 +0000 Message-Id: <41777a0d41ba61011c7ef96d44de36f37ec5e8ea.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 4aa84f2ffa81f71e15e5cffc2cc6090dbee78f8e ] CPU0 CPU1 ---- ---- lock(&new->fa_lock); local_irq_disable(); lock(&ctx->completion_lock); lock(&new->fa_lock); lock(&ctx->completion_lock); *** DEADLOCK *** Move kill_fasync() out of io_commit_cqring() to io_cqring_ev_posted(), so it doesn't hold completion_lock while doing it. That saves from the reported deadlock, and it's just nice to shorten the locking time and untangle nested locks (compl_lock -> wq_head::lock). Cc: stable@vger.kernel.org # 5.5+ Reported-by: syzbot+91ca3f25bd7f795f019c@syzkaller.appspotmail.com Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 5ead8b6aeda2..1c5d71829bf5 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1213,11 +1213,6 @@ static void __io_commit_cqring(struct io_ring_ctx *ctx) /* order cqe stores with ring update */ smp_store_release(&rings->cq.tail, ctx->cached_cq_tail); - - if (wq_has_sleeper(&ctx->cq_wait)) { - wake_up_interruptible(&ctx->cq_wait); - kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN); - } } static void io_put_identity(struct io_uring_task *tctx, struct io_kiocb *req) @@ -1584,6 +1579,10 @@ static inline bool io_should_trigger_evfd(struct io_ring_ctx *ctx) static void io_cqring_ev_posted(struct io_ring_ctx *ctx) { + if (wq_has_sleeper(&ctx->cq_wait)) { + wake_up_interruptible(&ctx->cq_wait); + kill_fasync(&ctx->cq_fasync, SIGIO, POLL_IN); + } if (waitqueue_active(&ctx->wait)) wake_up(&ctx->wait); if (ctx->sq_data && waitqueue_active(&ctx->sq_data->wait)) From patchwork Tue Jan 26 11:17:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 372441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3231CC433E6 for ; Tue, 26 Jan 2021 11:23:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0279B22795 for ; Tue, 26 Jan 2021 11:23:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392247AbhAZLXb (ORCPT ); Tue, 26 Jan 2021 06:23:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392345AbhAZLW6 (ORCPT ); Tue, 26 Jan 2021 06:22:58 -0500 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11DF8C061794 for ; Tue, 26 Jan 2021 03:21:10 -0800 (PST) Received: by mail-ej1-x631.google.com with SMTP id a10so22415175ejg.10 for ; Tue, 26 Jan 2021 03:21:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H8jaeSNFaDfrMbH20xY2YYq+yl/M/sDJfXI3SIGJV/o=; b=ZbfN9DWroEUtEy3ihgHVbEqXlOTQBU4J48JMU+nDkBEUJQbIkmULtnVO5dE8xOo0jd qCTFPSWpwVVz96P6yvVnaNg5A+u0Wv2ruBZGQA/69n77h4lU6EQzXYRwkmRU8fa174zV T/QhPdp6x1ip0pwlxiGCFBcM+c72mtICJBiofvPL5VZ8PIXsv1BJ6ZMOtNCAT3joseTL 6nTRZgQSRRUHaF8n+lx2/xuLp7SvHTnzabgNKV8twjC6GVl1cWzRn2rUoUTacp4DSXQT YWoDdnWQ6iKFxMuotsw28hqEVU639oJg4EbGt9zuFgsT+O4UXy9yCKaKuptrELBsVwf/ T79w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H8jaeSNFaDfrMbH20xY2YYq+yl/M/sDJfXI3SIGJV/o=; b=eEL0s0nyE3TSt5FUaYCvsEuXak8eE+bhgAI86YdcVeEkleiVgGjp9m1O5VUBKS6dIM CoqwYzky2Xzvq3UufVedlzQk1DrzIs29JcoYGMGZ9sb9D0i0fK7iWd0Uyccqp6e49+j+ SijmWhixH9BDJeTt/Y0Y6Hd4awCXP9sMDTxjw5TYrb8k7HF7iftyd7vrYb4Y4ZhPFe04 0JySE/uwmBj5T12BmcUdVto9hfXp4ousFIkkoSU7jtNLSTq7blW2rZLw+MTOBGbjLg5m X8ywTKf6I8zeeEnyejXGH6pU2FX7WcwbDGVSYFyV8wXcqACDWwqqsGKW2MTANbkrvnXr cenw== X-Gm-Message-State: AOAM530mEll4ycmBeSivxL/ZaEQyt0iXXuWMGp6IYVV72R3o2os44rld EskKhM+VvjOx2KxkqpGdM03Is9jUTAuXHA== X-Google-Smtp-Source: ABdhPJziU1O5y5pkltAF7hnarhkEP3bGfusLW4eecujstGW+0LZsUPYjL4J7sHCd2bY7AhiMzBH/Eg== X-Received: by 2002:a17:906:25db:: with SMTP id n27mr3182414ejb.552.1611660068557; Tue, 26 Jan 2021 03:21:08 -0800 (PST) Received: from localhost.localdomain ([148.252.129.161]) by smtp.gmail.com with ESMTPSA id o17sm12167128edr.17.2021.01.26.03.21.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 03:21:08 -0800 (PST) From: Pavel Begunkov To: stable@vger.kernel.org Cc: Jens Axboe , Abaci , Joseph Qi , Xiaoguang Wang Subject: [PATCH stable 11/11] io_uring: fix sleeping under spin in __io_clean_op Date: Tue, 26 Jan 2021 11:17:10 +0000 Message-Id: <61e93a6403ea6cc28764e7508cd877ca30345371.1611659564.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org [ Upstream commit 9d5c8190683a462dbc787658467a0da17011ea5f ] [ 27.629441] BUG: sleeping function called from invalid context at fs/file.c:402 [ 27.631317] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 1012, name: io_wqe_worker-0 [ 27.633220] 1 lock held by io_wqe_worker-0/1012: [ 27.634286] #0: ffff888105e26c98 (&ctx->completion_lock) {....}-{2:2}, at: __io_req_complete.part.102+0x30/0x70 [ 27.649249] Call Trace: [ 27.649874] dump_stack+0xac/0xe3 [ 27.650666] ___might_sleep+0x284/0x2c0 [ 27.651566] put_files_struct+0xb8/0x120 [ 27.652481] __io_clean_op+0x10c/0x2a0 [ 27.653362] __io_cqring_fill_event+0x2c1/0x350 [ 27.654399] __io_req_complete.part.102+0x41/0x70 [ 27.655464] io_openat2+0x151/0x300 [ 27.656297] io_issue_sqe+0x6c/0x14e0 [ 27.660991] io_wq_submit_work+0x7f/0x240 [ 27.662890] io_worker_handle_work+0x501/0x8a0 [ 27.664836] io_wqe_worker+0x158/0x520 [ 27.667726] kthread+0x134/0x180 [ 27.669641] ret_from_fork+0x1f/0x30 Instead of cleaning files on overflow, return back overflow cancellation into io_uring_cancel_files(). Previously it was racy to clean REQ_F_OVERFLOW flag, but we got rid of it, and can do it through repetitive attempts targeting all matching requests. Cc: stable@vger.kernel.org # 5.9+ Reported-by: Abaci Reported-by: Joseph Qi Cc: Xiaoguang Wang Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe --- fs/io_uring.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 1c5d71829bf5..f77821626a92 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -970,6 +970,7 @@ static ssize_t io_import_iovec(int rw, struct io_kiocb *req, static int io_setup_async_rw(struct io_kiocb *req, const struct iovec *iovec, const struct iovec *fast_iov, struct iov_iter *iter, bool force); +static void io_req_drop_files(struct io_kiocb *req); static struct kmem_cache *req_cachep; @@ -990,8 +991,7 @@ EXPORT_SYMBOL(io_uring_get_socket); static inline void io_clean_op(struct io_kiocb *req) { - if (req->flags & (REQ_F_NEED_CLEANUP | REQ_F_BUFFER_SELECTED | - REQ_F_INFLIGHT)) + if (req->flags & (REQ_F_NEED_CLEANUP | REQ_F_BUFFER_SELECTED)) __io_clean_op(req); } @@ -1255,6 +1255,8 @@ static void io_req_clean_work(struct io_kiocb *req) free_fs_struct(fs); req->work.flags &= ~IO_WQ_WORK_FS; } + if (req->flags & REQ_F_INFLIGHT) + io_req_drop_files(req); io_put_identity(req->task->io_uring, req); } @@ -5929,9 +5931,6 @@ static void __io_clean_op(struct io_kiocb *req) } req->flags &= ~REQ_F_NEED_CLEANUP; } - - if (req->flags & REQ_F_INFLIGHT) - io_req_drop_files(req); } static int io_issue_sqe(struct io_kiocb *req, bool force_nonblock, @@ -8669,6 +8668,8 @@ static bool io_uring_cancel_files(struct io_ring_ctx *ctx, break; /* cancel this request, or head link requests */ io_attempt_cancel(ctx, cancel_req); + io_cqring_overflow_flush(ctx, true, task, files); + io_put_req(cancel_req); /* cancellations _may_ trigger task work */ io_run_task_work();