From patchwork Wed Aug 7 14:17:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 170759 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp7207556ile; Wed, 7 Aug 2019 07:18:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqzvUPyB6BX85KXiYHTWdK/P5tC5HLebk7dsFotkRTZGBofz/Ekp0DoxeG8AYNonRUHBwx6U X-Received: by 2002:a17:90a:ca0f:: with SMTP id x15mr206422pjt.82.1565187494679; Wed, 07 Aug 2019 07:18:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565187494; cv=none; d=google.com; s=arc-20160816; b=u+SnbBaNzXkrs4+TSoRQL7cHtn166Ahb3SYgJkk1GpcQv/JGrPqy2soQt3zwp2SbTO OS6Gz6UbBUYWf39s3a07KAAyIVg/hu9YjYqYTKZZqTS/s5jo0QC5VB8DjcHlP0DIDf4R Lgmpx3BhDVWT28uUerHbDFjBijKqxwWeaobF0ONuE8huCeo6h76t9a2lXTCWzg5X3oq4 B9IpEZHSykHMFGmpYR5X8Ss5Is+yUwfIJtZ+a6lZF3t0gN3IIY1JLKdNkzvQZpadDiAL zA5eundBH5tlWNWN7uo4GJkj1UgM0wYV7JSu7V7IxEw+atIMqwF8g6SlNK2EjF6fFOyy 1gng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kkeCGHY0DrsSxD9DKRdcnC2Wr0bb+DymKOjtK802aV8=; b=ogdZEN0xqSQ1i3kde6wYtSp9Mfd+/WgIsoPzxnILSd/dk9RPupB9LgdAdUcjyVDhu+ NjEXWTfNn2vrRpAsBbVM2+FchpKzkST+fyRHMkx4nyx3Kk8VtmxOGuiTVuS6Yw6EvSpd HVyvKwihTd2uK4Kvozt8Isgcreavtpgeeb4c1OPSFHDRX6v3bGfs/39EqpaHIEf0cCAY sulQhsNmj3zIp80rgSBYKYzzRW6aWd0GQWoWHv3sk3URSdHLuSD5a7hzYnNjZJtAe3jY +/PE/lGDiV5Vls8VtwcqDah+QhAI7/VcarTUXGlc0uYMHPIMe/P82ePjlbX85d4L3hh0 7a9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s487BfnV; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 80si52468243pgf.5.2019.08.07.07.18.14; Wed, 07 Aug 2019 07:18:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s487BfnV; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388325AbfHGOSN (ORCPT + 28 others); Wed, 7 Aug 2019 10:18:13 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:51073 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729999AbfHGOSK (ORCPT ); Wed, 7 Aug 2019 10:18:10 -0400 Received: by mail-wm1-f68.google.com with SMTP id v15so244729wml.0 for ; Wed, 07 Aug 2019 07:18:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kkeCGHY0DrsSxD9DKRdcnC2Wr0bb+DymKOjtK802aV8=; b=s487BfnVB9DFRp3LPdjVxRcYZly5YMoGZEBCfc4Yk3Hs9KDNnZp/jkEHPUUrLPNVG3 22uvC0GjMLtxHzpnrmH7k6IQ5EMXulSzuzzEeZ+TMPpXRsM4Cy3q9X03+8x+lXnO8Q0W wvAGp6vk06yhqPgWh7GJ9aU0iNmuzyYJv9CwoZBkxC+7m14HlpyA5RmRFBLmj5qrwJ4F ThM4d4PIwnK7kYhw5T+tfJJO1O7m0Mps2/BVDGzUn/rxYMUwJTptf6n2oWzhBwxlGf8S XjbswJu8iiM2PFjuwp+Wn/jsK5ShQ/5zx1PodtxwKmgV0tUjPBAAxoAvlIEYsDuff5yz Zjjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kkeCGHY0DrsSxD9DKRdcnC2Wr0bb+DymKOjtK802aV8=; b=Ddxx4gsJ6hz/BV+Hx2ibjUfcR8GMID+8Xos+7WRaIIUTACccH+q2/1qKl92x51zA0i pJVxnJXu6+m6H1lMVwrNarK+opArxic77cROfSjh1MGWSaT+DdL1OItmjiPGpWIT7TVb lCHkfozGBeUuDLPtPD0NSqOy238m9luxQ8xcNFYr9bwXp7qodsqDWszVCTPBdeSPV33x miQ2V5wrGBlscD1aI02jiZdJHsvN3UaLj/4MzVaFyNYh+kvUZ80wODfeM7ZoCCZuMnEk ubUKknj0tiyCE2HW4iw6yFVxeQroir8g0MaM8sj8trrxU5XOp8uTghjVIRqYQ811jJKq OI8g== X-Gm-Message-State: APjAAAXyv8xiF5lVEm5ZPlFPuXIjAngu5qH6TMn8qfHgmdUQxP4fmkw6 QAB9Sj58m/siOnuWlUoMgedikQ== X-Received: by 2002:a1c:4d01:: with SMTP id o1mr204052wmh.55.1565187488110; Wed, 07 Aug 2019 07:18:08 -0700 (PDT) Received: from localhost.localdomain (88-147-66-140.dyn.eolo.it. [88.147.66.140]) by smtp.gmail.com with ESMTPSA id o7sm83472wmc.36.2019.08.07.07.18.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Aug 2019 07:18:07 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, pavel@denx.de, Paolo Valente , Douglas Anderson Subject: [PATCH BUGFIX 1/2] block, bfq: reset last_completed_rq_bfqq if the pointed queue is freed Date: Wed, 7 Aug 2019 16:17:53 +0200 Message-Id: <20190807141754.3567-2-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190807141754.3567-1-paolo.valente@linaro.org> References: <20190807141754.3567-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Since commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O"), BFQ stores, in a per-device pointer last_completed_rq_bfqq, the last bfq_queue that had an I/O request completed. If some bfq_queue receives new I/O right after the last request of last_completed_rq_bfqq has been completed, then last_completed_rq_bfqq may be a waker bfq_queue. But if the bfq_queue last_completed_rq_bfqq points to is freed, then last_completed_rq_bfqq becomes a dangling reference. This commit resets last_completed_rq_bfqq if the pointed bfq_queue is freed. Fixes: 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") Reported-by: Douglas Anderson Tested-by: Douglas Anderson Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 586fcfe227ea..b2009650afc2 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1924,12 +1924,13 @@ static void bfq_add_request(struct request *rq) * confirmed no later than during the next * I/O-plugging interval for bfqq. */ - if (!bfq_bfqq_has_short_ttime(bfqq) && + if (bfqd->last_completed_rq_bfqq && + !bfq_bfqq_has_short_ttime(bfqq) && ktime_get_ns() - bfqd->last_completion < 200 * NSEC_PER_USEC) { if (bfqd->last_completed_rq_bfqq != bfqq && - bfqd->last_completed_rq_bfqq != - bfqq->waker_bfqq) { + bfqd->last_completed_rq_bfqq != + bfqq->waker_bfqq) { /* * First synchronization detected with * a candidate waker queue, or with a @@ -4808,6 +4809,9 @@ void bfq_put_queue(struct bfq_queue *bfqq) bfqq->bfqd->burst_size--; } + if (bfqq->bfqd && bfqq->bfqd->last_completed_rq_bfqq == bfqq) + bfqq->bfqd->last_completed_rq_bfqq = NULL; + kmem_cache_free(bfq_pool, bfqq); #ifdef CONFIG_BFQ_GROUP_IOSCHED bfqg_and_blkg_put(bfqg); From patchwork Wed Aug 7 14:17:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 170760 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp7207670ile; Wed, 7 Aug 2019 07:18:19 -0700 (PDT) X-Google-Smtp-Source: APXvYqxvlnxtfrscpB87Fwhw/ifxE3qBiAvuzlS/CHCejHm9JGQdyV6yWOZ46rCyyGytxobdddDw X-Received: by 2002:a65:4546:: with SMTP id x6mr7805871pgr.266.1565187499110; Wed, 07 Aug 2019 07:18:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565187499; cv=none; d=google.com; s=arc-20160816; b=RXatzupbCby+vqDF3DALeT1Pazn4/yaajuBEpFQCw+EWLup1OlTshSXARX4/ATzDMk uaJll99WR3/lDKgcsK/Iv6gInnmsNC2g1tpwgv1ggF5Llp41Rdn8F7H0CHiIqulHHxOG 4BgFhXmGkRYCZFc+iD20xEHWeGhaX5EoItmCQGsTUQJzXW+oF1qvLFU7MCHpyHXjTma4 GjKhMoxXnwoOQaPH74ftD7gpDuwzsxaWUFyJ+U9xUFTcEX/90TsknXQYIaxznLK6+jhj rn7ziT0qGAH4kPJxraq6lbVDUBMvexEaFeG+r2K6s5rbNU3p3isITcge08Mwzgv3M/RN qQnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2fEGd1avRjzNgoZKRJo57rOsvWSouUmLScAruU7orM4=; b=nkDdTTwODoMV83dHzEVCtbBfYCQxDpFsh4ltPheOmoisStYlRJ8QmKxdkrwjeER9pQ TTiwP65YGdZPHCFT1SrZwGrUNdBXfrHu68hmkrMuNCX28Uv+VirKTwYl/KdArspZ6yPP qbYtcze6hkbgW2xW6U7YKq6QDUdp2jvEMu0WWZhg4baRfE+m778xbD6rnc5RUfU9wW2t LyHHgY5Fb2XfHceT7wVKV5q5cYpzUf3XI2e0A4SXsJIOeEpyr0TkiEh0qFBwRNnU8SgT 1RaMUZinJ4pzXLell+fYUhxlsFvkqSB3eZ0s8JE0Aj8hCOHFmVEGCLz700MH9mnF4km+ V8aQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CDf3h5lu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r1si48038882pls.69.2019.08.07.07.18.18; Wed, 07 Aug 2019 07:18:19 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CDf3h5lu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388348AbfHGOSR (ORCPT + 28 others); Wed, 7 Aug 2019 10:18:17 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:54422 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388283AbfHGOSM (ORCPT ); Wed, 7 Aug 2019 10:18:12 -0400 Received: by mail-wm1-f66.google.com with SMTP id p74so220183wme.4 for ; Wed, 07 Aug 2019 07:18:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2fEGd1avRjzNgoZKRJo57rOsvWSouUmLScAruU7orM4=; b=CDf3h5luG1Il3ysYALeHWQ6nAmZqFgCk9fpa7U9ICPkY270+BhZcQd1bpfu9LpL/cW 01/f7VTeMqbP+MFgqN7O087injQ04LCr9x8jQzbG7vzOiXIkQmXy6B19cyzm+lte6gie kqfl9GXrDqh+UN78SCyYhcv1oQ7uZOpfHYTvW8HGDhfucQtMZxypnsAYiai/SXLIpEvB 8i0Mx/Roa1Tx3g1TdvMXc5BFGIGd8zDcU7tw206biQfAAWpwz77CIYp3lbJ/uApynJxu +FOOPcNPL+mQmuU14xJh/0aDwunKYcmCasNuIlg5DEkxVEUSZ6o3NlMT+TBiw6I03F/o qROQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2fEGd1avRjzNgoZKRJo57rOsvWSouUmLScAruU7orM4=; b=sjOPFackJaUIdxBmq8aH6FM9BT55JWaknAwH2JDvq4o++ght1cEvDhB1q6TeiNLAxq Yl82OPut2WLh6LxD4ecIzg5nk/EPCUDWa4B8WUfcPtD4BfPGb4vc43CKrwNgk9QN9uPk NNE6g9ACbsnB+0uRhel7mgsUVbaq3GecyfJBokqnVhe3YFrhXCqxmI9jBP1ULLB8AI1x U1o2YJWXJTeAMcuURoJ3J35+teHYkZduy6E31ox80Uva8TF5yi1PGNoLCMOrsUIP72oT l5/4hCjySiY+ZlnHTjnF2d7J7LLpY7F9gSyjq80kKwpVOnd40om4vtMs2n1v2cGeR2pS VH+w== X-Gm-Message-State: APjAAAUsF955xljBVBi6y/03QrT9XEBLWx3+eYtixO7Zqbm/rOfSoYlh OuEj0uoM9qykDHu1xe+1Bw2bqg== X-Received: by 2002:a1c:9696:: with SMTP id y144mr194876wmd.73.1565187489338; Wed, 07 Aug 2019 07:18:09 -0700 (PDT) Received: from localhost.localdomain (88-147-66-140.dyn.eolo.it. [88.147.66.140]) by smtp.gmail.com with ESMTPSA id o7sm83472wmc.36.2019.08.07.07.18.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Aug 2019 07:18:08 -0700 (PDT) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, pavel@denx.de, Paolo Valente , Douglas Anderson Subject: [PATCH BUGFIX 2/2] block, bfq: move update of waker and woken list to queue freeing Date: Wed, 7 Aug 2019 16:17:54 +0200 Message-Id: <20190807141754.3567-3-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190807141754.3567-1-paolo.valente@linaro.org> References: <20190807141754.3567-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Since commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O"), every bfq_queue has a pointer to a waker bfq_queue and a list of the bfq_queues it may wake. In this respect, when a bfq_queue, say Q, remains with no I/O source attached to it, Q cannot be woken by any other bfq_queue, and cannot wake any other bfq_queue. Then Q must be removed from the woken list of its possible waker bfq_queue, and all bfq_queues in the woken list of Q must stop having a waker bfq_queue. Q remains with no I/O source in two cases: when the last process associated with Q exits or when such a process gets associated with a different bfq_queue. Unfortunately, commit 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") performed the above updates only in the first case. This commit fixes this bug by moving these updates to when Q gets freed. This is a simple and safe way to handle all cases, as both the above events, process exit and re-association, lead to Q being freed soon, and because dangling references would come out only after Q gets freed (if no update were performed). Fixes: 13a857a4c4e8 ("block, bfq: detect wakers and unconditionally inject their I/O") Reported-by: Douglas Anderson Tested-by: Douglas Anderson Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index b2009650afc2..5f477501bb3d 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4765,6 +4765,8 @@ static struct request *bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) */ void bfq_put_queue(struct bfq_queue *bfqq) { + struct bfq_queue *item; + struct hlist_node *n; #ifdef CONFIG_BFQ_GROUP_IOSCHED struct bfq_group *bfqg = bfqq_group(bfqq); #endif @@ -4809,6 +4811,33 @@ void bfq_put_queue(struct bfq_queue *bfqq) bfqq->bfqd->burst_size--; } + /* + * bfqq does not exist any longer, so it cannot be woken by + * any other queue, and cannot wake any other queue. Then bfqq + * must be removed from the woken list of its possible waker + * queue, and all queues in the woken list of bfqq must stop + * having a waker queue. Strictly speaking, these updates + * should be performed when bfqq remains with no I/O source + * attached to it, which happens before bfqq gets freed. In + * particular, this happens when the last process associated + * with bfqq exits or gets associated with a different + * queue. However, both events lead to bfqq being freed soon, + * and dangling references would come out only after bfqq gets + * freed. So these updates are done here, as a simple and safe + * way to handle all cases. + */ + /* remove bfqq from woken list */ + if (!hlist_unhashed(&bfqq->woken_list_node)) + hlist_del_init(&bfqq->woken_list_node); + + /* reset waker for all queues in woken list */ + hlist_for_each_entry_safe(item, n, &bfqq->woken_list, + woken_list_node) { + item->waker_bfqq = NULL; + bfq_clear_bfqq_has_waker(item); + hlist_del_init(&item->woken_list_node); + } + if (bfqq->bfqd && bfqq->bfqd->last_completed_rq_bfqq == bfqq) bfqq->bfqd->last_completed_rq_bfqq = NULL; @@ -4839,9 +4868,6 @@ static void bfq_put_cooperator(struct bfq_queue *bfqq) static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) { - struct bfq_queue *item; - struct hlist_node *n; - if (bfqq == bfqd->in_service_queue) { __bfq_bfqq_expire(bfqd, bfqq, BFQQE_BUDGET_TIMEOUT); bfq_schedule_dispatch(bfqd); @@ -4851,18 +4877,6 @@ static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) bfq_put_cooperator(bfqq); - /* remove bfqq from woken list */ - if (!hlist_unhashed(&bfqq->woken_list_node)) - hlist_del_init(&bfqq->woken_list_node); - - /* reset waker for all queues in woken list */ - hlist_for_each_entry_safe(item, n, &bfqq->woken_list, - woken_list_node) { - item->waker_bfqq = NULL; - bfq_clear_bfqq_has_waker(item); - hlist_del_init(&item->woken_list_node); - } - bfq_put_queue(bfqq); /* release process reference */ }