From patchwork Tue Jan 29 11:06:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156970 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523594jaa; Tue, 29 Jan 2019 03:08:08 -0800 (PST) X-Google-Smtp-Source: ALg8bN570pR7IQI7ai1vdQW8eHhjcK+tLILAa5uTEFx/E/d8FjzpegCNqRfW33iDZAnrAarN7bx0 X-Received: by 2002:a63:4948:: with SMTP id y8mr23239178pgk.32.1548760088202; Tue, 29 Jan 2019 03:08:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760088; cv=none; d=google.com; s=arc-20160816; b=aTrhJkwqtwQhA+qiiKn+i3+5Uw0Q6bRjslvC6ve7zek1l9SIGF+iey4rZH2zvR76AN Qi7TEhf5W5uFd8jr+3lhSzObyGq9eJ8H+D5UC4yhMp106iAUbE+fsXyZr/pHZs97tPEi D4JX04PdzN/t3JNrf3svHlnhEHMbgWBm4wZbbpy16xEJC+AlMjf+k6p7LnNlhFsjsTe9 vmJSTQcQrIgVMgRLoT1PqTd2B7cGsXEQR1gQsYlGgziPj9b432oZIqkARyPALdx2yxgj X7058fbyywHmWhQBISBG1oHPLELN951ziGt9tZS4MDGKEVlYQDzHv6LkFJ22YW0l3ydF eF7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KqztSqIPzKKBWIFY2MqAio0w3pO/+CETJ20qHxYP8sU=; b=bhXSVw6oxdgj+xNW2PWauB3U4Fwkq6/A1MzHYo75gjGx+onlCEqrbbNKXjii4fRxbc n++/P7EiJzetrRMqDOIJBwu8vWlpijMtqNnzWfcai75QVmHzurBJHnqUmmfdq1NyzaXu NxKZY3w3tKKwVJpPO0TIY3PnNhEQC8PgrB+PjtpDomM9Ds047M8LnCBhkYyV78CrOWdD +byK5vOHF2xq5FRdobdMk/I2OBh0PXHL5hBJcx35TJSpvCnEEG/uRIGcNTvo4pe0TsDL EwLC2vOo5IPWK/chgNun7sdqRfsSzLwNt/z5C0y05BTCphxvMHSTlYfZogTda67tqgua 9TBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="dtSOU/Wk"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c4si3361454pfi.110.2019.01.29.03.08.07; Tue, 29 Jan 2019 03:08:08 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="dtSOU/Wk"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728026AbfA2LHF (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:05 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:37843 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726893AbfA2LHB (ORCPT ); Tue, 29 Jan 2019 06:07:01 -0500 Received: by mail-wm1-f65.google.com with SMTP id g67so17260894wmd.2 for ; Tue, 29 Jan 2019 03:06:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KqztSqIPzKKBWIFY2MqAio0w3pO/+CETJ20qHxYP8sU=; b=dtSOU/WkVIhSYUqeW3JTn3QUufABzxHPLxowScogLO1dE4ZwITfBQdRm0f+ZtddNKB UXUi1A/1zNrj6MnKUUu6LpqcGQ1WdyfuVgpmnqma4bCJGWtcxZaEfLgdfbanuGNMHgWo vhkVrylJKJu+zYCvMj/ntuM4/9rogp/gl39Lk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KqztSqIPzKKBWIFY2MqAio0w3pO/+CETJ20qHxYP8sU=; b=JLOuJn9TGrvoRmlFYHxHUKB5sjjks4jJOBX9exDsRzzlIBXeQ6fmQqbqVLuFYCQRTl hbbqDegRWJtIR+1LG3qMF7mVHrN0rAMRnQKViTzOS7W45hrBfn+sP/s017LDuQsoXQV8 wWzYbdirXvTAOQ3y3Nm/CH40D8AHVMCbbtN0nmLCyKdFzk9eaRFrFcJ3e5KL+FdeiLhb ZrOO0awaRS378kPRvGPQOetn+hpQHOhRw6XnkIvEK1PTpgyrCDv6lG3bV55aBeUAWRyO D8orWA/fcSH4um2Xs0goKRfX1dKrkIZNofUrynJXSXoGP9stjsrt34DeeAyebMYBICBb 80KQ== X-Gm-Message-State: AJcUukfqXGzfgLebIfYdXZfPbBO26oGf0tfCCwoK9xVkBgX+QomtaBJE f7Ip4E+TgqpF77eonoystcTkVA== X-Received: by 2002:a1c:c87:: with SMTP id 129mr19704056wmm.116.1548760018406; Tue, 29 Jan 2019 03:06:58 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.06.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:06:57 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 01/14] block, bfq: do not consider interactive queues in srt filtering Date: Tue, 29 Jan 2019 12:06:25 +0100 Message-Id: <20190129110638.12652-2-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The speed at which a bfq_queue receives I/O is one of the parameters by which bfq decides whether the queue is soft real-time (i.e., whether the queue contains the I/O of a soft real-time application). In particular, when a bfq_queue remains without outstanding I/O requests, bfq computes the minimum time instant, named soft_rt_next_start, at which the next request of the queue may arrive for the queue to be deemed as soft real time. Unfortunately this filtering may cause problems with a queue in interactive weight raising. In fact, such a queue may be conveying the I/O needed to load a soft real-time application. The latter will actually exhibit a soft real-time I/O pattern after it finally starts doing its job. But, if soft_rt_next_start is updated for an interactive bfq_queue, and the queue has received a lot of service before remaining with no outstanding request (likely to happen on a fast device), then soft_rt_next_start is assigned such a high value that, for a very long time, the queue is prevented from being possibly considered as soft real time. This commit removes the updating of soft_rt_next_start for bfq_queues in interactive weight raising. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index cd307767a134..c7a4a15c7c19 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -3274,16 +3274,32 @@ void bfq_bfqq_expire(struct bfq_data *bfqd, * requests, then the request pattern is isochronous * (see the comments on the function * bfq_bfqq_softrt_next_start()). Thus we can compute - * soft_rt_next_start. If, instead, the queue still - * has outstanding requests, then we have to wait for - * the completion of all the outstanding requests to - * discover whether the request pattern is actually - * isochronous. + * soft_rt_next_start. And we do it, unless bfqq is in + * interactive weight raising. We do not do it in the + * latter subcase, for the following reason. bfqq may + * be conveying the I/O needed to load a soft + * real-time application. Such an application will + * actually exhibit a soft real-time I/O pattern after + * it finally starts doing its job. But, if + * soft_rt_next_start is computed here for an + * interactive bfqq, and bfqq had received a lot of + * service before remaining with no outstanding + * request (likely to happen on a fast device), then + * soft_rt_next_start would be assigned such a high + * value that, for a very long time, bfqq would be + * prevented from being possibly considered as soft + * real time. + * + * If, instead, the queue still has outstanding + * requests, then we have to wait for the completion + * of all the outstanding requests to discover whether + * the request pattern is actually isochronous. */ - if (bfqq->dispatched == 0) + if (bfqq->dispatched == 0 && + bfqq->wr_coeff != bfqd->bfq_wr_coeff) bfqq->soft_rt_next_start = bfq_bfqq_softrt_next_start(bfqd, bfqq); - else { + else if (bfqq->dispatched > 0) { /* * Schedule an update of soft_rt_next_start to when * the task may be discovered to be isochronous. @@ -4834,11 +4850,14 @@ static void bfq_completed_request(struct bfq_queue *bfqq, struct bfq_data *bfqd) * isochronous, and both requisites for this condition to hold * are now satisfied, then compute soft_rt_next_start (see the * comments on the function bfq_bfqq_softrt_next_start()). We - * schedule this delayed check when bfqq expires, if it still - * has in-flight requests. + * do not compute soft_rt_next_start if bfqq is in interactive + * weight raising (see the comments in bfq_bfqq_expire() for + * an explanation). We schedule this delayed update when bfqq + * expires, if it still has in-flight requests. */ if (bfq_bfqq_softrt_update(bfqq) && bfqq->dispatched == 0 && - RB_EMPTY_ROOT(&bfqq->sort_list)) + RB_EMPTY_ROOT(&bfqq->sort_list) && + bfqq->wr_coeff != bfqd->bfq_wr_coeff) bfqq->soft_rt_next_start = bfq_bfqq_softrt_next_start(bfqd, bfqq); From patchwork Tue Jan 29 11:06:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156958 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522528jaa; Tue, 29 Jan 2019 03:07:08 -0800 (PST) X-Google-Smtp-Source: ALg8bN5BUc+WSZmMnuKWylBglc6DYjiaEM7JxkPWh38X0eN9KQrLftzfGKBxXziPtrix4w9zuDE0 X-Received: by 2002:a63:5402:: with SMTP id i2mr22691155pgb.79.1548760028468; Tue, 29 Jan 2019 03:07:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760028; cv=none; d=google.com; s=arc-20160816; b=jjlaJQWwmk3s0XySukG5VGqcYd6eLovxbM70QfgDzMT6wJ/Bcpi6+7EQrfqerC2GYa a1vUFTDubLSv1JFp5C3CgnX9RZ4wC6HOEp9X10M3tgtlFc1wkYTAr6qe8whJWLaRR/7O F5yoNAhuWgLIXiFK3KdeOE3dgmVS1qfzoflmtJYa9/CJFGOpjlNDW+N/chrllc00zeAM 8exxNYVH8zzylgo9cyFpVZXQdX5zSJZ1koikFVe6fUAnRPft08b7U3hInri2Jlg/U/Zh on+szC3WChte/cHKMziAJmJFS4ariAKZ8roUaH7htt82HUzmm4cJeunm3LXQMY6sJ3TD Zw8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=gAUJ5pwSD9lzILEQVJ0hCKATJxbw5gWvL06xocd7ofI=; b=yGFkEInnCest1TlQUZ2X7lH4Ros966f4jQxjYdIIOGDhGSD+zTl2HOk1gLGUiIC0MC hcZ5Hx7p1CuAr0WL62O/hC/1Lw94NxzkIHTKYGUc8VMiqQigj5sZtKRa6uesLkDge+TF 0qtX2Xxd4Ln6WGfjpqOSXK9Y6rHg035wfiu1+aR53Xx+X/jk3J5H5h3NYPnbOwEecgfE CHA3C0PDfOZ64SJtCV9/ps9LfzFOJtQ7v0hYPogzj7TA+dcRpoVZdlOtklTKRLMOyXDg +dwj5nziz3mSP+zzRDnuJOTqez23g9bkoZbsev/6Y6bGoiqtwstvMMfIIYG6mkJE9VGX bmiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QvbYcrkZ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 187si11085318pfv.238.2019.01.29.03.07.08; Tue, 29 Jan 2019 03:07:08 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QvbYcrkZ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728366AbfA2LHG (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:06 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:46972 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725804AbfA2LHB (ORCPT ); Tue, 29 Jan 2019 06:07:01 -0500 Received: by mail-wr1-f67.google.com with SMTP id l9so21486017wrt.13 for ; Tue, 29 Jan 2019 03:07:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gAUJ5pwSD9lzILEQVJ0hCKATJxbw5gWvL06xocd7ofI=; b=QvbYcrkZ78lOm1jmQmedgdRvvGotGVNAELS2tmY8vO3gAAYJdENhs+i6K5OIIS+aBo xUCOofevIfoCvVZg2QYNSy+FXYOWoznVsUprE60jpyQfKtFMCKboVcVrfLJ5xT07Nnt9 1ToHzHBJJd9kTtBhEl43pS4OGRudDsvoqg7rY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gAUJ5pwSD9lzILEQVJ0hCKATJxbw5gWvL06xocd7ofI=; b=eMRqut/2j2KY0b/jR6e2dIOn72oNGh8CxsFOph8wBMborLTn7f2ozoi8yQwaJIWIze svVSOMWPvLB567w1YuuRa8FZJ93YEncDRRLip+wFDQnFvhlKvmwnnwmfEkL6dl8xZ0WM ezJj0F/+YSzRAMvbvft5IVIhORfRp7CLTs+jsLoOmuHqJNS7Za+yq3d5WU8LfXn7Gsnd IsjkwzwBL5nkVhSEg14+i7sBgb7yEDKiCTYkkoZRoC6RfJ6E3FbhddB/4gohwvl43ZQ/ P1ninS3OFmD1tpe0+1auWjFwsq+QWBUOu8Xdki3/QU8JSx6I1lhysoY1DyAW+vix/k+B h8mg== X-Gm-Message-State: AHQUAuYc7uI1mIFDughRTWd1123PH0bMXCCntYCBAkQm0Y8JdMMouPTW l5S+E7isUgti6P7GJD3dWiitGQ== X-Received: by 2002:a5d:4046:: with SMTP id w6mr4707214wrp.92.1548760019712; Tue, 29 Jan 2019 03:06:59 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.06.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:06:59 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 02/14] block, bfq: avoid selecting a queue w/o budget Date: Tue, 29 Jan 2019 12:06:26 +0100 Message-Id: <20190129110638.12652-3-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To boost throughput on devices with internal queueing and in scenarios where device idling is not strictly needed, bfq immediately starts serving a new bfq_queue if the in-service bfq_queue remains without pending I/O, even if new I/O may arrive soon for the latter queue. Then, if such I/O actually arrives soon, bfq preempts the new in-service bfq_queue so as to give the previous queue a chance to go on being served (in case the previous queue should actually be the one to be served, according to its timestamps). However, the in-service bfq_queue, say Q, may also be without further budget when it remains also pending I/O. Since bfq changes budgets dynamically to fit the needs of bfq_queues, this happens more often than one may expect. If this happens, then there is no point in trying to go on serving Q when new I/O arrives for it soon: Q would be expired immediately after being selected for service. This would only cause useless overhead. This commit avoids such a useless selection. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index c7a4a15c7c19..9ea2c4f42501 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -1380,7 +1380,15 @@ static bool bfq_bfqq_update_budg_for_activation(struct bfq_data *bfqd, { struct bfq_entity *entity = &bfqq->entity; - if (bfq_bfqq_non_blocking_wait_rq(bfqq) && arrived_in_time) { + /* + * In the next compound condition, we check also whether there + * is some budget left, because otherwise there is no point in + * trying to go on serving bfqq with this same budget: bfqq + * would be expired immediately after being selected for + * service. This would only cause useless overhead. + */ + if (bfq_bfqq_non_blocking_wait_rq(bfqq) && arrived_in_time && + bfq_bfqq_budget_left(bfqq) > 0) { /* * We do not clear the flag non_blocking_wait_rq here, as * the latter is used in bfq_activate_bfqq to signal From patchwork Tue Jan 29 11:06:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156971 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523666jaa; Tue, 29 Jan 2019 03:08:12 -0800 (PST) X-Google-Smtp-Source: ALg8bN6fZjTig9U2+GS1e1eUG4w3XyuFtSWeYzBSK1Oqpwy/zJSJjJALg8bxDWaOX2YoF+IdEDmC X-Received: by 2002:a63:6b05:: with SMTP id g5mr22667161pgc.15.1548760092777; Tue, 29 Jan 2019 03:08:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760092; cv=none; d=google.com; s=arc-20160816; b=u5wcOstjnQANPOa8J8z5Si3NbdkkvemWdT5F2GUHooFS0xU6Haku7Mv2TgksC0GnjB vLFEzUzXxiUlKLRXWc18sHppnPUwoi2Wut2V/LSyR8hREQOMysUqISgtzUjbGKcruydL +J/uRwtMoNt1BY4SWBIPUOiJ1JWhHwVwGSgv3ZogNkD97Oi85iuvTOUxucNkWjoREKTN wlhZ9yf7nhTGAkwXf25xSYv4ZNj3HjmcBHFutMPki+nauNl5Ckuap5VlEOJvdAQzd7Rc f0eZ3dE4x7jA8mP2uWHkmSwifVkhy7p/kJ79bFLYNkTC9lml6/zLLrKROrebbEzk68uG l8Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=sfh0pAFFntluWsfkKiHNCWmGd04mxXjXMAOMMF2nCLo=; b=jWbZmZ2Oqvh4pbwj8lktWx4os0pjYEQXG9U4HBepIBiTkPHBb9K/CUHXqBzTGdm7LJ WdzWV3O4qvjV2xsUOo8w3vsWQij1d0OUCM9NpU6BWbJtdsgl3fTob8Yra5w0+O1hB86B A+QFlSjM5BI8cOZQlFpfej8xstD5fzNgz2XxaDhTpFmlfKlIEsIeV+cI35w95njCe1+D Ca6ui1NBCBf//M1TxEE8y9Hes7Zsd2TW5ypRXZdXMFZm6qihLjez6LHTfh09sOuWXCU5 cp4oWcXGqg6kS8Qq85cz7hwA9XYu3HpLUqzFsfmondUKT4pIIZVve/sbXwAbbmAnbSRK G42A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bbEl1Xzg; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 32si15859405plg.29.2019.01.29.03.08.12; Tue, 29 Jan 2019 03:08:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bbEl1Xzg; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727445AbfA2LHE (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:04 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:51794 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727176AbfA2LHC (ORCPT ); Tue, 29 Jan 2019 06:07:02 -0500 Received: by mail-wm1-f66.google.com with SMTP id b11so17358024wmj.1 for ; Tue, 29 Jan 2019 03:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sfh0pAFFntluWsfkKiHNCWmGd04mxXjXMAOMMF2nCLo=; b=bbEl1Xzgyv62vtDcTw804jba5WzfBOVjKpHPxC3U5g2tcPH4HKqtB+uTCqHcWc7xfx WgdKQhwnUJws1yCsxGdrqh5MOEpOUBRGDkjY7U14LxizTJ5yiae2dwFGTATB4rNJdjbr bf7Ph+MwLgcCr8DcYkS2jIlYcoixT+wlG6DFU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sfh0pAFFntluWsfkKiHNCWmGd04mxXjXMAOMMF2nCLo=; b=nWK423gzkGbBoSpUVyMYeLO90vH0t3psZYSBsay9AkJpgkySj18sAaATegAP/8dMGJ J8NTAXpB4iIfWXO6rHq5bC8bKEfHJL/R85VyH+DFPJsqlkOl0FMsUoRcgK12WWo5ipCK Qtlk3isN8+N+DFphRBkNZpNKnWxD8HH2LHO7vvDvbkWqTh2ccbWznYnvPBJTMUsbw5Hn 4zWWy64xeu4K6lXxFMfJmiFqzWdv9eyXnP60WTioqxuEpSBNr7gEATtMRqGl4+++pid7 NaHNAd70MQrU1JfYFaJB4IGETk/0WPhjaCnUaSDQA+Rl2cuYwEKWPepH354PJO6hBNBn 5XmA== X-Gm-Message-State: AJcUukffD9ogsEgrYNCWSY8wTg+M2IAlQ+m5srkZw7GMEvA+9dETC8DA 3DdyxK95TkaS7UMO8ppzsLdNsw== X-Received: by 2002:a1c:8b09:: with SMTP id n9mr20820350wmd.38.1548760020924; Tue, 29 Jan 2019 03:07:00 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.06.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:00 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 03/14] block, bfq: make sure queue budgets are not below service received Date: Tue, 29 Jan 2019 12:06:27 +0100 Message-Id: <20190129110638.12652-4-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org With some unlucky sequences of events, the function bfq_updated_next_req updates the current budget of a bfq_queue to a lower value than the service received by the queue using such a budget. Unfortunately, if this happens, then the return value of the function bfq_bfqq_budget_left becomes inconsistent. This commit solves this problem by lower-bounding the budget computed in bfq_updated_next_req to the service currently charged to the queue. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 9ea2c4f42501..b0e8006475be 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -907,8 +907,10 @@ static void bfq_updated_next_req(struct bfq_data *bfqd, */ return; - new_budget = max_t(unsigned long, bfqq->max_budget, - bfq_serv_to_charge(next_rq, bfqq)); + new_budget = max_t(unsigned long, + max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(next_rq, bfqq)), + entity->service); if (entity->budget != new_budget) { entity->budget = new_budget; bfq_log_bfqq(bfqd, bfqq, "updated next rq: new budget %lu", From patchwork Tue Jan 29 11:06:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156967 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523294jaa; Tue, 29 Jan 2019 03:07:52 -0800 (PST) X-Google-Smtp-Source: ALg8bN63jLiomdBV0sQcDgHx6WCNTH2lH7Y8yvZh8faBs8uoKYcL24QpLSIpCSd6ye2lA//BtoAD X-Received: by 2002:a62:6385:: with SMTP id x127mr26035869pfb.15.1548760071891; Tue, 29 Jan 2019 03:07:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760071; cv=none; d=google.com; s=arc-20160816; b=bYXFOC7DKvIl2WOIjky/jn0o+Bp3RmxMEgT0WDSpWB1Z1+Iz9MfztRbxi/qp6eF59f jCDHM7Y1stWD87uqbLO2uaIys5g/Y1CmK6l+00c9BqBff1SS5sT9phgvdCkZp1LWnTUw u2WEFaGF3OevzEXck8K1UDt2TU9fKsm8Vpjihf4oSxl/8nlatGO8BxdEjFUASrbWDI4k yhOm/AXsj9DuumAY3DsRwFF7gYXciVvUrYuNKteo9ztKtpX8QBFlvDrD9HaE6IZLw6cs 6e39nll6B5QJgXM+ofavmg7rUBYpM6dAOqTM1DT055rpSCCiixPnmSpi0BXDc25t/fUk Adlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NergthX5oSNRCeJESFYVsMGHKG3cwu/m8eBx+7n5JOE=; b=j9HPqQ6hXJIM3hWEtXZKBDfkxkl4Njbpbjev3N3uLOefxlPsSNHqvvxjj+zwSLE1pf 2qvejNoHVL0/4WBwZ8TLH0Ay2OMTODL2rL78FIRv6m0JORStYlYEcvpXwN0UoHwKJ/Pm uPp6e+uF4yz5mXWB8WV98EVJOsuAR5hlPziqBdr1BktRV/H67qd1+UpvK4g9AVoSrco6 pqBxG2Qp+z5Yki+668an1hHyvOWWON5lYHfItM7CKNQJnsAvsGSu2lhJvSWDoeHrKFIK ajS2MlcRo08OlOeTn8GOLt4tjgi5Opuh9NR5I5ogROm9HmbtDtqYhSI508RpxDSsQFN4 3yOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=djoXGnsK; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 87si34902407pfs.7.2019.01.29.03.07.51; Tue, 29 Jan 2019 03:07:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=djoXGnsK; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728510AbfA2LHJ (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:09 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:35731 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727193AbfA2LHD (ORCPT ); Tue, 29 Jan 2019 06:07:03 -0500 Received: by mail-wr1-f66.google.com with SMTP id 96so21587909wrb.2 for ; Tue, 29 Jan 2019 03:07:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NergthX5oSNRCeJESFYVsMGHKG3cwu/m8eBx+7n5JOE=; b=djoXGnsK11deAS6J4IpmOiUJzQK9Kh7UxFQ1sIfWXNRnsCRYdWvW0iAmNOyyvH1U5l ORyYOSUQzOHLOCqOg8RLHcMw3X44QxZocHBXJf/enpRVP2QWwBZEQTd1ADwMLsAu7YOh D74Y4PF0RgKe7o69H7P4Au761wDwXZIkzxtm4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NergthX5oSNRCeJESFYVsMGHKG3cwu/m8eBx+7n5JOE=; b=jU4PZSREA/iWQiFUfzVHXMtG1wPRUX2xXoLQuhBHhRzK5T09W0fdEe7NOySthAvUYf A8ykWXQDsHA8pLVr1VjCXVLzs3mOobUB4lSh5i7ifh0QKhyOrNEHKsvKHvtCuH3Fq7gu OKZm60oYNXx3TtYlVy1pBUSImgoPOEtQGyojdyOitqm3h4AQY48xf0dnW4D8Vsd0ezRl jQTZcn5/IWljDSEvJmFeXqKXN449GRatZCtH+wPPoe4cXmDCI64rRVxdC687wCjgrvxa hH5/CC00wKZn4tnF5/FiktiG5WL+hJHaIa/brQz4BPYf3DRgpBVJh5wq21WSeOM9/udv 0EeQ== X-Gm-Message-State: AJcUuketau9O8h7xOe+Pg1BvyHL6FjHsDi+agvWvWJupo0/5GPJzTCGR rU4wWbMFMcNK8M0yosv96VW21Q== X-Received: by 2002:adf:9361:: with SMTP id 88mr24827980wro.204.1548760022218; Tue, 29 Jan 2019 03:07:02 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:01 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 04/14] block, bfq: remove case of redirected bic from insert_request Date: Tue, 29 Jan 2019 12:06:28 +0100 Message-Id: <20190129110638.12652-5-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Before commit 18e5a57d7987 ("block, bfq: postpone rq preparation to insert or merge"), the destination queue for a request was chosen by a different hook than the one that then inserted the request. So, between the execution of the two hooks, the bic of the process generating the request could happen to be redirected to a different bfq_queue. As a consequence, the destination bfq_queue stored in the request could be wrong. Such an event does not need to ba handled any longer. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 2 -- 1 file changed, 2 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index b0e8006475be..a9275ed57726 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4633,8 +4633,6 @@ static bool __bfq_insert_request(struct bfq_data *bfqd, struct request *rq) bool waiting, idle_timer_disabled = false; if (new_bfqq) { - if (bic_to_bfqq(RQ_BIC(rq), 1) != bfqq) - new_bfqq = bic_to_bfqq(RQ_BIC(rq), 1); /* * Release the request's reference to the old bfqq * and make sure one is taken to the shared queue. From patchwork Tue Jan 29 11:06:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156969 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523507jaa; Tue, 29 Jan 2019 03:08:03 -0800 (PST) X-Google-Smtp-Source: ALg8bN6INb3vptJMdAz3XcbCCAtfTXXAGYWyeN5gOTDg7g5UdusY6BvkWqEAbj7Sh5dJ0IP0+bJj X-Received: by 2002:a62:9419:: with SMTP id m25mr26833309pfe.147.1548760083386; Tue, 29 Jan 2019 03:08:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760083; cv=none; d=google.com; s=arc-20160816; b=iYEcscLBc57UxXkrepOBt04clrC9aSU7oUsBIAroR8YaQAGZP2XOCHx/oAL0Jt04GX Qdx7e78K/ly5UP3dHhXP/aj7gmKroow2Zf1wYx7j9c5w8f4+0c150pc4RNmfbhJasc+a uIDNo7uKzra9J8/rb3L1e5atrr9udS39NadOvRCUpT6mMJAyxL1+Nt958UHh/F4SSTD3 EfwVBFSh5QVSAWiZf0EodaauodwydwHntHgmL4auOsuo/dbO6h2lSVbdI8kZnkbsRc2o OIGQk52G9+nAAGkJlNBcJwCpSxhtN6VegRJKnrAaup+F3KFoHFc5kCxmET5pwas4e3BM qI8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pqaJMhHhfa+weQuFHVAWQG1S7lc+lO9DghKMucIuInE=; b=il5/27/pp09RlHG4yAIegz3pwaeOzR1a4W6uTRS3JMvJkzMhaeQwLCpob5CfAf1JaA 0HjEpc5GKsx3BeuXUWSJy5Jt0Mcm7Ig8m7EWI0FvRM5opHkbrgsrlWPqBpre2ifmGjzv J1D23giYSeUK4cuFOOJotimtflgCb/oAxDBxbHlxQhRbu7u7FJ+1F+T3QvRIWHu0YwZQ bdsQA+KmBM3Artn9OU1WlKXHlZBLzMmFXVLzUUC7KgpKnuwoYGTEt5DldGcMZRS6NC3l vcnHHmkkiQOT7u8LkUGj3PF4VoS+/orc8nesYPI+k5KwWtcFoqJiOdjmgGoD4I5zahW9 VMGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQyLqjJk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a9si12465644plp.323.2019.01.29.03.08.03; Tue, 29 Jan 2019 03:08:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQyLqjJk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728698AbfA2LIB (ORCPT + 31 others); Tue, 29 Jan 2019 06:08:01 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:53593 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727176AbfA2LHG (ORCPT ); Tue, 29 Jan 2019 06:07:06 -0500 Received: by mail-wm1-f66.google.com with SMTP id d15so17344477wmb.3 for ; Tue, 29 Jan 2019 03:07:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pqaJMhHhfa+weQuFHVAWQG1S7lc+lO9DghKMucIuInE=; b=RQyLqjJkLnXNErvrLwkz6/bNzX/uXTTeUHDCjG6NXq6xDaRWgLdnjLRygLiCuODTsx L/FBcJHYMh6Rj7nfy/e3yHNwqE1Q+nk3jp99ALN6zDKFZjcTqsYZJWJ98u0kfWiryCFU QNwJf/9aBcD34eR7Eti129p1MBKHZz7TcmHIw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pqaJMhHhfa+weQuFHVAWQG1S7lc+lO9DghKMucIuInE=; b=n7Af2xceE8TIhxwH+0trTf2IqIcEh18SCJEMh7RgsVrWZvJtmYy3onttQ7UUjiCdqy IIP9ER/Vi4kvWRlpQivNs1h81jmGp5F8BexjH7VveUSfDdRjznQ+tuZKxTGO1Vqr0ERM jXrwctywewoBjXBYR8r1GonnwKAttXLSxsqhnIvdxlAvG6YtxiiyRrg0q5F9QTYPvykO jDlzfU0hcR21TlqMvgoujP8lS1EjLBfuwUnIENTCoVrJFo9+PrT6T9WxXt7NmAzEGJyC zlo46VOo5iWkYUkKQnqb6ojajke5oRPlTXPE8GmdpinN8COfKcDll1GKzUnimONFdeCK Q//w== X-Gm-Message-State: AHQUAuYqVZF+sPkA0Qe1U3//aInAxYpwZwP8PngICeALaaetxpNjy2R2 y1pjXWOEO+UUQZC6D3nkJB+hdBdf47Y= X-Received: by 2002:a1c:4c9:: with SMTP id 192mr10565730wme.135.1548760023620; Tue, 29 Jan 2019 03:07:03 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:02 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 05/14] block, bfq: consider also ioprio classes in symmetry detection Date: Tue, 29 Jan 2019 12:06:29 +0100 Message-Id: <20190129110638.12652-6-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In asymmetric scenarios, i.e., when some bfq_queue or bfq_group needs to be guaranteed a different bandwidth than other bfq_queues or bfq_groups, these service guaranteed can be provided only by plugging I/O dispatch, completely or partially, when the queue in service remains temporarily empty. A case where asymmetry is particularly strong is when some active bfq_queues belong to a higher-priority class than some other active bfq_queues. Unfortunately, this important case is not considered at all in the code for detecting asymmetric scenarios. This commit adds the missing logic. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 86 ++++++++++++++++++++++++--------------------- block/bfq-iosched.h | 8 +++-- block/bfq-wf2q.c | 12 +++++-- 3 files changed, 59 insertions(+), 47 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index a9275ed57726..6bfbfa65610b 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -623,26 +623,6 @@ void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) bfqq->pos_root = NULL; } -/* - * Tell whether there are active queues with different weights or - * active groups. - */ -static bool bfq_varied_queue_weights_or_active_groups(struct bfq_data *bfqd) -{ - /* - * For queue weights to differ, queue_weights_tree must contain - * at least two nodes. - */ - return (!RB_EMPTY_ROOT(&bfqd->queue_weights_tree) && - (bfqd->queue_weights_tree.rb_node->rb_left || - bfqd->queue_weights_tree.rb_node->rb_right) -#ifdef CONFIG_BFQ_GROUP_IOSCHED - ) || - (bfqd->num_groups_with_pending_reqs > 0 -#endif - ); -} - /* * The following function returns true if every queue must receive the * same share of the throughput (this condition is used when deciding @@ -651,25 +631,48 @@ static bool bfq_varied_queue_weights_or_active_groups(struct bfq_data *bfqd) * * Such a scenario occurs when: * 1) all active queues have the same weight, - * 2) all active groups at the same level in the groups tree have the same - * weight, + * 2) all active queues belong to the same I/O-priority class, * 3) all active groups at the same level in the groups tree have the same + * weight, + * 4) all active groups at the same level in the groups tree have the same * number of children. * * Unfortunately, keeping the necessary state for evaluating exactly * the last two symmetry sub-conditions above would be quite complex - * and time consuming. Therefore this function evaluates, instead, - * only the following stronger two sub-conditions, for which it is + * and time consuming. Therefore this function evaluates, instead, + * only the following stronger three sub-conditions, for which it is * much easier to maintain the needed state: * 1) all active queues have the same weight, - * 2) there are no active groups. + * 2) all active queues belong to the same I/O-priority class, + * 3) there are no active groups. * In particular, the last condition is always true if hierarchical * support or the cgroups interface are not enabled, thus no state * needs to be maintained in this case. */ static bool bfq_symmetric_scenario(struct bfq_data *bfqd) { - return !bfq_varied_queue_weights_or_active_groups(bfqd); + /* + * For queue weights to differ, queue_weights_tree must contain + * at least two nodes. + */ + bool varied_queue_weights = !RB_EMPTY_ROOT(&bfqd->queue_weights_tree) && + (bfqd->queue_weights_tree.rb_node->rb_left || + bfqd->queue_weights_tree.rb_node->rb_right); + + bool multiple_classes_busy = + (bfqd->busy_queues[0] && bfqd->busy_queues[1]) || + (bfqd->busy_queues[0] && bfqd->busy_queues[2]) || + (bfqd->busy_queues[1] && bfqd->busy_queues[2]); + + /* + * For queue weights to differ, queue_weights_tree must contain + * at least two nodes. + */ + return !(varied_queue_weights || multiple_classes_busy +#ifdef BFQ_GROUP_IOSCHED_ENABLED + || bfqd->num_groups_with_pending_reqs > 0 +#endif + ); } /* @@ -728,15 +731,14 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, /* * In the unlucky event of an allocation failure, we just * exit. This will cause the weight of queue to not be - * considered in bfq_varied_queue_weights_or_active_groups, - * which, in its turn, causes the scenario to be deemed - * wrongly symmetric in case bfqq's weight would have been - * the only weight making the scenario asymmetric. On the - * bright side, no unbalance will however occur when bfqq - * becomes inactive again (the invocation of this function - * is triggered by an activation of queue). In fact, - * bfq_weights_tree_remove does nothing if - * !bfqq->weight_counter. + * considered in bfq_symmetric_scenario, which, in its turn, + * causes the scenario to be deemed wrongly symmetric in case + * bfqq's weight would have been the only weight making the + * scenario asymmetric. On the bright side, no unbalance will + * however occur when bfqq becomes inactive again (the + * invocation of this function is triggered by an activation + * of queue). In fact, bfq_weights_tree_remove does nothing + * if !bfqq->weight_counter. */ if (unlikely(!bfqq->weight_counter)) return; @@ -2227,7 +2229,7 @@ bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, return NULL; /* If there is only one backlogged queue, don't search. */ - if (bfqd->busy_queues == 1) + if (bfq_tot_busy_queues(bfqd) == 1) return NULL; in_service_bfqq = bfqd->in_service_queue; @@ -3681,7 +3683,8 @@ static bool bfq_better_to_idle(struct bfq_queue *bfqq) * the requests already queued in the device have been served. */ asymmetric_scenario = (bfqq->wr_coeff > 1 && - bfqd->wr_busy_queues < bfqd->busy_queues) || + bfqd->wr_busy_queues < + bfq_tot_busy_queues(bfqd)) || !bfq_symmetric_scenario(bfqd); /* @@ -3960,7 +3963,7 @@ static struct request *bfq_dispatch_rq_from_bfqq(struct bfq_data *bfqd, * belongs to CLASS_IDLE and other queues are waiting for * service. */ - if (!(bfqd->busy_queues > 1 && bfq_class_idle(bfqq))) + if (!(bfq_tot_busy_queues(bfqd) > 1 && bfq_class_idle(bfqq))) goto return_rq; bfq_bfqq_expire(bfqd, bfqq, false, BFQQE_BUDGET_EXHAUSTED); @@ -3978,7 +3981,7 @@ static bool bfq_has_work(struct blk_mq_hw_ctx *hctx) * most a call to dispatch for nothing */ return !list_empty_careful(&bfqd->dispatch) || - bfqd->busy_queues > 0; + bfq_tot_busy_queues(bfqd) > 0; } static struct request *__bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) @@ -4032,9 +4035,10 @@ static struct request *__bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) goto start_rq; } - bfq_log(bfqd, "dispatch requests: %d busy queues", bfqd->busy_queues); + bfq_log(bfqd, "dispatch requests: %d busy queues", + bfq_tot_busy_queues(bfqd)); - if (bfqd->busy_queues == 0) + if (bfq_tot_busy_queues(bfqd) == 0) goto exit; /* diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 0b02bf302de0..30be669be465 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -501,10 +501,11 @@ struct bfq_data { unsigned int num_groups_with_pending_reqs; /* - * Number of bfq_queues containing requests (including the - * queue in service, even if it is idling). + * Per-class (RT, BE, IDLE) number of bfq_queues containing + * requests (including the queue in service, even if it is + * idling). */ - int busy_queues; + unsigned int busy_queues[3]; /* number of weight-raised busy @bfq_queues */ int wr_busy_queues; /* number of queued requests */ @@ -974,6 +975,7 @@ extern struct blkcg_policy blkcg_policy_bfq; struct bfq_group *bfq_bfqq_to_bfqg(struct bfq_queue *bfqq); struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity); +unsigned int bfq_tot_busy_queues(struct bfq_data *bfqd); struct bfq_service_tree *bfq_entity_service_tree(struct bfq_entity *entity); struct bfq_entity *bfq_entity_of(struct rb_node *node); unsigned short bfq_ioprio_to_weight(int ioprio); diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index 72adbbe975d5..ce37d709a34f 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -44,6 +44,12 @@ static unsigned int bfq_class_idx(struct bfq_entity *entity) BFQ_DEFAULT_GRP_CLASS - 1; } +unsigned int bfq_tot_busy_queues(struct bfq_data *bfqd) +{ + return bfqd->busy_queues[0] + bfqd->busy_queues[1] + + bfqd->busy_queues[2]; +} + static struct bfq_entity *bfq_lookup_next_entity(struct bfq_sched_data *sd, bool expiration); @@ -1513,7 +1519,7 @@ struct bfq_queue *bfq_get_next_queue(struct bfq_data *bfqd) struct bfq_sched_data *sd; struct bfq_queue *bfqq; - if (bfqd->busy_queues == 0) + if (bfq_tot_busy_queues(bfqd) == 0) return NULL; /* @@ -1665,7 +1671,7 @@ void bfq_del_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq, bfq_clear_bfqq_busy(bfqq); - bfqd->busy_queues--; + bfqd->busy_queues[bfqq->ioprio_class - 1]--; if (!bfqq->dispatched) bfq_weights_tree_remove(bfqd, bfqq); @@ -1688,7 +1694,7 @@ void bfq_add_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq) bfq_activate_bfqq(bfqd, bfqq); bfq_mark_bfqq_busy(bfqq); - bfqd->busy_queues++; + bfqd->busy_queues[bfqq->ioprio_class - 1]++; if (!bfqq->dispatched) if (bfqq->wr_coeff == 1) From patchwork Tue Jan 29 11:06:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156968 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523371jaa; Tue, 29 Jan 2019 03:07:55 -0800 (PST) X-Google-Smtp-Source: ALg8bN7YIPctNl/cO92oJIzk0XO1GVUhdFR9qT/dN7Fk8AsgQX8fVxF5WnL3jogr9d2XOkHNaRQn X-Received: by 2002:a63:5b48:: with SMTP id l8mr23379488pgm.80.1548760075192; Tue, 29 Jan 2019 03:07:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760075; cv=none; d=google.com; s=arc-20160816; b=BOY1ewjBiMR1MnbOhWBF2TdIuSI4Y5VoJROb9+AmhHQXO9qRLhqWLJ+BobymCY2LED 0qNQ3/EE6kyINH1WDp2uLguhApIR8tXMwObKG9HYthVgB/jZk4vo6jLcKuBQYWc5YekJ p326uhenE50EHIUpfLAICAozAR7BJjJmdG3W3Fhv0GjjRv3YcLNubxWc4awhBoYqicWI rywtu420B6lUF9wObE8iqluEtPrSCPGTsNg5920TRwCQ1f3JcVCATFNom+kjjg/uk9xJ E0werySw9ZG8dqZOSBEmmH4immeooN3Yj/fYlKSstwVC8ncDKpMvW5RsnhgdqHvfSOds mK8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QUU3CUC8cc8cju82LpP1ETMRx5xYgBZFuzHMif5PKgE=; b=PxJ6UZG8YDFDtCoVsoZccK8yIzlMEtmblZuCMcUhOaFUIKdco/9+UyMe3iXVe4y3fG BtT8YHc/FdyI3YokMUOWsLC1unge5rqoIKM66YJvuYVAZWlDrkkJn8kX57oqwmzeAZRE ic4rghuh5GZPRq47l4Q1Sp8Mfc30MiQFb/zx2sg9FhTe/GkDDV6831/sss9VUx4fLOS8 KV1KHjyI9PsXBbTKl1Jyyfr4cOibYeDbf3p/vPd+h6gnLUXGhF5a9jYjlqCzKUBRbK9v QbklnujQF9YDuq8je2CxpQhMLBAV8cZMef1dfxcXxO5WtjTjOd/xrgmstkHsjL2MOvcZ 1O+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I7GipiBy; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b5si14103712pfg.121.2019.01.29.03.07.54; Tue, 29 Jan 2019 03:07:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I7GipiBy; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728686AbfA2LHw (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:52 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:52742 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726893AbfA2LHI (ORCPT ); Tue, 29 Jan 2019 06:07:08 -0500 Received: by mail-wm1-f67.google.com with SMTP id m1so17344315wml.2 for ; Tue, 29 Jan 2019 03:07:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QUU3CUC8cc8cju82LpP1ETMRx5xYgBZFuzHMif5PKgE=; b=I7GipiBy1jwfZzcBnanIeEjUnph/c3fRUsd0smPewaxwZgXl13jxCG7lJDGG7pRnWe YnObOi6zGYuBEL6aqupczOMweeg9M3PBxM3SgnsWgpKWDIyhQRz6U+vCKr0Ur6gjFbxH jvRqkwyKPT6zMzU2EqfnFgpdeCH0endlgmAd4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QUU3CUC8cc8cju82LpP1ETMRx5xYgBZFuzHMif5PKgE=; b=kXEXWYB4BeDfD4x4c3qoNcYuEwESkD1GaDBrFIie/dA7CKMWTZAvGndWwGsJpvbqF+ X+0riEpTYHL0NMR5qaHUbA+IKbxeZArgkB0/VWH1xbP5WubYJILcerbxO+DwhlzXRWvI 9yYudPaDDxRHy3o9qYADRGURk7HP2U++Sj6O57eegrL10qVz0a6NNe6fh8v/BtX+eYWl +vQT/Mibi6BX5iIVjpgMSZJ892xvMUdNisNMnENYm7j/z0gZudyh+fK8tZWOwWF6kFxz n6gNtqO0yr3Q4SVuDd8VTDU8QW7MShCcr2JhCX6mE2NYLLHqEh8FcO83hUUVkAigDeWB Q51A== X-Gm-Message-State: AJcUukeoozIfUM6l82tGSROWuCxcfgXWMdS2phepiPS0oq7yFcWERbq1 96RSQi53MEGmijTP5lkKuNCBEg== X-Received: by 2002:a1c:2457:: with SMTP id k84mr20732866wmk.139.1548760024837; Tue, 29 Jan 2019 03:07:04 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:04 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 06/14] block, bfq: split function bfq_better_to_idle Date: Tue, 29 Jan 2019 12:06:30 +0100 Message-Id: <20190129110638.12652-7-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a preparatory commit for commits that need to check only one of the two main reasons for idling. This change should also improve the quality of the code a little bit, by splitting a function that contains very long, non-trivial and little related comments. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 155 +++++++++++++++++++++++--------------------- 1 file changed, 82 insertions(+), 73 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 6bfbfa65610b..2756f4b1432b 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -3404,53 +3404,13 @@ static bool bfq_may_expire_for_budg_timeout(struct bfq_queue *bfqq) bfq_bfqq_budget_timeout(bfqq); } -/* - * For a queue that becomes empty, device idling is allowed only if - * this function returns true for the queue. As a consequence, since - * device idling plays a critical role in both throughput boosting and - * service guarantees, the return value of this function plays a - * critical role in both these aspects as well. - * - * In a nutshell, this function returns true only if idling is - * beneficial for throughput or, even if detrimental for throughput, - * idling is however necessary to preserve service guarantees (low - * latency, desired throughput distribution, ...). In particular, on - * NCQ-capable devices, this function tries to return false, so as to - * help keep the drives' internal queues full, whenever this helps the - * device boost the throughput without causing any service-guarantee - * issue. - * - * In more detail, the return value of this function is obtained by, - * first, computing a number of boolean variables that take into - * account throughput and service-guarantee issues, and, then, - * combining these variables in a logical expression. Most of the - * issues taken into account are not trivial. We discuss these issues - * individually while introducing the variables. - */ -static bool bfq_better_to_idle(struct bfq_queue *bfqq) +static bool idling_boosts_thr_without_issues(struct bfq_data *bfqd, + struct bfq_queue *bfqq) { - struct bfq_data *bfqd = bfqq->bfqd; bool rot_without_queueing = !blk_queue_nonrot(bfqd->queue) && !bfqd->hw_tag, bfqq_sequential_and_IO_bound, - idling_boosts_thr, idling_boosts_thr_without_issues, - idling_needed_for_service_guarantees, - asymmetric_scenario; - - if (bfqd->strict_guarantees) - return true; - - /* - * Idling is performed only if slice_idle > 0. In addition, we - * do not idle if - * (a) bfqq is async - * (b) bfqq is in the idle io prio class: in this case we do - * not idle because we want to minimize the bandwidth that - * queues in this class can steal to higher-priority queues - */ - if (bfqd->bfq_slice_idle == 0 || !bfq_bfqq_sync(bfqq) || - bfq_class_idle(bfqq)) - return false; + idling_boosts_thr; bfqq_sequential_and_IO_bound = !BFQQ_SEEKY(bfqq) && bfq_bfqq_IO_bound(bfqq) && bfq_bfqq_has_short_ttime(bfqq); @@ -3482,8 +3442,7 @@ static bool bfq_better_to_idle(struct bfq_queue *bfqq) bfqq_sequential_and_IO_bound); /* - * The value of the next variable, - * idling_boosts_thr_without_issues, is equal to that of + * The return value of this function is equal to that of * idling_boosts_thr, unless a special case holds. In this * special case, described below, idling may cause problems to * weight-raised queues. @@ -3500,32 +3459,35 @@ static bool bfq_better_to_idle(struct bfq_queue *bfqq) * which enqueue several requests in advance, and further * reorder internally-queued requests. * - * For this reason, we force to false the value of - * idling_boosts_thr_without_issues if there are weight-raised - * busy queues. In this case, and if bfqq is not weight-raised, - * this guarantees that the device is not idled for bfqq (if, - * instead, bfqq is weight-raised, then idling will be - * guaranteed by another variable, see below). Combined with - * the timestamping rules of BFQ (see [1] for details), this - * behavior causes bfqq, and hence any sync non-weight-raised - * queue, to get a lower number of requests served, and thus - * to ask for a lower number of requests from the request - * pool, before the busy weight-raised queues get served - * again. This often mitigates starvation problems in the - * presence of heavy write workloads and NCQ, thereby - * guaranteeing a higher application and system responsiveness - * in these hostile scenarios. + * For this reason, we force to false the return value if + * there are weight-raised busy queues. In this case, and if + * bfqq is not weight-raised, this guarantees that the device + * is not idled for bfqq (if, instead, bfqq is weight-raised, + * then idling will be guaranteed by another variable, see + * below). Combined with the timestamping rules of BFQ (see + * [1] for details), this behavior causes bfqq, and hence any + * sync non-weight-raised queue, to get a lower number of + * requests served, and thus to ask for a lower number of + * requests from the request pool, before the busy + * weight-raised queues get served again. This often mitigates + * starvation problems in the presence of heavy write + * workloads and NCQ, thereby guaranteeing a higher + * application and system responsiveness in these hostile + * scenarios. */ - idling_boosts_thr_without_issues = idling_boosts_thr && + return idling_boosts_thr && bfqd->wr_busy_queues == 0; +} +static bool idling_needed_for_service_guarantees(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ /* - * There is then a case where idling must be performed not - * for throughput concerns, but to preserve service - * guarantees. + * There is a case where idling must be performed not for + * throughput concerns, but to preserve service guarantees. * * To introduce this case, we can note that allowing the drive - * to enqueue more than one request at a time, and hence + * to enqueue more than one request at a time, and thereby * delegating de facto final scheduling decisions to the * drive's internal scheduler, entails loss of control on the * actual request service order. In particular, the critical @@ -3682,9 +3644,9 @@ static bool bfq_better_to_idle(struct bfq_queue *bfqq) * to let requests be served in the desired order until all * the requests already queued in the device have been served. */ - asymmetric_scenario = (bfqq->wr_coeff > 1 && - bfqd->wr_busy_queues < - bfq_tot_busy_queues(bfqd)) || + bool asymmetric_scenario = (bfqq->wr_coeff > 1 && + bfqd->wr_busy_queues < + bfq_tot_busy_queues(bfqd)) || !bfq_symmetric_scenario(bfqd); /* @@ -3701,17 +3663,64 @@ static bool bfq_better_to_idle(struct bfq_queue *bfqq) * now establish when idling is actually needed to preserve * service guarantees. */ - idling_needed_for_service_guarantees = - asymmetric_scenario && !bfq_bfqq_in_large_burst(bfqq); + return asymmetric_scenario && !bfq_bfqq_in_large_burst(bfqq); +} + +/* + * For a queue that becomes empty, device idling is allowed only if + * this function returns true for that queue. As a consequence, since + * device idling plays a critical role for both throughput boosting + * and service guarantees, the return value of this function plays a + * critical role as well. + * + * In a nutshell, this function returns true only if idling is + * beneficial for throughput or, even if detrimental for throughput, + * idling is however necessary to preserve service guarantees (low + * latency, desired throughput distribution, ...). In particular, on + * NCQ-capable devices, this function tries to return false, so as to + * help keep the drives' internal queues full, whenever this helps the + * device boost the throughput without causing any service-guarantee + * issue. + * + * Most of the issues taken into account to get the return value of + * this function are not trivial. We discuss these issues in the two + * functions providing the main pieces of information needed by this + * function. + */ +static bool bfq_better_to_idle(struct bfq_queue *bfqq) +{ + struct bfq_data *bfqd = bfqq->bfqd; + bool idling_boosts_thr_with_no_issue, idling_needed_for_service_guar; + + if (unlikely(bfqd->strict_guarantees)) + return true; + + /* + * Idling is performed only if slice_idle > 0. In addition, we + * do not idle if + * (a) bfqq is async + * (b) bfqq is in the idle io prio class: in this case we do + * not idle because we want to minimize the bandwidth that + * queues in this class can steal to higher-priority queues + */ + if (bfqd->bfq_slice_idle == 0 || !bfq_bfqq_sync(bfqq) || + bfq_class_idle(bfqq)) + return false; + + idling_boosts_thr_with_no_issue = + idling_boosts_thr_without_issues(bfqd, bfqq); + + idling_needed_for_service_guar = + idling_needed_for_service_guarantees(bfqd, bfqq); /* - * We have now all the components we need to compute the + * We have now the two components we need to compute the * return value of the function, which is true only if idling * either boosts the throughput (without issues), or is * necessary to preserve service guarantees. */ - return idling_boosts_thr_without_issues || - idling_needed_for_service_guarantees; + return idling_boosts_thr_with_no_issue || + idling_needed_for_service_guar; } /* From patchwork Tue Jan 29 11:06:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156959 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522581jaa; Tue, 29 Jan 2019 03:07:12 -0800 (PST) X-Google-Smtp-Source: ALg8bN5c3u8lxF4dSCzzIU6Lebs3pYTGPdE4lrPBK0qe0sjUA2mI8MlNL/IGf2eWptWNHdcDxWwa X-Received: by 2002:a63:441e:: with SMTP id r30mr23552491pga.128.1548760031948; Tue, 29 Jan 2019 03:07:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760031; cv=none; d=google.com; s=arc-20160816; b=MD0qoUnigkBrwaxypO4K5oIB+Aw26ELk0U6EyszEijXSZjkC1CygbJTMqrwQct1BPO jpptLEZ66pfoPSFccBeZ3AgkIyV0jYIC6D4zWSVXqbvmRmJOtKRQ7qfAqfDFn6Rg+0J3 Wk0Hh2Mq7SxKCbvG57d4LjnYFtRYJ7QzCUKf6BsytxuS54lR4mnZtBtLxZBrj1IQFXC+ 3Ua46aWm8puS65sZu9BsH0CvN9ROKDhViz803/6o0Rnclnt6cxSDzUVUYUAn0rE5LAnd e4XHGXgHeYi0bMh0VP3+B5PfU+MjYm1ZQHlNu+BjO7GbuABX/Q8Xbyhocs3u7PvWs4+n jRsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aWB7QVcaljANFueMNZpw7vbeVw33aKruWnTjcF4CgUA=; b=CkXM2ADUPL3YjbUGoSWECtgqwnCAVzSaYQ+rN4DPGdx76uR47o5fiRH8tYcRqxvu5a k57HdYJcWxPrBqtPz6RAOPPIunKw9NjvGly+a9mEcdnDQQ2lW/EekHwZJecUAKhXoxvx lTyN3Rn5lmk0FvDSO28pku1iqWeKSZsqZ9NozhZ63c2pntn8T8auhj+RkBppjb97vcLZ 8E1PVs11bWpxmIe145A5ZWUefsNptvogIq/jRv7+xyX1xvodg32YQDzl8+Odw5eJMRJT YjxpdzXstV900R38LbiWfFoUht2P7sci4Wnhx7SZvNbv7CwiAY+tvcxmABxtQEkXYhQ2 srDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kM1EsQuH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o33si36301688pld.121.2019.01.29.03.07.11; Tue, 29 Jan 2019 03:07:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kM1EsQuH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728562AbfA2LHK (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:10 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:33676 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728367AbfA2LHH (ORCPT ); Tue, 29 Jan 2019 06:07:07 -0500 Received: by mail-wm1-f68.google.com with SMTP id r24so12685267wmh.0 for ; Tue, 29 Jan 2019 03:07:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aWB7QVcaljANFueMNZpw7vbeVw33aKruWnTjcF4CgUA=; b=kM1EsQuHwH9AJMB8rxg28Ik28/xZAi7sJvABLEcNMGaHmqVDqRJJR82Tjy6I/OcK93 inMy91trQkSDaTol6pj/Aau2E+kAh+mmQ/PWw1ScIHdN1+JKhHr4oonLBu07wLNa0rgC zGrqDdox+p4OWTyHWlKNcD6uirgyB/rljFY6E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aWB7QVcaljANFueMNZpw7vbeVw33aKruWnTjcF4CgUA=; b=FWFssse5gPiWjdb5aGLcNFNW/0skxII6L6TrRdRnHn0nVOcWxkVJ6wmou1JpCAtSh0 BGnE9OkZmdwE0PYvwH5s72qzhNNJ6dVCiGSqkVbvIKdL413nB+SAhJvVkEtrmcio4tkc M1f8vmY7ywg6Kn30AlPAZDdaGxltIeR9kF8ZKMa5mMFV0BV6Fmh5vWFNSxz6ivdVhaHg PIWFQ5UQkLtPltMjq72SEkVKJio7n4ndd/lbVs70rLX/URuAuA5SNDXR3brWYCU+Ra8P LNi1Zcga81SPD3TUVbTtCUJOHX61FWBLEQjYo2paxSgKdyF1pJtoV8A8B+lwNUYlBNkx htZA== X-Gm-Message-State: AJcUukfuTd+ZbDUc4cZ4IuH1M447yGLoh32M/qyaUrLooSoaNrXGoXE9 XjFyQwNso/KEuJiCvu1ROHe0wg== X-Received: by 2002:a1c:6243:: with SMTP id w64mr20309254wmb.153.1548760026112; Tue, 29 Jan 2019 03:07:06 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:05 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 07/14] block, bfq: do not plug I/O of in-service queue when harmful Date: Tue, 29 Jan 2019 12:06:31 +0100 Message-Id: <20190129110638.12652-8-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If the in-service bfq_queue is sync and remains temporarily idle, then I/O dispatching (from other queues) may be plugged. It may be dome for two reasons: either to boost throughput, or to preserve the bandwidth share of the in-service queue. In the first case, if the I/O of the in-service queue, when it finally arrives, consists only of one small I/O request, then it makes sense to plug even the I/O of the in-service queue. In fact, serving such a small request immediately is likely to lower throughput instead of boosting it, whereas waiting a little bit is likely to let that request grow, thanks to request merging, and become more profitable in terms of throughput (this is likely to happen exactly because the I/O of the queue has been detected to boost throughput). On the opposite end, if I/O dispatching is being plugged only to preserve the bandwidth of the in-service queue, then it would be better not to plug also the I/O of the in-service queue, because such a plugging is likely to cause only loss of bandwidth for the queue. Unfortunately, no distinction is made between the two cases, and the I/O of the in-service queue is always plugged in case just a small I/O request arrives. This commit draws this missing distinction and does not perform harmful plugging. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2756f4b1432b..a6fe60114ade 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4599,28 +4599,31 @@ static void bfq_rq_enqueued(struct bfq_data *bfqd, struct bfq_queue *bfqq, bool budget_timeout = bfq_bfqq_budget_timeout(bfqq); /* - * There is just this request queued: if the request - * is small and the queue is not to be expired, then - * just exit. + * There is just this request queued: if + * - the request is small, and + * - we are idling to boost throughput, and + * - the queue is not to be expired, + * then just exit. * * In this way, if the device is being idled to wait * for a new request from the in-service queue, we * avoid unplugging the device and committing the - * device to serve just a small request. On the - * contrary, we wait for the block layer to decide - * when to unplug the device: hopefully, new requests - * will be merged to this one quickly, then the device - * will be unplugged and larger requests will be - * dispatched. + * device to serve just a small request. In contrast + * we wait for the block layer to decide when to + * unplug the device: hopefully, new requests will be + * merged to this one quickly, then the device will be + * unplugged and larger requests will be dispatched. */ - if (small_req && !budget_timeout) + if (small_req && idling_boosts_thr_without_issues(bfqd, bfqq) && + !budget_timeout) return; /* - * A large enough request arrived, or the queue is to - * be expired: in both cases disk idling is to be - * stopped, so clear wait_request flag and reset - * timer. + * A large enough request arrived, or idling is being + * performed to preserve service guarantees, or + * finally the queue is to be expired: in all these + * cases disk idling is to be stopped, so clear + * wait_request flag and reset timer. */ bfq_clear_bfqq_wait_request(bfqq); hrtimer_try_to_cancel(&bfqd->idle_slice_timer); From patchwork Tue Jan 29 11:06:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156961 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522667jaa; Tue, 29 Jan 2019 03:07:17 -0800 (PST) X-Google-Smtp-Source: ALg8bN54axzq3ehyFMAhXZh+Hi4jhX1ibyxxtcrD3vNMV1BznpjACtwMDOLTF8inS1gPYhjTvjhE X-Received: by 2002:a65:4683:: with SMTP id h3mr22300610pgr.225.1548760037662; Tue, 29 Jan 2019 03:07:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760037; cv=none; d=google.com; s=arc-20160816; b=rMC7lx6sizsH0R8iRAgSkUP4DiMq/gadhA74F4h4XdYu+IbectohHxyG+M6asvGVtY VFjvJEPmhOB/PiRSguDNhho0vyBx0SSUsRqAXSAFfEfpu9dKEz2qSY0nO5bIX9Wp67Nm rbeLaMVQb7CM0G42AXSC4NZunlYtSf31dT/mSjApqqhybDk2ZN+WR6MTkR0Bo/KMhxeN Hexj8YR35u2q1looSP71Sl4lVwVpclM7C7imA5FmYNfm7NanHPJxZ/FZp7XAfDpSyRjK 7Y13bzA/4Mpp7zgmVq/91etybaFRi3AOZPu1Ke7Kqdg10GR3Ye0OwjWLAjuhsv42AxOK S73g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HhC/EOzI5AWKwqzzKX+9bur5mem9K87e/EC40bQoIXA=; b=fwwxkwJ4KSTOaIeaMIkcGJcq9pmfqKy01WjHWF8kVQTpz8LveTvBCecpXPvuQIWZOR RwhskaqPVwN6TMO9vw8G4rksKisRwvyJEMZ4F4WusIVedeLuKKhqCNCaUuUvjEt5lSFT N0cyohdrKp7WnZtfyK0dqYZa5Oj/kiqXmIhJg02LhO0S6inl7hWkRTJ1r/F1sy2VnhFH L5d1qkyPV/C+y/LcJTmfukZeOWWLbSo40z20w+YgWY9Vsjh5kuwpsfdCCYfxbb49xX+7 S/x8EsP5/vmB7ZMm/Ky6hbGz96TZJVpZGzqF0bFv+ZvUp0h/3sCg6nKF7197wFiPE0YF jbew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=A1zDoa5S; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id bj11si34997288plb.21.2019.01.29.03.07.17; Tue, 29 Jan 2019 03:07:17 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=A1zDoa5S; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728609AbfA2LHP (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:15 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:35163 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728474AbfA2LHL (ORCPT ); Tue, 29 Jan 2019 06:07:11 -0500 Received: by mail-wm1-f66.google.com with SMTP id t200so17405926wmt.0 for ; Tue, 29 Jan 2019 03:07:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HhC/EOzI5AWKwqzzKX+9bur5mem9K87e/EC40bQoIXA=; b=A1zDoa5SPTJfDqxoD2QjiAUeJMiNW7/5lHf+GeLqqWqmA7mtaIvcgNjz5A0uFNirMt u6l1P5tGtpxcr+ZgGqCn8lDwCW4+ptQJg0NHioR2F/kTwbnQ4Lqk5Y7QFITSuRDmVH9I QPoBTBQ6FPVpQ5doMz5yMrrC5BCECiv/i3KV0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HhC/EOzI5AWKwqzzKX+9bur5mem9K87e/EC40bQoIXA=; b=sl2Wp7wOaONYQtZz/cSGZRZTeXYJoLITyQkov+Gnh0q95iVkCLbxikiItQze7tcK04 W9+5RqlirFe42wUO+13pbb01sRYnQaznQpAvnclBuj0EpK1VSu5JIh0k26qCdFPTllVx hCLtTPocevfW/uBRND1RRt3+uZeUVfbhz0gfcM+4u7TRsig9hHbVwtlIBkjO7d4WOVse MqxQmz0Q3EEWvxZ7PUext74uzqimoDcQb+51Ou4n0vjTKOYslnBxwQZJjnFYrU/dd7uN 6k3sOFRqo04Q/KX/oaonhdOjn51Hgn6WDBWhqQSSj+4nOPlSS2FLacrUNOqpIcNU8RPi OqZg== X-Gm-Message-State: AJcUukdGzCKoic/tBtH66jRfVsve91fjbac8Er2cMTKD29FdiwoP68/6 LLTySQUf7GAX7VXChCIT8dELbg== X-Received: by 2002:a1c:de57:: with SMTP id v84mr20483408wmg.55.1548760027573; Tue, 29 Jan 2019 03:07:07 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:06 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 08/14] block, bfq: unconditionally plug I/O in asymmetric scenarios Date: Tue, 29 Jan 2019 12:06:32 +0100 Message-Id: <20190129110638.12652-9-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org bfq detects the creation of multiple bfq_queues shortly after each other, namely a burst of queue creations in the terminology used in the code. If the burst is large, then no queue in the burst is granted - either I/O-dispatch plugging when the queue remains temporarily idle while in service; - or weight raising, because it causes even longer plugging. In fact, such a plugging tends to lower throughput, while these bursts are typically due to applications or services that spawn multiple processes, to reach a common goal as soon as possible. Examples are a "git grep" or the booting of a system. Unfortunately, disabling plugging may cause a loss of service guarantees in asymmetric scenarios, i.e., if queue weights are differentiated or if more than one group is active. This commit addresses this issue by no longer disabling I/O-dispatch plugging for queues in large bursts. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 346 +++++++++++++++++++++----------------------- 1 file changed, 165 insertions(+), 181 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index a6fe60114ade..c1bb5e5fcdc4 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -3479,191 +3479,175 @@ static bool idling_boosts_thr_without_issues(struct bfq_data *bfqd, bfqd->wr_busy_queues == 0; } +/* + * There is a case where idling must be performed not for + * throughput concerns, but to preserve service guarantees. + * + * To introduce this case, we can note that allowing the drive + * to enqueue more than one request at a time, and hence + * delegating de facto final scheduling decisions to the + * drive's internal scheduler, entails loss of control on the + * actual request service order. In particular, the critical + * situation is when requests from different processes happen + * to be present, at the same time, in the internal queue(s) + * of the drive. In such a situation, the drive, by deciding + * the service order of the internally-queued requests, does + * determine also the actual throughput distribution among + * these processes. But the drive typically has no notion or + * concern about per-process throughput distribution, and + * makes its decisions only on a per-request basis. Therefore, + * the service distribution enforced by the drive's internal + * scheduler is likely to coincide with the desired + * device-throughput distribution only in a completely + * symmetric scenario where: + * (i) each of these processes must get the same throughput as + * the others; + * (ii) the I/O of each process has the same properties, in + * terms of locality (sequential or random), direction + * (reads or writes), request sizes, greediness + * (from I/O-bound to sporadic), and so on. + * In fact, in such a scenario, the drive tends to treat + * the requests of each of these processes in about the same + * way as the requests of the others, and thus to provide + * each of these processes with about the same throughput + * (which is exactly the desired throughput distribution). In + * contrast, in any asymmetric scenario, device idling is + * certainly needed to guarantee that bfqq receives its + * assigned fraction of the device throughput (see [1] for + * details). + * The problem is that idling may significantly reduce + * throughput with certain combinations of types of I/O and + * devices. An important example is sync random I/O, on flash + * storage with command queueing. So, unless bfqq falls in the + * above cases where idling also boosts throughput, it would + * be important to check conditions (i) and (ii) accurately, + * so as to avoid idling when not strictly needed for service + * guarantees. + * + * Unfortunately, it is extremely difficult to thoroughly + * check condition (ii). And, in case there are active groups, + * it becomes very difficult to check condition (i) too. In + * fact, if there are active groups, then, for condition (i) + * to become false, it is enough that an active group contains + * more active processes or sub-groups than some other active + * group. More precisely, for condition (i) to hold because of + * such a group, it is not even necessary that the group is + * (still) active: it is sufficient that, even if the group + * has become inactive, some of its descendant processes still + * have some request already dispatched but still waiting for + * completion. In fact, requests have still to be guaranteed + * their share of the throughput even after being + * dispatched. In this respect, it is easy to show that, if a + * group frequently becomes inactive while still having + * in-flight requests, and if, when this happens, the group is + * not considered in the calculation of whether the scenario + * is asymmetric, then the group may fail to be guaranteed its + * fair share of the throughput (basically because idling may + * not be performed for the descendant processes of the group, + * but it had to be). We address this issue with the + * following bi-modal behavior, implemented in the function + * bfq_symmetric_scenario(). + * + * If there are groups with requests waiting for completion + * (as commented above, some of these groups may even be + * already inactive), then the scenario is tagged as + * asymmetric, conservatively, without checking any of the + * conditions (i) and (ii). So the device is idled for bfqq. + * This behavior matches also the fact that groups are created + * exactly if controlling I/O is a primary concern (to + * preserve bandwidth and latency guarantees). + * + * On the opposite end, if there are no groups with requests + * waiting for completion, then only condition (i) is actually + * controlled, i.e., provided that condition (i) holds, idling + * is not performed, regardless of whether condition (ii) + * holds. In other words, only if condition (i) does not hold, + * then idling is allowed, and the device tends to be + * prevented from queueing many requests, possibly of several + * processes. Since there are no groups with requests waiting + * for completion, then, to control condition (i) it is enough + * to check just whether all the queues with requests waiting + * for completion also have the same weight. + * + * Not checking condition (ii) evidently exposes bfqq to the + * risk of getting less throughput than its fair share. + * However, for queues with the same weight, a further + * mechanism, preemption, mitigates or even eliminates this + * problem. And it does so without consequences on overall + * throughput. This mechanism and its benefits are explained + * in the next three paragraphs. + * + * Even if a queue, say Q, is expired when it remains idle, Q + * can still preempt the new in-service queue if the next + * request of Q arrives soon (see the comments on + * bfq_bfqq_update_budg_for_activation). If all queues and + * groups have the same weight, this form of preemption, + * combined with the hole-recovery heuristic described in the + * comments on function bfq_bfqq_update_budg_for_activation, + * are enough to preserve a correct bandwidth distribution in + * the mid term, even without idling. In fact, even if not + * idling allows the internal queues of the device to contain + * many requests, and thus to reorder requests, we can rather + * safely assume that the internal scheduler still preserves a + * minimum of mid-term fairness. + * + * More precisely, this preemption-based, idleless approach + * provides fairness in terms of IOPS, and not sectors per + * second. This can be seen with a simple example. Suppose + * that there are two queues with the same weight, but that + * the first queue receives requests of 8 sectors, while the + * second queue receives requests of 1024 sectors. In + * addition, suppose that each of the two queues contains at + * most one request at a time, which implies that each queue + * always remains idle after it is served. Finally, after + * remaining idle, each queue receives very quickly a new + * request. It follows that the two queues are served + * alternatively, preempting each other if needed. This + * implies that, although both queues have the same weight, + * the queue with large requests receives a service that is + * 1024/8 times as high as the service received by the other + * queue. + * + * The motivation for using preemption instead of idling (for + * queues with the same weight) is that, by not idling, + * service guarantees are preserved (completely or at least in + * part) without minimally sacrificing throughput. And, if + * there is no active group, then the primary expectation for + * this device is probably a high throughput. + * + * We are now left only with explaining the additional + * compound condition that is checked below for deciding + * whether the scenario is asymmetric. To explain this + * compound condition, we need to add that the function + * bfq_symmetric_scenario checks the weights of only + * non-weight-raised queues, for efficiency reasons (see + * comments on bfq_weights_tree_add()). Then the fact that + * bfqq is weight-raised is checked explicitly here. More + * precisely, the compound condition below takes into account + * also the fact that, even if bfqq is being weight-raised, + * the scenario is still symmetric if all queues with requests + * waiting for completion happen to be + * weight-raised. Actually, we should be even more precise + * here, and differentiate between interactive weight raising + * and soft real-time weight raising. + * + * As a side note, it is worth considering that the above + * device-idling countermeasures may however fail in the + * following unlucky scenario: if idling is (correctly) + * disabled in a time period during which all symmetry + * sub-conditions hold, and hence the device is allowed to + * enqueue many requests, but at some later point in time some + * sub-condition stops to hold, then it may become impossible + * to let requests be served in the desired order until all + * the requests already queued in the device have been served. + */ static bool idling_needed_for_service_guarantees(struct bfq_data *bfqd, struct bfq_queue *bfqq) { - /* - * There is a case where idling must be performed not for - * throughput concerns, but to preserve service guarantees. - * - * To introduce this case, we can note that allowing the drive - * to enqueue more than one request at a time, and thereby - * delegating de facto final scheduling decisions to the - * drive's internal scheduler, entails loss of control on the - * actual request service order. In particular, the critical - * situation is when requests from different processes happen - * to be present, at the same time, in the internal queue(s) - * of the drive. In such a situation, the drive, by deciding - * the service order of the internally-queued requests, does - * determine also the actual throughput distribution among - * these processes. But the drive typically has no notion or - * concern about per-process throughput distribution, and - * makes its decisions only on a per-request basis. Therefore, - * the service distribution enforced by the drive's internal - * scheduler is likely to coincide with the desired - * device-throughput distribution only in a completely - * symmetric scenario where: - * (i) each of these processes must get the same throughput as - * the others; - * (ii) the I/O of each process has the same properties, in - * terms of locality (sequential or random), direction - * (reads or writes), request sizes, greediness - * (from I/O-bound to sporadic), and so on. - * In fact, in such a scenario, the drive tends to treat - * the requests of each of these processes in about the same - * way as the requests of the others, and thus to provide - * each of these processes with about the same throughput - * (which is exactly the desired throughput distribution). In - * contrast, in any asymmetric scenario, device idling is - * certainly needed to guarantee that bfqq receives its - * assigned fraction of the device throughput (see [1] for - * details). - * The problem is that idling may significantly reduce - * throughput with certain combinations of types of I/O and - * devices. An important example is sync random I/O, on flash - * storage with command queueing. So, unless bfqq falls in the - * above cases where idling also boosts throughput, it would - * be important to check conditions (i) and (ii) accurately, - * so as to avoid idling when not strictly needed for service - * guarantees. - * - * Unfortunately, it is extremely difficult to thoroughly - * check condition (ii). And, in case there are active groups, - * it becomes very difficult to check condition (i) too. In - * fact, if there are active groups, then, for condition (i) - * to become false, it is enough that an active group contains - * more active processes or sub-groups than some other active - * group. More precisely, for condition (i) to hold because of - * such a group, it is not even necessary that the group is - * (still) active: it is sufficient that, even if the group - * has become inactive, some of its descendant processes still - * have some request already dispatched but still waiting for - * completion. In fact, requests have still to be guaranteed - * their share of the throughput even after being - * dispatched. In this respect, it is easy to show that, if a - * group frequently becomes inactive while still having - * in-flight requests, and if, when this happens, the group is - * not considered in the calculation of whether the scenario - * is asymmetric, then the group may fail to be guaranteed its - * fair share of the throughput (basically because idling may - * not be performed for the descendant processes of the group, - * but it had to be). We address this issue with the - * following bi-modal behavior, implemented in the function - * bfq_symmetric_scenario(). - * - * If there are groups with requests waiting for completion - * (as commented above, some of these groups may even be - * already inactive), then the scenario is tagged as - * asymmetric, conservatively, without checking any of the - * conditions (i) and (ii). So the device is idled for bfqq. - * This behavior matches also the fact that groups are created - * exactly if controlling I/O is a primary concern (to - * preserve bandwidth and latency guarantees). - * - * On the opposite end, if there are no groups with requests - * waiting for completion, then only condition (i) is actually - * controlled, i.e., provided that condition (i) holds, idling - * is not performed, regardless of whether condition (ii) - * holds. In other words, only if condition (i) does not hold, - * then idling is allowed, and the device tends to be - * prevented from queueing many requests, possibly of several - * processes. Since there are no groups with requests waiting - * for completion, then, to control condition (i) it is enough - * to check just whether all the queues with requests waiting - * for completion also have the same weight. - * - * Not checking condition (ii) evidently exposes bfqq to the - * risk of getting less throughput than its fair share. - * However, for queues with the same weight, a further - * mechanism, preemption, mitigates or even eliminates this - * problem. And it does so without consequences on overall - * throughput. This mechanism and its benefits are explained - * in the next three paragraphs. - * - * Even if a queue, say Q, is expired when it remains idle, Q - * can still preempt the new in-service queue if the next - * request of Q arrives soon (see the comments on - * bfq_bfqq_update_budg_for_activation). If all queues and - * groups have the same weight, this form of preemption, - * combined with the hole-recovery heuristic described in the - * comments on function bfq_bfqq_update_budg_for_activation, - * are enough to preserve a correct bandwidth distribution in - * the mid term, even without idling. In fact, even if not - * idling allows the internal queues of the device to contain - * many requests, and thus to reorder requests, we can rather - * safely assume that the internal scheduler still preserves a - * minimum of mid-term fairness. - * - * More precisely, this preemption-based, idleless approach - * provides fairness in terms of IOPS, and not sectors per - * second. This can be seen with a simple example. Suppose - * that there are two queues with the same weight, but that - * the first queue receives requests of 8 sectors, while the - * second queue receives requests of 1024 sectors. In - * addition, suppose that each of the two queues contains at - * most one request at a time, which implies that each queue - * always remains idle after it is served. Finally, after - * remaining idle, each queue receives very quickly a new - * request. It follows that the two queues are served - * alternatively, preempting each other if needed. This - * implies that, although both queues have the same weight, - * the queue with large requests receives a service that is - * 1024/8 times as high as the service received by the other - * queue. - * - * The motivation for using preemption instead of idling (for - * queues with the same weight) is that, by not idling, - * service guarantees are preserved (completely or at least in - * part) without minimally sacrificing throughput. And, if - * there is no active group, then the primary expectation for - * this device is probably a high throughput. - * - * We are now left only with explaining the additional - * compound condition that is checked below for deciding - * whether the scenario is asymmetric. To explain this - * compound condition, we need to add that the function - * bfq_symmetric_scenario checks the weights of only - * non-weight-raised queues, for efficiency reasons (see - * comments on bfq_weights_tree_add()). Then the fact that - * bfqq is weight-raised is checked explicitly here. More - * precisely, the compound condition below takes into account - * also the fact that, even if bfqq is being weight-raised, - * the scenario is still symmetric if all queues with requests - * waiting for completion happen to be - * weight-raised. Actually, we should be even more precise - * here, and differentiate between interactive weight raising - * and soft real-time weight raising. - * - * As a side note, it is worth considering that the above - * device-idling countermeasures may however fail in the - * following unlucky scenario: if idling is (correctly) - * disabled in a time period during which all symmetry - * sub-conditions hold, and hence the device is allowed to - * enqueue many requests, but at some later point in time some - * sub-condition stops to hold, then it may become impossible - * to let requests be served in the desired order until all - * the requests already queued in the device have been served. - */ - bool asymmetric_scenario = (bfqq->wr_coeff > 1 && - bfqd->wr_busy_queues < - bfq_tot_busy_queues(bfqd)) || + return (bfqq->wr_coeff > 1 && + bfqd->wr_busy_queues < + bfq_tot_busy_queues(bfqd)) || !bfq_symmetric_scenario(bfqd); - - /* - * Finally, there is a case where maximizing throughput is the - * best choice even if it may cause unfairness toward - * bfqq. Such a case is when bfqq became active in a burst of - * queue activations. Queues that became active during a large - * burst benefit only from throughput, as discussed in the - * comments on bfq_handle_burst. Thus, if bfqq became active - * in a burst and not idling the device maximizes throughput, - * then the device must no be idled, because not idling the - * device provides bfqq and all other queues in the burst with - * maximum benefit. Combining this and the above case, we can - * now establish when idling is actually needed to preserve - * service guarantees. - */ - return asymmetric_scenario && !bfq_bfqq_in_large_burst(bfqq); } /* From patchwork Tue Jan 29 11:06:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156960 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522625jaa; Tue, 29 Jan 2019 03:07:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN6Mgla1OKXb9U9yYX66KwyEYg/8J65OD1FIB/q1F0U/42+KOGWsAYtT+oonUm3mMNhm/Jge X-Received: by 2002:a62:f247:: with SMTP id y7mr25773047pfl.25.1548760034703; Tue, 29 Jan 2019 03:07:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760034; cv=none; d=google.com; s=arc-20160816; b=H6hp0rOoMLjLNmSRNdrAZbGRIiyyQxGj2ZCRfIK7tKr42I9fUuAewBk9WWPJYoppgI dF584yowOpIQE3Qvp1iACfnMAzsbgLIBOeE05FvNsx+kFmYnKxLKWOTvAnOOwr7DdDlp WFfZTA7DK8SublBpTLAA0TxXh4RIPJJTT98I4QSM9sNjQev7KLDMgykZYhcQAeTaTTkF TgQDlDu818dpg7amZf+12K8xXaAXYb44Qp0aMGkDWGeroIbherA5GA5oNoNplBEaUizh QXJ+ZY4IwArZutCr+8kq5m2n3FgXKzCj+rJiTYaV1ZUQjEopQfahIlQOi0i/kp4qd6lj bgmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VwIs+zonmjtiCZ1USOo/5sS6ODQShYjtQFUwoinOfyA=; b=M2inLxHKb7sT/vCoySNoQ2NihUB10KakMp9UKjK8dF4f403jNep+mZHtjiZdr1VcS5 9mZtXyBk5QJ9ItHe6lXO7u62Ln9YIaEr7QRju3o7x8fDe69IoVM2JTE/AfGezwJHYXlw en6PvTogX/W4w8HJeHpSRAWzO6L5CNQDsERNDd/vwbSKSEGG9jiRbN/JkbU0wTt6Q9Bc QQ6TOdaT8c8yGFU/sBFDOYrGho4/FXZoxZzw0N4ZFIGvwtrpz+cb3Zbr8mD7oAVB2pY/ 4kGDM9NRIT65LB2EJ10sNJgSNooADOrRDCdxU11FgOr1lkIqIBvpg+dC1uPgMnPqmCHK dVjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dzNNNTGG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j10si34363595pll.179.2019.01.29.03.07.14; Tue, 29 Jan 2019 03:07:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dzNNNTGG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728590AbfA2LHN (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:13 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:39544 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728512AbfA2LHK (ORCPT ); Tue, 29 Jan 2019 06:07:10 -0500 Received: by mail-wr1-f65.google.com with SMTP id t27so21538420wra.6 for ; Tue, 29 Jan 2019 03:07:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VwIs+zonmjtiCZ1USOo/5sS6ODQShYjtQFUwoinOfyA=; b=dzNNNTGG5PdbsAoIUTXnKMvPIzXU2GiE2VkYxsWTf2UiyCY0wriVi0q9foVNjfoQol P/FaGtCg2bXNR6bff01uCTbsm3htu93+aSrkN86YiYT7JF4IpxTl1xTYqfozHHkmz1rD xO5QtpvJKdv15+MIghqrBcKbEoaxfmx0cjXzM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VwIs+zonmjtiCZ1USOo/5sS6ODQShYjtQFUwoinOfyA=; b=t4Rl6UXFSUUT8/V1n6TcO3XdPIuFs4/NnyatPk1o4PboOGF7z4+It5YWwxBoZGy0or gIRDdlMRBJG5IfSHVGo3f97DYO2sbA8P+PG2kNiaMynwle0IRc9eXEo99iucdf+5oEtI B6Q8suEKSC/cDJrsyjdUeM2xUulMp8HvmfTIvTHVPBISqpbFsaTFwlsTLFAmQy2eTzf8 XtGO4TbSxGwLFxSLPJc4AnA5w3wvKE2mpPFLKqNTxKKGaW5ZX3BqPJRH7vEtPs2Vom6c HLFvpMZjd1dAnTJ9jVkGEcJPrJy8FkxcjKZeMJaNhfiwpSd8ltqra7mzfBkooIiLDTlq HrnQ== X-Gm-Message-State: AJcUukcJRHF05aQ71eq/1qJtaz9h7Xq4aNXnqwpIIdYIsaAaR0BMBSkh 7+SAka6VI+DRznBJRoOERBWziA== X-Received: by 2002:adf:f3c6:: with SMTP id g6mr21053279wrp.111.1548760028824; Tue, 29 Jan 2019 03:07:08 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:08 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 09/14] block, bfq: fix sequential rq detection in rate estimation Date: Tue, 29 Jan 2019 12:06:33 +0100 Message-Id: <20190129110638.12652-10-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In bfq_update_peak_rate, to check whether an I/O request rq is sequential, only the seek distance of rq w.r.t. the last request dispatched is controlled. This is not sufficient for non-rotational storage, where the size of rq is at least as relevant. This commit adds the missing control. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index c1bb5e5fcdc4..12228af16198 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -235,6 +235,11 @@ static struct kmem_cache *bfq_pool; #define BFQQ_SEEK_THR (sector_t)(8 * 100) #define BFQQ_SECT_THR_NONROT (sector_t)(2 * 32) +#define BFQ_RQ_SEEKY(bfqd, last_pos, rq) \ + (get_sdist(last_pos, rq) > \ + BFQQ_SEEK_THR && \ + (!blk_queue_nonrot(bfqd->queue) || \ + blk_rq_sectors(rq) < BFQQ_SECT_THR_NONROT)) #define BFQQ_CLOSE_THR (sector_t)(8 * 1024) #define BFQQ_SEEKY(bfqq) (hweight32(bfqq->seek_history) > 19) @@ -2754,7 +2759,7 @@ static void bfq_update_peak_rate(struct bfq_data *bfqd, struct request *rq) if ((bfqd->rq_in_driver > 0 || now_ns - bfqd->last_completion < BFQ_MIN_TT) - && get_sdist(bfqd->last_position, rq) < BFQQ_SEEK_THR) + && !BFQ_RQ_SEEKY(bfqd, bfqd->last_position, rq)) bfqd->sequential_samples++; bfqd->tot_sectors_dispatched += blk_rq_sectors(rq); @@ -4511,10 +4516,7 @@ bfq_update_io_seektime(struct bfq_data *bfqd, struct bfq_queue *bfqq, struct request *rq) { bfqq->seek_history <<= 1; - bfqq->seek_history |= - get_sdist(bfqq->last_request_pos, rq) > BFQQ_SEEK_THR && - (!blk_queue_nonrot(bfqd->queue) || - blk_rq_sectors(rq) < BFQQ_SECT_THR_NONROT); + bfqq->seek_history |= BFQ_RQ_SEEKY(bfqd, bfqq->last_request_pos, rq); } static void bfq_update_has_short_ttime(struct bfq_data *bfqd, From patchwork Tue Jan 29 11:06:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156966 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523200jaa; Tue, 29 Jan 2019 03:07:44 -0800 (PST) X-Google-Smtp-Source: ALg8bN5hA/yC7n3qc8QCTqgXPrjyjoGHURNOP7gdjEScg1Mg5ZfF9lKfpiUiloivJUSPfM560neB X-Received: by 2002:a17:902:b48b:: with SMTP id y11mr24570197plr.200.1548760064209; Tue, 29 Jan 2019 03:07:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760064; cv=none; d=google.com; s=arc-20160816; b=lUspsQomGqwzwzkXsXbc/B9GjaeiJt2Nefb3gy+/9T835EDoHALJCCJFomNVmrlOdR tAWfNEPK/TZ6OqxdGHpIhZC23fOGmQAXwbSKpIk89eVSWqdEy7qitMLbRsvAjTiui1Dw +4y9n6RE9o63pOLb/FnTBbaBicTStuLAGric1DNNJ+/7dASsmUdbi8wySiyKKBh1VDFA QLO4pS0fmWjJXL81iuPgQB/JyW4IFJ1WvUV5CsXjLjBbT6vs8LBRx1Wzrf3H6ZAlAbGW I2CQwDljPkXGxPQCZ4kIF+7vKWn9qcMfaUX4k9qqP7/iwzTEfsfJF2sWivQDe66dHeq6 7glg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ui0RozgVj782eZenovCwcdQYstEPpPQMeO8MT0fjT5o=; b=J3XWo+la9ehanyrB1I8RCY3lxMOUweBpcaXfLsaaHGVBjHdEXX4BHvjC3TeONLaKm0 HK/YgBVtoYpW2wKw9rK0PsFOQTcMt20qHXcoeFMGo/xRylfXTOnzn9suFJr5xMV3hbNQ 5r+vebmDMecFe56Eril4tta2yIP11We9dEJejSzvO4KZrC6jy79hmNjtR56JWNpNp2WE 5ZNMHt61kI4yXy1PZeNeiD93zlwzGya7uAGUtB5WL7bwqmFrAUDJXyHPvUsI8E/e4dE0 wE7Km4trIiw0LxSnN1+uOCFV/RLcjoOn4zXwAO4VEf0RZ2oCrDfB/VZtdZ4DQ6JrHzz/ fG6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L/8rgA1a"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o3si35074388pgq.139.2019.01.29.03.07.43; Tue, 29 Jan 2019 03:07:44 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L/8rgA1a"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728681AbfA2LHm (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:42 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:45596 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727728AbfA2LHL (ORCPT ); Tue, 29 Jan 2019 06:07:11 -0500 Received: by mail-wr1-f66.google.com with SMTP id t6so21506279wrr.12 for ; Tue, 29 Jan 2019 03:07:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ui0RozgVj782eZenovCwcdQYstEPpPQMeO8MT0fjT5o=; b=L/8rgA1a0X/Tzr6jFJ7LPCtro61Yhi/kU9AXGI8zscrdDCQQoDzxOAgtOlxQEJmkqQ Im3eeKa7fN2xw9GQceKjWdHYXTC11GBPY67zB3gdzRXMhhAjAD6CISk+xyKiBEO5cAtj amn9Kk1fJqkf9621GzFX8GpON9gGM72ruLoC0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ui0RozgVj782eZenovCwcdQYstEPpPQMeO8MT0fjT5o=; b=EbYvGDpai0a6O1sH7u2t3Q/OXeKlRlcjpTQjya8P7AGde2s0gSwc5CATx3uSLJZncu nGFj/lRNlTOJFb57+U7Sy/QOSfHZaFvAoro8od7g4Ax4RZhAPpIvDMTjeDLWz4e/xoQ0 uK3OyKK4VAKs2scNogN+w5xYjd7mjkoEE6d0nZcc06e15gg4Ic6VlQE5VgsaKyn1aNuK WSG4usSoNExrGYbrYHhnSZd01RSjhO+9AIIikNTfDk0t+XugnXM3J8U7B6M50bJpQlEs mH0VNOcdEC8lLhtdgfA6JyZcd9JJUC2X4lXbAkRMvjxhtyLFnds2hxgYMCYrprEvl1p8 k1nA== X-Gm-Message-State: AJcUukea3OhXGo/58R1pftbd5wxn2EQgZyWzqK1ORSqXbxfE9muDb5Eu osNPlMA1zkIf9mcerjC+LOew3Q== X-Received: by 2002:adf:8068:: with SMTP id 95mr26094476wrk.181.1548760030187; Tue, 29 Jan 2019 03:07:10 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:09 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 10/14] block, bfq: fix queue removal from weights tree Date: Tue, 29 Jan 2019 12:06:34 +0100 Message-Id: <20190129110638.12652-11-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org bfq maintains an ordered list, through a red-black tree, of unique weights of active bfq_queues. This list is used to detect whether there are active queues with differentiated weights. The weight of a queue is removed from the list when both the following two conditions become true: (1) the bfq_queue is flagged as inactive (2) the has no in-flight request any longer; Unfortunately, in the rare cases where condition (2) becomes true before condition (1), the removal fails, because the function to remove the weight of the queue (bfq_weights_tree_remove) is rightly invoked in the path that deactivates the bfq_queue, but mistakenly invoked *before* the function that actually performs the deactivation (bfq_deactivate_bfqq). This commits moves the invocation of bfq_weights_tree_remove for condition (1) to after bfq_deactivate_bfqq. As a consequence of this move, it is necessary to add a further reference to the queue when the weight of a queue is added, because the queue might otherwise be freed before bfq_weights_tree_remove is invoked. This commit adds this reference and makes all related modifications. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 17 +++++++++++++---- block/bfq-wf2q.c | 6 +++--- 2 files changed, 16 insertions(+), 7 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 12228af16198..bf585ad29bb5 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -754,6 +754,7 @@ void bfq_weights_tree_add(struct bfq_data *bfqd, struct bfq_queue *bfqq, inc_counter: bfqq->weight_counter->num_active++; + bfqq->ref++; } /* @@ -778,6 +779,7 @@ void __bfq_weights_tree_remove(struct bfq_data *bfqd, reset_entity_pointer: bfqq->weight_counter = NULL; + bfq_put_queue(bfqq); } /* @@ -789,9 +791,6 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, { struct bfq_entity *entity = bfqq->entity.parent; - __bfq_weights_tree_remove(bfqd, bfqq, - &bfqd->queue_weights_tree); - for_each_entity(entity) { struct bfq_sched_data *sd = entity->my_sched_data; @@ -825,6 +824,15 @@ void bfq_weights_tree_remove(struct bfq_data *bfqd, bfqd->num_groups_with_pending_reqs--; } } + + /* + * Next function is invoked last, because it causes bfqq to be + * freed if the following holds: bfqq is not in service and + * has no dispatched request. DO NOT use bfqq after the next + * function invocation. + */ + __bfq_weights_tree_remove(bfqd, bfqq, + &bfqd->queue_weights_tree); } /* @@ -1020,7 +1028,8 @@ bfq_bfqq_resume_state(struct bfq_queue *bfqq, struct bfq_data *bfqd, static int bfqq_process_refs(struct bfq_queue *bfqq) { - return bfqq->ref - bfqq->allocated - bfqq->entity.on_st; + return bfqq->ref - bfqq->allocated - bfqq->entity.on_st - + (bfqq->weight_counter != NULL); } /* Empty burst list and add just bfqq (see comments on bfq_handle_burst) */ diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index ce37d709a34f..63311d1ff1ed 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -1673,15 +1673,15 @@ void bfq_del_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq, bfqd->busy_queues[bfqq->ioprio_class - 1]--; - if (!bfqq->dispatched) - bfq_weights_tree_remove(bfqd, bfqq); - if (bfqq->wr_coeff > 1) bfqd->wr_busy_queues--; bfqg_stats_update_dequeue(bfqq_group(bfqq)); bfq_deactivate_bfqq(bfqd, bfqq, true, expiration); + + if (!bfqq->dispatched) + bfq_weights_tree_remove(bfqd, bfqq); } /* From patchwork Tue Jan 29 11:06:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156965 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4523072jaa; Tue, 29 Jan 2019 03:07:38 -0800 (PST) X-Google-Smtp-Source: ALg8bN4QpUoEHLnb09LptKxu/DRPEzaW/KWmTYOz1vN1U7iNaIlohqm4Y4wOBD7Dq+NyZ+koAJf+ X-Received: by 2002:a17:902:29a7:: with SMTP id h36mr25890931plb.244.1548760058273; Tue, 29 Jan 2019 03:07:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760058; cv=none; d=google.com; s=arc-20160816; b=rLgiWr83VadNBkfnD+VOoI9IybnJfoR0mMAQ7UbPBzqB5uW7W+7H+3tL1rzjkyR/QG yxVSLVNDJRgb1FBYqmmCrA4er3pijRxOnNoM0AzAL5+ebr3umb3FJXf4Z4Ql0Mx6A3f9 YNYHuEF2rP5q3x48xvYldafib6V/26UKfDa8EZ5xkiq6VChFpUvKUO7+SCDkv8+MZlYI d2P1hIv41Atl3vCH0i+VOh1RmXYyICO3jJnX7d/DwNEo0U7skaQ8j6h1+HDtx1PTizi6 VIJD090UtV25nYkwGrHKTBGKLjj7qcDXj7Cm4S7FJUKJ34+3b/e6Xtm88qfanRwWRxL3 yKcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/L0sIFrF4VuBSatFaX/8mVU9i1rVnBBzRyuN7LWEkY0=; b=NkbuVcN0wkbQ28uZrdM5fPmCpaLvEE6oPW1ezpoxQghGS+SqsEzHK3lXxN8xXNPF9s zfLuDGfmUQrYHiOq+wgYtX6DwO35uDpdOhkVNgENg3Ola5F/gg47ChvCYCSMa913QRFn xGk4zAawULZJv/onVvRiLqshyAUcxKum7JdREF+3QmqpUSAfAp2SvgqD+XMO23pVU+Zz PngZFWq5ly2ISby3Mu7JMuaazwBeCVaiR22L3bNBN4Uq8+Vflv9SXC2fPKlOgeEHikYK 0jK0MVfTaSDYHFbTxuI5M/hx+QwlCfBW/EgO00JmVJopW66Yg1MMYQGJJC3DthEMBiwN yzVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T8SwCMBx; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e129si34632251pgc.333.2019.01.29.03.07.37; Tue, 29 Jan 2019 03:07:38 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T8SwCMBx; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728669AbfA2LHf (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:35 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:56031 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728580AbfA2LHN (ORCPT ); Tue, 29 Jan 2019 06:07:13 -0500 Received: by mail-wm1-f66.google.com with SMTP id y139so17190725wmc.5 for ; Tue, 29 Jan 2019 03:07:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/L0sIFrF4VuBSatFaX/8mVU9i1rVnBBzRyuN7LWEkY0=; b=T8SwCMBxHn6msupWEQMC7s4HLPYVlg1FbZACnt/EPRze8AUAKrACng21WGluOoVdo2 L7cL+ds9pHdQQ2VaaL4qtXOdX7QcYXDB4xr7IgGJ4/SanZvahBsefcbqgUZLDb0WY7fX j5Vk+mwVIseQWtHLXWsOlAwYh9i8eN/GHSiBQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/L0sIFrF4VuBSatFaX/8mVU9i1rVnBBzRyuN7LWEkY0=; b=BzrbzDUrPt7rV5UGgpWHSpekPUV0SN4R43jU+lNnVEaD3ogYAwjSnhu/+FgZMkFu/N 1QYDQVaprHd6NoXb7T83QDqfGXppBGyvrXlqDs19gDRMaN+iQ1jZNY1hSFhWqqhGsvY7 egozWY1el8i7zwECXVzBggjve6RodW8LyVkYtlXU0S5q30+tobxWlqUF6rdcXTjJ6Q80 oWYub7SuTBn7/Z7dtZJyCmsu7BOPtiVNMsbz1MtIV48+9h9F/+cow0YHHuHTvKkuQbDS 5cmi54Hjddclg0WwiCy4XHBcvEl4md/zvRABlPapQjYwy3hrjr3w3/e1qTysH7WT3yrc pnQw== X-Gm-Message-State: AJcUukfQDzxjDE97jLpK/OjbDapMXjK/4Yjw1JKRfDBSnwQkFCGMLz0P sn4FZCI5eoQesN2gblwKER0cpg== X-Received: by 2002:a1c:7d06:: with SMTP id y6mr20407313wmc.7.1548760031541; Tue, 29 Jan 2019 03:07:11 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:10 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 11/14] block, bfq: reduce threshold for detecting command queueing Date: Tue, 29 Jan 2019 12:06:35 +0100 Message-Id: <20190129110638.12652-12-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org bfq borrowed from cfq a simple heuristic for detecting whether the drive performs command queueing: check whether the average number of in-flight requests is above a given threshold. Unfortunately this heuristic does fail to detect queueing (on drives with queueing) if processes doing I/O are few and issue I/O with a low depth. To reduce false negatives, this commit lowers the threshold. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index bf585ad29bb5..48b579032d14 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -230,7 +230,7 @@ static struct kmem_cache *bfq_pool; #define BFQ_MIN_TT (2 * NSEC_PER_MSEC) /* hw_tag detection: parallel requests threshold and min samples needed. */ -#define BFQ_HW_QUEUE_THRESHOLD 4 +#define BFQ_HW_QUEUE_THRESHOLD 3 #define BFQ_HW_QUEUE_SAMPLES 32 #define BFQQ_SEEK_THR (sector_t)(8 * 100) @@ -4798,7 +4798,7 @@ static void bfq_update_hw_tag(struct bfq_data *bfqd) * sum is not exact, as it's not taking into account deactivated * requests. */ - if (bfqd->rq_in_driver + bfqd->queued < BFQ_HW_QUEUE_THRESHOLD) + if (bfqd->rq_in_driver + bfqd->queued <= BFQ_HW_QUEUE_THRESHOLD) return; if (bfqd->hw_tag_samples++ < BFQ_HW_QUEUE_SAMPLES) From patchwork Tue Jan 29 11:06:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156964 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522940jaa; Tue, 29 Jan 2019 03:07:32 -0800 (PST) X-Google-Smtp-Source: ALg8bN7dWn5B/te5qcYx3364WBX/F+3J32WxNOMgqxyyypx9+y8CjMFPbtknxOp3PGlB7m8zyz0n X-Received: by 2002:a63:b0a:: with SMTP id 10mr23373169pgl.423.1548760052468; Tue, 29 Jan 2019 03:07:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760052; cv=none; d=google.com; s=arc-20160816; b=C8zUmfZE3TuGYPRGlvznCMsqA1sVgIjhJmf3MmDMwYrrFr+A8uYjddB2iunw9aZRAo QT/UZfILEC9XJoZKraUQc7JcSGuSiHNYGZ2EraKMvDwlXxpH1qvsK4xBxo4Ub8IG888E ATmdIEfTTIad3Dn50QEQMNbiouVUMS9LjVNBVcGAlKjqIXJpKJH6sR+l5kFQDt43U3Qu 55Zb9jOsy2slx/8vYJiJ798yN1CnKaqpgbKD2/EFf2ZSP/JmLoaCCJhQA6FO2sfATtS+ sjCNkiFmSCpOjhrOqLpDlLfOtH48XD3yqjqDC/5L05NyaP6mGlrI18fpZJQBaUcGhZVd z/0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xDzJdQaJUVbOMOgsobVLGeLVo1CynjuO1LWZuQEuQ0s=; b=MmzmfudJU4Q1pn7XSJ3gM5T5rqdibPhEGJXzYofFOLBWzF4NAtB7cFRwyPI+zFh4ic ej/15SXNgX90uLEA45aQpkAfae3/X1QVfBnwqt963VYY3aBikuzmwYfi7nCggpqnJQKX v3s4N47F0w+XFIfFGIAQLxY8BQ1MC/ikKzT/nV4p2MrFvvF3CUToyQtoCOBuGwxRyIOM 6us3UpGtggUBROrIyFz4nYp/OwXXnF0eJxxGGoGHjK7dag3jPGfGKGqqdTXqXRBBsJi+ Aigw2BbQwRBDZsAUyw8RXmq5G0TJgQY9D3nXabIATb+ZS37byRhHMozSv0q/C0NyNVnb USEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YlNrCME0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j132si35082951pfc.84.2019.01.29.03.07.32; Tue, 29 Jan 2019 03:07:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YlNrCME0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728654AbfA2LHa (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:30 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:33491 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728594AbfA2LHO (ORCPT ); Tue, 29 Jan 2019 06:07:14 -0500 Received: by mail-wr1-f65.google.com with SMTP id p7so21606788wru.0 for ; Tue, 29 Jan 2019 03:07:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xDzJdQaJUVbOMOgsobVLGeLVo1CynjuO1LWZuQEuQ0s=; b=YlNrCME02AvhfJTfFzDA7cXLW6GkN4P602kH9FqZdMo8dfqIjrdqLc0SZVieJXCoRk EjdcaZrCdjJRqKoBG9NcdFH+dOguGYF0kGl4fWzV9VPBr3jVZNgBhaVc8Z87DiCt5APe jwOLIrtW+c8AGnIapiBil70XxXFnbCiHytrkY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xDzJdQaJUVbOMOgsobVLGeLVo1CynjuO1LWZuQEuQ0s=; b=UzCx4KUe0UytLqhjroHrLilW3Hx234XFV5oVNNQSFh01jkHx+YD6stbDpXpyU0VuNa a8wNbeFsQBRGj+6GBKeLTVAOXT1PNDyErbGLXRS8cNC4ambQNCcgreE1OR5XAJvv3YJw 7x9bFdBVV1Hw1BmCQ/ljfIc9gbyGDb1YdgViGTiOaF+tlPnOOAFG1/Ju+XnDny+Cc4Ox Zs3ngq+lHoj3Popdx8De/7RgTXyiZBomxmfdrpBPonSvLldIvAoIPUCfE+pAmJ/KPo2R t8LjzsHydDyG/aeLka7nZt2RW56Ix0MXxEkZ7CCP6au23//87DBB9a681V2IETr/zSt+ FMmw== X-Gm-Message-State: AJcUukfvzXgq5XX750AockvLWJ5hTzeTKHbjP5w08CTRnh1Tc4Cf7oto 6pGy8EoiUDWr9EuzzOspE1Xo7w== X-Received: by 2002:adf:c711:: with SMTP id k17mr24732765wrg.197.1548760032900; Tue, 29 Jan 2019 03:07:12 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:12 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 12/14] block, bfq: port commit "cfq-iosched: improve hw_tag detection" Date: Tue, 29 Jan 2019 12:06:36 +0100 Message-Id: <20190129110638.12652-13-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The original commit is commit 1a1238a7dd48 ("cfq-iosched: improve hw_tag detection") and has the following commit message. If active queue hasn't enough requests and idle window opens, cfq will not dispatch sufficient requests to hardware. In such situation, current code will zero hw_tag. But this is because cfq doesn't dispatch enough requests instead of hardware queue doesn't work. Don't zero hw_tag in such case. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 48b579032d14..2ab53d93ba12 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -4786,6 +4786,8 @@ static void bfq_insert_requests(struct blk_mq_hw_ctx *hctx, static void bfq_update_hw_tag(struct bfq_data *bfqd) { + struct bfq_queue *bfqq = bfqd->in_service_queue; + bfqd->max_rq_in_driver = max_t(int, bfqd->max_rq_in_driver, bfqd->rq_in_driver); @@ -4801,6 +4803,17 @@ static void bfq_update_hw_tag(struct bfq_data *bfqd) if (bfqd->rq_in_driver + bfqd->queued <= BFQ_HW_QUEUE_THRESHOLD) return; + /* + * If active queue hasn't enough requests and can idle, bfq might not + * dispatch sufficient requests to hardware. Don't zero hw_tag in this + * case + */ + if (bfqq && bfq_bfqq_has_short_ttime(bfqq) && + bfqq->dispatched + bfqq->queued[0] + bfqq->queued[1] < + BFQ_HW_QUEUE_THRESHOLD && + bfqd->rq_in_driver < BFQ_HW_QUEUE_THRESHOLD) + return; + if (bfqd->hw_tag_samples++ < BFQ_HW_QUEUE_SAMPLES) return; From patchwork Tue Jan 29 11:06:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156963 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522811jaa; Tue, 29 Jan 2019 03:07:26 -0800 (PST) X-Google-Smtp-Source: ALg8bN6y4//L2gvZ0vfdmJS7aUuq7mWhEKMs2Icc5KEl0kxIE1aVVcRgEYOL1DNOLdmHgdfh7vkU X-Received: by 2002:a17:902:6acc:: with SMTP id i12mr24942359plt.148.1548760046059; Tue, 29 Jan 2019 03:07:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760046; cv=none; d=google.com; s=arc-20160816; b=Xg3P6zcjWszonoqEuYnsc1fMyU1VrhNieahTGo5q7IeCOcdpXnCuR3JmEeUxyXtK7/ MLKwscA/3e3sCr0BlAiuTZKA0LYjBe949CrLx4Ln52TMwHFTTqjlxrnZIwBETzrJdG4X BUx72u3P9i0/lmwGO/fuZ54QBrAsE+eBdSPBCIXUZt/BkLQp5FAu3DWKYyFTS3RhuFVI nPQ5aNQi4Iy9yVaP0UmSNWIQguDbWMopPeFczXjor6ug535mCkbkk/koOwAI5vhbU+PY Pr9VSWKO3hNYqJ7vrMIbQOsar78t/FzpKwA0SLdTz+Hx1Wqc2WeIx7L9nZjr8cWWIcf/ /ZuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8JaSsc/YYZ/kdnvfBrrFjWTAlWsvvc6dlhkWZwzIj+Q=; b=r9HYDATCXG+Rkkc42pt3RFbvcjhkt6VES27TP7j7w5cePraGbb9V8Tbll8dr4oS3NU HTydUVvrSOZb5KV1ilvzzDPo9XdkMS6PpTW+tCNDh+kGNphnSDvPxsj0a9pQPHPNrEPv 8clzXGkK6BpJAsZ5GGGY3BMdzO62fyPVQwL+Nf8TDm3QI0plMPF+0QDd1ryj0E2Y2Dyt QaMhXxDXEsUyo2TDCeeOmiqVRBOAEUsCyGAuvf/PSQmJDJYvwQLA/RZy9P24DQ5EVEc2 +xurttrX57p6P92156SGD0P6JVDvnZIEoyAQB6zwgkkxuAMUzT2I8pVNS6/Lznj7M8JB 1C1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jDOzgYh+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l135si35120525pga.69.2019.01.29.03.07.25; Tue, 29 Jan 2019 03:07:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jDOzgYh+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728626AbfA2LHT (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:19 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:41232 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728604AbfA2LHQ (ORCPT ); Tue, 29 Jan 2019 06:07:16 -0500 Received: by mail-wr1-f67.google.com with SMTP id x10so21518157wrs.8 for ; Tue, 29 Jan 2019 03:07:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8JaSsc/YYZ/kdnvfBrrFjWTAlWsvvc6dlhkWZwzIj+Q=; b=jDOzgYh+7JYBxH0+MkncQ3vaM6G1Wwz/zf4/BEGiViuG5e2T+PP/awx4hfjQQJzq4o YZf+PzUJlNnIkncTTrYZrkQRjIGafwLxTEYhosPBqcbspQ3tc3VqYsXYT9acbSshwXsm o+krKWN7u2PKp1ijOnxKTWPCL6r8kjlcvyqS8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8JaSsc/YYZ/kdnvfBrrFjWTAlWsvvc6dlhkWZwzIj+Q=; b=VqnIbN58amhC9EVFTsVuSwyaUv78MaLv7Hz0/uLo8hZXI1/dEgIGF/yKQD/WBbR1Yc XDsmpUL6tIPkSNHNSfVssA15mMCCX6gjqpM//dfIZQui4wsANrWSCXfgKt+hQIN+uqGC ODxFx6etfC0pWm5uGJVm8nZvBJ33ln+TedKmfPeNpMfHzDQRTEHyeXdT6TqrlOs/8gcb PzOZuq71D1gZrIN2jkN9VPEm/hrJqMnuslptRymPrppAt/uNgwqT9/541qFEVGTf5tjR igf6IkghWjzThwPgsnXKCTCbRbbaTvz+rTd3U06fzvo9c7vUxC1tymE2l6NVlfFsYbql Q8xA== X-Gm-Message-State: AHQUAuY3A1hULV/SJeJOAkR1RZeaq90p063lDByTG5TYkg+NxkHi2nJ9 oAQbyUBxhHbIIeiydXfa6cF3nw== X-Received: by 2002:adf:dcd0:: with SMTP id x16mr4708238wrm.143.1548760034065; Tue, 29 Jan 2019 03:07:14 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:13 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 13/14] block, bfq: do not overcharge writes in asymmetric scenarios Date: Tue, 29 Jan 2019 12:06:37 +0100 Message-Id: <20190129110638.12652-14-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Writes tend to starve reads. bfq counters this problem by overcharging writes with an inflated service w.r.t. the actual service (number of sector written) they receive. Yet his overcharging is useless, and actually causes unfairness in the opposite direction, when bfq happens to be enforcing strong I/O control. bfq does this enforcing when the scenario is asymmetric, i.e., when some bfq_queue or group of bfq_queues is to be granted a different bandwidth than some other bfq_queue or group of bfq_queues. So, in such a scenario, this commit disables write overcharging. Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 2ab53d93ba12..06268449d2ca 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -888,7 +888,8 @@ static struct request *bfq_find_next_rq(struct bfq_data *bfqd, static unsigned long bfq_serv_to_charge(struct request *rq, struct bfq_queue *bfqq) { - if (bfq_bfqq_sync(bfqq) || bfqq->wr_coeff > 1) + if (bfq_bfqq_sync(bfqq) || bfqq->wr_coeff > 1 || + !bfq_symmetric_scenario(bfqq->bfqd)) return blk_rq_sectors(rq); return blk_rq_sectors(rq) * bfq_async_charge_factor; From patchwork Tue Jan 29 11:06:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 156962 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp4522748jaa; Tue, 29 Jan 2019 03:07:22 -0800 (PST) X-Google-Smtp-Source: ALg8bN7MicuHmj1V7JJGMab/4yf5Q8KMOVsFhjxwZX3HUiXLvsmuQYy5xRjdDCgd6ovqVCGJimqO X-Received: by 2002:a17:902:24e7:: with SMTP id l36mr25701754plg.61.1548760042399; Tue, 29 Jan 2019 03:07:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548760042; cv=none; d=google.com; s=arc-20160816; b=KiNMF1f9E/uqTcAtIED0skrQgAlQBoecfBGWte/RdtpfarQ8/tkcw4BflXLbbfhhvW n7H/ocpeFXExhN+rLTXaSys7zydK+Yv/7hwNMi5oi+lOps5XBQor9/lIKX85q0r7fOH2 vhYqi9LDJoZAso79VM2ccIcdI+5GjvMEatMCl/A/4R2b3Aa40va4l6MifKUJ7OMK6SCf WrpTMFENQRK+bhyb6gFyKf7nSt9Fq1rE2/DSvss/2EJU8xhO1CNbtIkj04NHqiXdqMfz kZkCeCG3270Xmvc/dM5sWmOzB4AZvqrjzhliUabVPGJxaHGiYf/a4w7OpAMFWdYT8gNZ GlOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=C6mbkxYARjOVWPicsF+8F1IWlVh58inCJzHv80eeZ00=; b=QglaLWng2ITitR+36w9FjF3V97XwkKYOcFMDmWkMAkfdvG4WcAiAOPC8XBmdtiUG5C 6pBM0JdvnrVMOCHohc3vpDeKsU3gOADWnIDwTGtVUSR5zzgUdQOYarEXTMw8X6Lom3B0 ZDBcJpW1RrWlUGqnGQ9t4RUViGj2/0Ne8CPilH1kQPNnupPc4avWBufqGbwa1mQpogq6 +VMSHaF581mjzMgc9FMPM1VNL2nzxLu/8eKcgo4AUFsI2UQ5Ok48z88389/Rp2a//Y3E L2fT7v6U4vDvn5O9XrpzPZV9CRyy+NUqv4dSdtMw9PvGm+5ykvyBWiVu9qbNA3n3yw5D CAJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WaAV8QPF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q9si7692080pgh.92.2019.01.29.03.07.21; Tue, 29 Jan 2019 03:07:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WaAV8QPF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728640AbfA2LHU (ORCPT + 31 others); Tue, 29 Jan 2019 06:07:20 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:37874 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728474AbfA2LHQ (ORCPT ); Tue, 29 Jan 2019 06:07:16 -0500 Received: by mail-wm1-f65.google.com with SMTP id g67so17261902wmd.2 for ; Tue, 29 Jan 2019 03:07:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C6mbkxYARjOVWPicsF+8F1IWlVh58inCJzHv80eeZ00=; b=WaAV8QPFZ1BYtgx4VTqu89ErSb86GwumiYIkwW57mgQfl7Nl8icwCI+CpZ6V/iKbv0 99j6c8j7vc8VEpAnOX39pOSRzg09Sv85zsy1n2cOa9URAETvg/4YZ621WnoNUPCzDTCp ATa00BVE+iqnjy+Teq0mqD9PtsCE6+Kx6VCCw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C6mbkxYARjOVWPicsF+8F1IWlVh58inCJzHv80eeZ00=; b=LaTeTPT8BO+B1jJ48JRXV+6w2Lt1naKDLzRXVyXFQSt0fC5kkMq/A9F4Py/pmJwXtz QC4l1kAxkBcmU/VfwKdY78lpOL+tUsxWuxZNpW3ku4xSPbVabSHtP5Ux1UOZ2HMKmU2x wjSlFK0wJE+2W28ZEs31+/egs7cakXMkdHJTQ8flqBobKb4lpsZYbUAq2sKqo63VrFwC lZeQQNAhrKae1+TGhxi+Fa9w77EFonZgD3WaqrLyCMFJgrGuneXJk0/zmjPZ1uGhKD+g loDJJCo8QdvXTwH8Djnf85CBU6B+4QM7JDAqr1qKvsBBEakjgOvQ47znqm/v5peNiib/ KECA== X-Gm-Message-State: AJcUukeSCMuCUNSP94ZngrnTzEdc26MUYtPthMxmHaRZiVRlEGT+jsdt bo+gr0UZLdorNBHUipztQJgRgw== X-Received: by 2002:a1c:ef11:: with SMTP id n17mr20436112wmh.112.1548760035222; Tue, 29 Jan 2019 03:07:15 -0800 (PST) Received: from localhost.localdomain ([88.147.67.218]) by smtp.gmail.com with ESMTPSA id s132sm2066112wmf.28.2019.01.29.03.07.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 03:07:14 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, linus.walleij@linaro.org, broonie@kernel.org, bfq-iosched@googlegroups.com, oleksandr@natalenko.name, mancha@tower-research.com, Paolo Valente Subject: [PATCH BUGFIX IMPROVEMENT 14/14] block, bfq: fix in-service-queue check for queue merging Date: Tue, 29 Jan 2019 12:06:38 +0100 Message-Id: <20190129110638.12652-15-paolo.valente@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129110638.12652-1-paolo.valente@linaro.org> References: <20190129110638.12652-1-paolo.valente@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When a new I/O request arrives for a bfq_queue, say Q, bfq checks whether that request is close to (a) the head request of some other queue waiting to be served, or (b) the last request dispatched for the in-service queue (in case Q itself is not the in-service queue) If a queue, say Q2, is found for which the above condition holds, then bfq merges Q and Q2, to hopefully get a more sequential I/O in the resulting merged queue, and thus a possibly higher throughput. Case (b) is checked by comparing the new request for Q with the last request dispatched, assuming that the latter necessarily belonged to the in-service queue. Unfortunately, this assumption is no longer always correct, since commit d0edc2473be9 ("block, bfq: inject other-queue I/O into seeky idle queues on NCQ flash"). When the assumption does not hold, queues that must not be merged may be merged, causing unexpected loss of control on per-queue service guarantees. This commit solves this problem by adding an extra field, which stores the actual last request dispatched for the in-service queue, and by using this new field to correctly check case (b). Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 5 ++++- block/bfq-iosched.h | 3 +++ 2 files changed, 7 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 06268449d2ca..4c592496a16a 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -2251,7 +2251,8 @@ bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, if (in_service_bfqq && in_service_bfqq != bfqq && likely(in_service_bfqq != &bfqd->oom_bfqq) && - bfq_rq_close_to_sector(io_struct, request, bfqd->last_position) && + bfq_rq_close_to_sector(io_struct, request, + bfqd->in_serv_last_pos) && bfqq->entity.parent == in_service_bfqq->entity.parent && bfq_may_be_close_cooperator(bfqq, in_service_bfqq)) { new_bfqq = bfq_setup_merge(bfqq, in_service_bfqq); @@ -2791,6 +2792,8 @@ static void bfq_update_peak_rate(struct bfq_data *bfqd, struct request *rq) bfq_update_rate_reset(bfqd, rq); update_last_values: bfqd->last_position = blk_rq_pos(rq) + blk_rq_sectors(rq); + if (RQ_BFQQ(rq) == bfqd->in_service_queue) + bfqd->in_serv_last_pos = bfqd->last_position; bfqd->last_dispatch = now_ns; } diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 30be669be465..062e1c4787f4 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -538,6 +538,9 @@ struct bfq_data { /* on-disk position of the last served request */ sector_t last_position; + /* position of the last served request for the in-service queue */ + sector_t in_serv_last_pos; + /* time of last request completion (ns) */ u64 last_completion;