From patchwork Wed Dec 27 12:15:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valente X-Patchwork-Id: 122775 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp2102875qgn; Wed, 27 Dec 2017 04:15:41 -0800 (PST) X-Google-Smtp-Source: ACJfBouYV30Xvj+cijijf5lnu+Z4PJdhLKYhOZU/tR88s9QIiuG6ljqc+/UDgcGzEAIadcJMI8V9 X-Received: by 10.159.235.132 with SMTP id f4mr28367765plr.122.1514376941056; Wed, 27 Dec 2017 04:15:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1514376941; cv=none; d=google.com; s=arc-20160816; b=ZRPItdBuIiYIbIwpSDTKEbXvyMTZAYyn1imJOVLxC9OkNAm8CGR99oiae8BQRMMKTu PKY0Djj3HeIrCGgiIDXIMir2ZVmGEPTScnesTAuQI8JONAO6Kwf2DMC0doYch7jaceHc 0Vknhlmww1nuEzzWO6MuWjyQLWLcqVMEqSIRRZkXQVbWzvKQTWHO5ars267DvLLnTZus y2GrcReAakQkhZ7iKAPon8vdQyM4rCQfx1ucrSZsoAj7+rjwIQJ3/fp1HhdFtPlVvi1g 2Zw/jl3aam9c6ZjEjN6kYvhiKoqSBIDdqiFZlx2mc2351tYgJHuSv+Z9z/h1I5jy7+J/ pPlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=LmPi9EV+X0tQdeIDjCwGX1FXGTY+52ixsBa61AAFsfY=; b=dgESW+I0GdjJ2cAb1vxGjhZjd69ocUOH36rjgs8hINstBt62sBT+g4XZQzeuQfpw/i 880FmbL+XeovEpNircB5YXlxTvZxlWVQqDmoGn3ZCJ9rXYpGza7Mq9e/GkTFKCTFR23a qiICvFRII6IjKGJtFGtPqEa84/7f4wIgTnO15JHIuWYZrnMuufxMqAcO0lUxAFJTgoUR 7QlOERyNMFno5eYPYs+/bsu/Xso+B+m+mCyqgwv17x8+Eqb0xXTdwY/wmePP4D5h7yIh gi7gRDIQM9Op+pnN7cAN4W1NohbAAxkkPcao2fW3bVLrimx1TXsJ8ZPZ4vJudjEb9siI abxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iLidzMfs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l1si24684870plb.703.2017.12.27.04.15.40; Wed, 27 Dec 2017 04:15:41 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iLidzMfs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751913AbdL0MPg (ORCPT + 28 others); Wed, 27 Dec 2017 07:15:36 -0500 Received: from mail-wr0-f194.google.com ([209.85.128.194]:35060 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751426AbdL0MPX (ORCPT ); Wed, 27 Dec 2017 07:15:23 -0500 Received: by mail-wr0-f194.google.com with SMTP id l19so24227003wrc.2 for ; Wed, 27 Dec 2017 04:15:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LmPi9EV+X0tQdeIDjCwGX1FXGTY+52ixsBa61AAFsfY=; b=iLidzMfsocMxZXGFx5zS0uBQbfVWdJcaK+0TLc335qYj+z+WjTBhS0hretUQexpp4U SsRZpsWyMVxTPq5b6Zs3E9hKVUEqJWgq1r/Z6UlfC4uOGF7XDA6tiFhPmes13MpovJTF M5ifMISYsJN1jFa6+2mX6bNGsdZ9FvB4X+47o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LmPi9EV+X0tQdeIDjCwGX1FXGTY+52ixsBa61AAFsfY=; b=uUJW1+gd7VR6uZdUlqmOfWpmCdFtAbZjczisU3Fr90wWux6r6jS/wnQQe4jBcK7F8d GtILdH4tddJ2qMJ+CG/CttQhi7k060g9UFU1/dVh7+829F4XGJ/kYn2UbhV7ukDxyuNf Q2aHLkyAIctxnhg/amEKPKo4PlkOuqI94nk/5vOaUpekX9S9WRXrDAO2R/YOXwoTh9IU eiqUsEcbDp5yu/CDZlFn5O7ovMU7WsP2Xx5pkGTxeQCI5W6P7yypuOK9mo+s1f/MeUQU k986/y0NZqtFj7DI6X1Ps6hoozPc06lCqY7Co5QOacE+fDclNdMSIQEmLWtDOwI2sYRE dhtQ== X-Gm-Message-State: AKGB3mINt915zgX5aoL0WpGJs2uUjxFIsD+ZAK52K0dbqEtjLFHApOHG QqONkHWA9Tu1HH/t9kRZT443hQ== X-Received: by 10.223.139.152 with SMTP id o24mr30547865wra.243.1514376922233; Wed, 27 Dec 2017 04:15:22 -0800 (PST) Received: from localhost.localdomain (146-241-25-197.dyn.eolo.it. [146.241.25.197]) by smtp.gmail.com with ESMTPSA id i47sm6387297wra.97.2017.12.27.04.15.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Dec 2017 04:15:21 -0800 (PST) From: Paolo Valente To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, ulf.hansson@linaro.org, broonie@kernel.org, linus.walleij@linaro.org, bfq-iosched@googlegroups.com, Paolo Valente Subject: [PATCH IMPROVEMENT/BUGFIX 1/1] block, bfq: limit tags for writes and async I/O Date: Wed, 27 Dec 2017 13:15:07 +0100 Message-Id: <20171227121507.4280-2-paolo.valente@linaro.org> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20171227121507.4280-1-paolo.valente@linaro.org> References: <20171227121507.4280-1-paolo.valente@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Asynchronous I/O can easily starve synchronous I/O (both sync reads and sync writes), by consuming all request tags. Similarly, storms of synchronous writes, such as those that sync(2) may trigger, can starve synchronous reads. In their turn, these two problems may also cause BFQ to loose control on latency for interactive and soft real-time applications. For example, on a PLEXTOR PX-256M5S SSD, LibreOffice Writer takes 0.6 seconds to start if the device is idle, but it takes more than 45 seconds (!) if there are sequential writes in the background. This commit addresses this issue by limiting the maximum percentage of tags that asynchronous I/O requests and synchronous write requests can consume. In particular, this commit grants a higher threshold to synchronous writes, to prevent the latter from being starved by asynchronous I/O. According to the above test, LibreOffice Writer now starts in about 1.2 seconds on average, regardless of the background workload, and apart from some rare outlier. To check this improvement, run, e.g., sudo ./comm_startup_lat.sh bfq 5 5 seq 10 "lowriter --terminate_after_init" for the comm_startup_lat benchmark in the S suite [1]. [1] https://github.com/Algodev-github/S Signed-off-by: Paolo Valente --- block/bfq-iosched.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++ block/bfq-iosched.h | 12 +++++++++ 2 files changed, 89 insertions(+) -- 2.15.1 diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index e33c5c4c9856..6f75015d18c0 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -417,6 +417,82 @@ static struct request *bfq_choose_req(struct bfq_data *bfqd, } } +/* + * See the comments on bfq_limit_depth for the purpose of + * the depths set in the function. + */ +static void bfq_update_depths(struct bfq_data *bfqd, struct sbitmap_queue *bt) +{ + bfqd->sb_shift = bt->sb.shift; + + /* + * In-word depths if no bfq_queue is being weight-raised: + * leaving 25% of tags only for sync reads. + * + * In next formulas, right-shift the value + * (1U<sb_shift), instead of computing directly + * (1U<<(bfqd->sb_shift - something)), to be robust against + * any possible value of bfqd->sb_shift, without having to + * limit 'something'. + */ + /* no more than 50% of tags for async I/O */ + bfqd->word_depths[0][0] = max((1U<sb_shift)>>1, 1U); + /* + * no more than 75% of tags for sync writes (25% extra tags + * w.r.t. async I/O, to prevent async I/O from starving sync + * writes) + */ + bfqd->word_depths[0][1] = max(((1U<sb_shift) * 3)>>2, 1U); + + /* + * In-word depths in case some bfq_queue is being weight- + * raised: leaving ~63% of tags for sync reads. This is the + * highest percentage for which, in our tests, application + * start-up times didn't suffer from any regression due to tag + * shortage. + */ + /* no more than ~18% of tags for async I/O */ + bfqd->word_depths[1][0] = max(((1U<sb_shift) * 3)>>4, 1U); + /* no more than ~37% of tags for sync writes (~20% extra tags) */ + bfqd->word_depths[1][1] = max(((1U<sb_shift) * 6)>>4, 1U); +} + +/* + * Async I/O can easily starve sync I/O (both sync reads and sync + * writes), by consuming all tags. Similarly, storms of sync writes, + * such as those that sync(2) may trigger, can starve sync reads. + * Limit depths of async I/O and sync writes so as to counter both + * problems. + */ +static void bfq_limit_depth(unsigned int op, struct blk_mq_alloc_data *data) +{ + struct blk_mq_tags *tags = blk_mq_tags_from_data(data); + struct bfq_data *bfqd = data->q->elevator->elevator_data; + struct sbitmap_queue *bt; + + if (op_is_sync(op) && !op_is_write(op)) + return; + + if (data->flags & BLK_MQ_REQ_RESERVED) { + if (unlikely(!tags->nr_reserved_tags)) { + WARN_ON_ONCE(1); + return; + } + bt = &tags->breserved_tags; + } else + bt = &tags->bitmap_tags; + + if (unlikely(bfqd->sb_shift != bt->sb.shift)) + bfq_update_depths(bfqd, bt); + + data->shallow_depth = + bfqd->word_depths[!!bfqd->wr_busy_queues][op_is_sync(op)]; + + bfq_log(bfqd, "[%s] wr_busy %d sync %d depth %u", + __func__, bfqd->wr_busy_queues, op_is_sync(op), + data->shallow_depth); +} + static struct bfq_queue * bfq_rq_pos_tree_lookup(struct bfq_data *bfqd, struct rb_root *root, sector_t sector, struct rb_node **ret_parent, @@ -5265,6 +5341,7 @@ static struct elv_fs_entry bfq_attrs[] = { static struct elevator_type iosched_bfq_mq = { .ops.mq = { + .limit_depth = bfq_limit_depth, .prepare_request = bfq_prepare_request, .finish_request = bfq_finish_request, .exit_icq = bfq_exit_icq, diff --git a/block/bfq-iosched.h b/block/bfq-iosched.h index 5d47b58d5fc8..fcd941008127 100644 --- a/block/bfq-iosched.h +++ b/block/bfq-iosched.h @@ -629,6 +629,18 @@ struct bfq_data { struct bfq_io_cq *bio_bic; /* bfqq associated with the task issuing current bio for merging */ struct bfq_queue *bio_bfqq; + + /* + * Cached sbitmap shift, used to compute depth limits in + * bfq_update_depths. + */ + unsigned int sb_shift; + + /* + * Depth limits used in bfq_limit_depth (see comments on the + * function) + */ + unsigned int word_depths[2][2]; }; enum bfqq_state_flags {