From patchwork Tue Nov 14 18:04:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 743957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6E8FC4167B for ; Tue, 14 Nov 2023 18:04:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233826AbjKNSE7 (ORCPT ); Tue, 14 Nov 2023 13:04:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233823AbjKNSEr (ORCPT ); Tue, 14 Nov 2023 13:04:47 -0500 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90EF2D50; Tue, 14 Nov 2023 10:04:33 -0800 (PST) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6c10f098a27so4680534b3a.2; Tue, 14 Nov 2023 10:04:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699985073; x=1700589873; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DrcMRVgoYbthv1/FFt3aRCjeeEnijopEtjBuXOLk8i4=; b=HLhiXW+G37hw9RQC7WJ23fVxbCQ7yyCpqKzZH1AOVrQ7ORlbhda0bVkjxkuUcPaPOd SOR2qFAjG1jrEif/pcbrb6R7ys3rmXFFCQL97hODp7TFA9pAjba2j5zSjlelo3Lu7rai QT80Orak18EPEKe8L+5f94P2x4IZtnRKdOWd5eY+ykeVElPkwfAqUfHa1NoplfsD1qhs Wj7rsuP+iSWxRjuZVw8gj47E5wvZaYzhnlqDH23e7rfEsL/B3ewOv/RhjQzWNro4koBz QEePB71Sak94PW2wd34itn/5lpi2fhhP633GJMQF5lXYDnQDwyJJ5AkvTvo7mS3de+Yz y0PQ== X-Gm-Message-State: AOJu0Yxc1F/r3GTugj54pKDDf6Df3X5jam5IKh9zFMzuHrx8spK9oTz7 muLQRtMWWy5o8xC14klM7NA= X-Google-Smtp-Source: AGHT+IFIA+7Qn9y6sdXM0PCQDeJiHzslY43h3MnvT7p6QgWniyg4q0JVTR1CFLUZYnU7neKkSdzKFA== X-Received: by 2002:a05:6a00:93a4:b0:6b2:5d32:58c with SMTP id ka36-20020a056a0093a400b006b25d32058cmr9440742pfb.22.1699985072830; Tue, 14 Nov 2023 10:04:32 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:0:1000:8411:2278:ad72:cefb:4d49]) by smtp.gmail.com with ESMTPSA id bq4-20020a056a02044400b0059d6f5196fasm5101937pgb.78.2023.11.14.10.04.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 10:04:32 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , Matthias Brugger , AngeloGioacchino Del Regno Subject: [PATCH v5 1/3] block: Introduce flag BLK_MQ_F_DISABLE_FAIR_TAG_SHARING Date: Tue, 14 Nov 2023 10:04:15 -0800 Message-ID: <20231114180426.1184601-2-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog In-Reply-To: <20231114180426.1184601-1-bvanassche@acm.org> References: <20231114180426.1184601-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The fair sharing algorithm has a negative performance impact for storage devices for which the full queue depth is required to reach peak performance, e.g. UFS devices. This is because it takes long after a request queue became inactive until tags are reassigned to the active request queue(s). Since making tag sharing fair is not needed if the request processing latency is similar for all request queues, introduce the hctx / tag set flag BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche --- block/blk-mq-debugfs.c | 1 + block/blk-mq.h | 3 ++- include/linux/blk-mq.h | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index 5cbeb9344f2f..f41408103106 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -198,6 +198,7 @@ static const char *const hctx_flag_name[] = { HCTX_FLAG_NAME(NO_SCHED), HCTX_FLAG_NAME(STACKING), HCTX_FLAG_NAME(TAG_HCTX_SHARED), + HCTX_FLAG_NAME(DISABLE_FAIR_TAG_SHARING), }; #undef HCTX_FLAG_NAME diff --git a/block/blk-mq.h b/block/blk-mq.h index f75a9ecfebde..eda6bd0611ea 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -416,7 +416,8 @@ static inline bool hctx_may_queue(struct blk_mq_hw_ctx *hctx, { unsigned int depth, users; - if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED)) + if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) || + (hctx->flags & BLK_MQ_F_DISABLE_FAIR_TAG_SHARING)) return true; /* diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 1ab3081c82ed..b12a0be839aa 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -662,6 +662,7 @@ enum { * or shared hwqs instead of 'mq-deadline'. */ BLK_MQ_F_NO_SCHED_BY_DEFAULT = 1 << 7, + BLK_MQ_F_DISABLE_FAIR_TAG_SHARING = 1 << 8, BLK_MQ_F_ALLOC_POLICY_START_BIT = 8, BLK_MQ_F_ALLOC_POLICY_BITS = 1, From patchwork Tue Nov 14 18:04:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 744596 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 529D1C41535 for ; Tue, 14 Nov 2023 18:04:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233889AbjKNSFA (ORCPT ); Tue, 14 Nov 2023 13:05:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233930AbjKNSEr (ORCPT ); Tue, 14 Nov 2023 13:04:47 -0500 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 747D51B9; Tue, 14 Nov 2023 10:04:35 -0800 (PST) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-6b77ab73c6fso24963b3a.1; Tue, 14 Nov 2023 10:04:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699985074; x=1700589874; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VIp72GlkjbCmnVUUISbqMrl5dUpklTIjfpOKSRHa5Eo=; b=bfzEDzTbkFYT2oijnzFTR+YlkGnQ8hFJ2KC9H3VaXkwtDDSBIdV24unnkxN87DUXD9 MOUaVZcb/HgIus5vbTNTKM2R1ePKuGmx7yKV8gjkvRTf4Vwb8uNwaJFu/lPzF2nRY7fm UZvcpmuVOboiJ6WhtbyDl4UExO6sHjU8H10jpd+zKz+tWH0AuzYBjHVf9FzbECpVIydD ScMiJEi0XoloYYX3Qzbumq7VqHPuTpjeVl/nWTyXb8Y8XUeL8iYIPG5skXSWra2/FKHe Npq/x90OBPTvmDVmkWK2JBn0UEHSmByVGl5MasaNRsM/bOEKc/quTAJ0nM7Dw78K2L3/ 4eAg== X-Gm-Message-State: AOJu0YyTJbwUfcYmK2Yu52RLw69f8PCDlr0RzaR0jnk7SNLkRMYeTrwy 2NAmqFU/waQqq2BN1xbBj54= X-Google-Smtp-Source: AGHT+IGfgDJwGyRHAhvicjmHZpf0ti1TIzSKi22ZY/RsAKF/XoPf4CH+u9bWyFzhgzG2qacWocGPCg== X-Received: by 2002:a05:6a00:2990:b0:68f:b5a3:5ec6 with SMTP id cj16-20020a056a00299000b0068fb5a35ec6mr5233874pfb.0.1699985074560; Tue, 14 Nov 2023 10:04:34 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:0:1000:8411:2278:ad72:cefb:4d49]) by smtp.gmail.com with ESMTPSA id bq4-20020a056a02044400b0059d6f5196fasm5101937pgb.78.2023.11.14.10.04.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 10:04:34 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , "James E.J. Bottomley" , Matthias Brugger , AngeloGioacchino Del Regno Subject: [PATCH v5 2/3] scsi: core: Support disabling fair tag sharing Date: Tue, 14 Nov 2023 10:04:16 -0800 Message-ID: <20231114180426.1184601-3-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog In-Reply-To: <20231114180426.1184601-1-bvanassche@acm.org> References: <20231114180426.1184601-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allow SCSI drivers to disable the block layer fair tag sharing algorithm. Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche --- drivers/scsi/hosts.c | 1 + drivers/scsi/scsi_lib.c | 2 ++ include/scsi/scsi_host.h | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c index d7f51b84f3c7..872f87001374 100644 --- a/drivers/scsi/hosts.c +++ b/drivers/scsi/hosts.c @@ -442,6 +442,7 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv shost->no_write_same = sht->no_write_same; shost->host_tagset = sht->host_tagset; shost->queuecommand_may_block = sht->queuecommand_may_block; + shost->disable_fair_tag_sharing = sht->disable_fair_tag_sharing; if (shost_eh_deadline == -1 || !sht->eh_host_reset_handler) shost->eh_deadline = -1; diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index cf3864f72093..291fbfacf542 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1984,6 +1984,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost) BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy); if (shost->queuecommand_may_block) tag_set->flags |= BLK_MQ_F_BLOCKING; + if (shost->disable_fair_tag_sharing) + tag_set->flags |= BLK_MQ_F_DISABLE_FAIR_TAG_SHARING; tag_set->driver_data = shost; if (shost->host_tagset) tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED; diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 3b907fc2ef08..04238ae9e22c 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -464,6 +464,9 @@ struct scsi_host_template { /* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */ unsigned queuecommand_may_block:1; + /* See also BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. */ + unsigned disable_fair_tag_sharing:1; + /* * Countdown for host blocking with no commands outstanding. */ @@ -662,6 +665,9 @@ struct Scsi_Host { /* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */ unsigned queuecommand_may_block:1; + /* See also BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. */ + unsigned disable_fair_tag_sharing:1; + /* Host responded with short (<36 bytes) INQUIRY result */ unsigned short_inquiry:1; From patchwork Tue Nov 14 18:04:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 744595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30E9DC4332F for ; Tue, 14 Nov 2023 18:05:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233893AbjKNSFC (ORCPT ); Tue, 14 Nov 2023 13:05:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233966AbjKNSEt (ORCPT ); Tue, 14 Nov 2023 13:04:49 -0500 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F68BD5E; Tue, 14 Nov 2023 10:04:45 -0800 (PST) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1cc1ee2d8dfso53011595ad.3; Tue, 14 Nov 2023 10:04:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699985084; x=1700589884; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VWHGrQLU71IG94aXxynwdk2LkNtFMni7Y7JI8/Qdscc=; b=pxMOYClZl5yVs/x8KQ7MZOUiA/f7vljtQnE0HRmqTxKu6qF5nPUvP84Buao9FR/sYr ZvWzAOUSeezUQasWX8F1kFDMCR6g21bM2yN8BQoaja8+sOzTgw0w7yFF4Ya3qlArjLdx CTEhJSYc+9YNJO/hyE6ae88P3zmfw6izkO9TJOZIRe6/43fPfnMoHOwACKxzYDcw1Biq 8mGZl6GsahciR0RggS8s58qM/Eexc56MGneZNsNHiQlQr5GKgGKSQcKhDHfRJ224MNoD znbM/Su2bvTatezy431+E+xxUJViCRQLA0GYHbh2/NPWOEOIYwPDMAFL47wDlt+t3bKb +pLg== X-Gm-Message-State: AOJu0YzWXgxZ6oMCeW0DiHiAOjxKCcRvoHoy/HFmdLn1oliWlwRzrLLN Lm4EQVo85QQmJL4l7IaWUso= X-Google-Smtp-Source: AGHT+IFrCIOVP+0mYEK72xr4IxarBqmedllQHRz7fEQJ+NsEZxA3JuktK5Hn9fhlhlBxTHog/yfeJA== X-Received: by 2002:a17:902:7d87:b0:1c9:e508:ad54 with SMTP id a7-20020a1709027d8700b001c9e508ad54mr2332638plm.13.1699985084067; Tue, 14 Nov 2023 10:04:44 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:0:1000:8411:2278:ad72:cefb:4d49]) by smtp.gmail.com with ESMTPSA id bq4-20020a056a02044400b0059d6f5196fasm5101937pgb.78.2023.11.14.10.04.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 10:04:43 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Avri Altman , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , "James E.J. Bottomley" , Matthias Brugger , AngeloGioacchino Del Regno , Stanley Chu , Manivannan Sadhasivam , Asutosh Das , "Bao D. Nguyen" , Peter Wang , Bean Huo , Arthur Simchaev Subject: [PATCH v5 3/3] scsi: ufs: Disable fair tag sharing Date: Tue, 14 Nov 2023 10:04:17 -0800 Message-ID: <20231114180426.1184601-4-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog In-Reply-To: <20231114180426.1184601-1-bvanassche@acm.org> References: <20231114180426.1184601-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Disable the block layer fair tag sharing algorithm because it significantly reduces performance of UFS devices with a maximum queue depth of 32. Reviewed-by: Avri Altman Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche --- drivers/ufs/core/ufshcd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 8b1031fb0a44..a2219cbb9720 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -8973,6 +8973,7 @@ static const struct scsi_host_template ufshcd_driver_template = { .max_host_blocked = 1, .track_queue_depth = 1, .skip_settle_delay = 1, + .disable_fair_tag_sharing = 1, .sdev_groups = ufshcd_driver_groups, .rpm_autosuspend_delay = RPM_AUTOSUSPEND_DELAY_MS, };