From patchwork Wed Jun 10 17:29:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 187757 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp688306ilo; Wed, 10 Jun 2020 10:33:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz0qk5hkAfWILmUFZ25RcwWczcjzvRJasydrhO2Gy3yQrlDcg/ywmNx/5PjtHERQJrJLr7L X-Received: by 2002:a17:906:4b18:: with SMTP id y24mr4414667eju.366.1591810412277; Wed, 10 Jun 2020 10:33:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591810412; cv=none; d=google.com; s=arc-20160816; b=nw3N2VDAmBLSj4jnrm+/u/swu1MQe3eIkX4Fwe/j7CHbotf0YoYcAD3Jpzu7Q+Q0jG 4lsZ6Lf16QtArF/3WaxGTauN8W/yXj7KMu7Yqvc2t2isLwHppBXKXAiptWJnkL5J/BTE dpf904CmO28w8m2xX+THbpkOSDwPS3C8sbbEJpGYHIYseBIC1ypl5Qe5mRtk8ZhFpLTv JgbdTdI3CPvcg7PIHUrgSR4j1nUxI1HyurQSn1/n66ktAr88e5tFXVcFtJnoX5fRm1DN yXnN3v3pX0o2cNxQGYwYgW4dSELt2jNjJQnO/uygJCOOqUykoo9XTRjM5NVwJSok6YPc 7zhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:message-id:date:subject:cc :to:from; bh=X96nTnXPYgA/n6tyUBTUNx+NnOMhy9HC0Jq+/cEitF8=; b=xsJ/qqcaXWEgGx5g6cOmlAl9zUyVUnlH4HNAAj9U8HX34qjnB60iPSiPoh3kNttU7v p/UmN7EOREa7AoUnlkriTXcJVZvuByuMZWuMR/dLgVp3ba+wTk7Ve3HlvKWSwCWR2gJa xbNEmG1PJB0yV8CUADRuDQODtPot0INUk3gvlnnE9gG02rdHLUCrE/1NJ/pjX+HpJnRm UgRgCV6egvT7yIjs6M4dFx+gjoATJBXmPPVjkNSTyVCI9egXy94oeOf2ZlihjSnc8to5 VF/Vpsx6mWMgQDdfuhm56+dvhEUBmxDX1rTRR7CsrTkhGy2Av3EdTU4KNntrLb78GR16 mdVA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id oz26si430501ejb.252.2020.06.10.10.33.32 for ; Wed, 10 Jun 2020 10:33:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726435AbgFJRda (ORCPT ); Wed, 10 Jun 2020 13:33:30 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:5805 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726257AbgFJRda (ORCPT ); Wed, 10 Jun 2020 13:33:30 -0400 Received: from DGGEMS409-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id E34FD4BC502FEA3BDBA1; Thu, 11 Jun 2020 01:33:26 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS409-HUB.china.huawei.com (10.3.19.209) with Microsoft SMTP Server id 14.3.487.0; Thu, 11 Jun 2020 01:33:17 +0800 From: John Garry To: , , , , , , , , , , CC: , , , , , John Garry Subject: [PATCH RFC v7 00/12] blk-mq/scsi: Provide hostwide shared tags for SCSI HBAs Date: Thu, 11 Jun 2020 01:29:07 +0800 Message-ID: <1591810159-240929-1-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Hi all, Here is v7 of the patchset. In this version of the series, we keep the shared sbitmap for driver tags, and introduce changes to fix up the tag budgeting across request queues ( and associated debugfs changes). Some performance figures: Using 12x SAS SSDs on hisi_sas v3 hw. mq-deadline results are included, but it is not always an appropriate scheduler to use. Tag depth 4000 (default) 260** Baseline: none sched: 2290K IOPS 894K mq-deadline sched: 2341K IOPS 2313K Final, host_tagset=0 in LLDD* none sched: 2289K IOPS 703K mq-deadline sched: 2337K IOPS 2291K Final: none sched: 2281K IOPS 1101K mq-deadline sched: 2322K IOPS 1278K * this is relevant as this is the performance in supporting but not enabling the feature ** depth=260 is relevant as some point where we are regularly waiting for tags to be available. Figures were are a bit unstable here for testing. A copy of the patches can be found here: https://github.com/hisilicon/kernel-dev/commits/private-topic-blk-mq-shared-tags-rfc-v7 And to progress this series, we the the following to go in first, when ready: https://lore.kernel.org/linux-scsi/20200430131904.5847-1-hare@suse.de/ Comments welcome, thanks! Differences to v6: - tag budgeting across request queues and associated changes - add any reviewed tags - rebase - I did not include any change related to updating shared sbitmap per-hctx wait pointer, based on lack of evidence of performance improvement. This was discussed here originally: https://lore.kernel.org/linux-scsi/ecaeccf029c6fe377ebd4f30f04df9f1@mail.gmail.com/ I may revisit. Differences to v5: - For now, drop the shared scheduler tags - Fix megaraid SAS queue selection and rebase - Omit minor unused arguments patch, which has now been merged - Add separate patch to introduce sbitmap pointer - Fixed hctx_tags_bitmap_show() for shared sbitmap - Various tidying Hannes Reinecke (5): blk-mq: rename blk_mq_update_tag_set_depth() scsi: Add template flag 'host_tagset' megaraid_sas: switch fusion adapters to MQ smartpqi: enable host tagset hpsa: enable host_tagset and switch to MQ John Garry (6): blk-mq: Use pointers for blk_mq_tags bitmap tags blk-mq: Facilitate a shared sbitmap per tagset blk-mq: Record nr_active_requests per queue for when using shared sbitmap blk-mq: Record active_queues_shared_sbitmap per tag_set for when using shared sbitmap blk-mq: Add support in hctx_tags_bitmap_show() for a shared sbitmap scsi: hisi_sas: Switch v3 hw to MQ Ming Lei (1): blk-mq: rename BLK_MQ_F_TAG_SHARED as BLK_MQ_F_TAG_QUEUE_SHARED block/bfq-iosched.c | 4 +- block/blk-core.c | 2 + block/blk-mq-debugfs.c | 120 ++++++++++++++- block/blk-mq-tag.c | 157 ++++++++++++++------ block/blk-mq-tag.h | 21 ++- block/blk-mq.c | 64 +++++--- block/blk-mq.h | 33 +++- block/kyber-iosched.c | 4 +- drivers/scsi/hisi_sas/hisi_sas.h | 3 +- drivers/scsi/hisi_sas/hisi_sas_main.c | 36 ++--- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 87 +++++------ drivers/scsi/hpsa.c | 44 +----- drivers/scsi/hpsa.h | 1 - drivers/scsi/megaraid/megaraid_sas.h | 1 - drivers/scsi/megaraid/megaraid_sas_base.c | 59 +++----- drivers/scsi/megaraid/megaraid_sas_fusion.c | 24 +-- drivers/scsi/scsi_lib.c | 2 + drivers/scsi/smartpqi/smartpqi_init.c | 38 +++-- include/linux/blk-mq.h | 9 +- include/linux/blkdev.h | 3 + include/scsi/scsi_host.h | 6 +- 21 files changed, 463 insertions(+), 255 deletions(-) -- 2.26.2