From patchwork Mon Jan 4 12:33:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 356372 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp14933785jai; Mon, 4 Jan 2021 04:39:52 -0800 (PST) X-Google-Smtp-Source: ABdhPJxkphohEk0JwR5BQzrAyrETxqWPD15/5GFO6q+EOo9c8lBxQY+7LbMGL79bKqmrxS7HdYRu X-Received: by 2002:a17:906:7f0b:: with SMTP id d11mr67144245ejr.7.1609763992435; Mon, 04 Jan 2021 04:39:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609763992; cv=none; d=google.com; s=arc-20160816; b=Xbm3lE4U6BetyXR8S7gUI4qKs+2RyjT1srX/9NLw1ZPok3WShGQG2f/zrANIGGnb3I QGYHaVOyefSEbG8PwJBFuB6QYAPNCmAoOvpEhqZGFUGoOgKPaFHTu24GOmPNQiPmSXnN g/yCAhOBK99basOz9BMeLNx9ZA3itG8mNcPPfEfRNAN1IkBV7yMCWq67Rf75lBqnM97W tpVf2JBGGg8cXNVMRRr7M7kDedeHwXnN6wtIq7Px46XjK7WJxuFVXBdxafQy6fkHseWf T0zPXtCy4zP3kSFLyfTuSUvCmygjrL7RDDUUS67PmGIjaRaCa1cbXZZXStwdqG8Z6eu+ DBcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=lns7U033GOp1YIHr9HV22mEPsWoSmIOsQv4DytSrVuw=; b=WGKK/yPssXMTDG1Cma6J8mxuHjUlJiRb0ICmxNB+aB0WQYzNbjLCxVvb2ePGuB4ZlI k/cxPo/44bUekfk+lRJ79LFMtbFNXYZc0jAN+x2sHnFLWt1mI3PMcppHq4jyumGNfISe /DPeGY3/+69bU1AF+pJ45rbIby08ZNVzc5sH5ylYWpf2+pbsZ7QJ67VSt9ZHIJsCc4Om IDE5fYj+91aa1cxuA5yw2Vb9TTOgvvkcv4RvdLl0VNZiyHuOpdestWciRwEiZ5uoYzZz F6zgqr5vQvu5FDbznG+jm+Lqib3zLffdZeCGSFYHqhTc0kY7Yfr1Yh+vaxTx87P8X5Vu eSVw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dv23si27869679ejb.408.2021.01.04.04.39.52 for ; Mon, 04 Jan 2021 04:39:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726811AbhADMig (ORCPT ); Mon, 4 Jan 2021 07:38:36 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:10107 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726333AbhADMif (ORCPT ); Mon, 4 Jan 2021 07:38:35 -0500 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4D8Zr60PGxzMFB6; Mon, 4 Jan 2021 20:36:38 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.498.0; Mon, 4 Jan 2021 20:37:37 +0800 From: John Garry To: , CC: , , , , , John Garry Subject: [PATCH 1/2] scsi: hisi_sas: Remove auto_affine_msi_experimental module_param Date: Mon, 4 Jan 2021 20:33:41 +0800 Message-ID: <1609763622-34119-2-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1609763622-34119-1-git-send-email-john.garry@huawei.com> References: <1609763622-34119-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Now that the driver always uses managed interrupts, delete auto_affine_msi_experimental module param. Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 5 ----- 1 file changed, 5 deletions(-) -- 2.26.2 diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c index 98e9844eba97..fac805ee4719 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c @@ -517,11 +517,6 @@ static int prot_mask; module_param(prot_mask, int, 0); MODULE_PARM_DESC(prot_mask, " host protection capabilities mask, def=0x0 "); -static bool auto_affine_msi_experimental; -module_param(auto_affine_msi_experimental, bool, 0444); -MODULE_PARM_DESC(auto_affine_msi_experimental, "Enable auto-affinity of MSI IRQs as experimental:\n" - "default is off"); - static void debugfs_work_handler_v3_hw(struct work_struct *work); static u32 hisi_sas_read32(struct hisi_hba *hisi_hba, u32 off) From patchwork Mon Jan 4 12:33:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 356374 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp14933804jai; Mon, 4 Jan 2021 04:39:54 -0800 (PST) X-Google-Smtp-Source: ABdhPJzSp49S/0dRzQpWIwG1/+1ZSsDZaOVDzGV3dJfirC9qfEQ9L8iGrjxRqQ9GB1EoKJzGDfNa X-Received: by 2002:a17:906:1488:: with SMTP id x8mr54137504ejc.124.1609763994323; Mon, 04 Jan 2021 04:39:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609763994; cv=none; d=google.com; s=arc-20160816; b=fQAUcwrCxQ3TZyxZPbMIfIbKgDA92Hdp2t5szl0l83q1QJnFYXj/PsMfWFgrvGYh3z b8U1uUGRx/L+jYRRhyv8nFRy3hdKfEbmtp3abB9C66yjRQmuNQn6yAgJB/F+fKgQpuCl mzha+oYHyp73sH7vCGuyoFSNillXtRDqDyY79Yr4ZB8zPzmoQ30rybTfy7K/ptn+UNgX Nsth2beYi/B97qTy4XY9YPUr0Ng+Vo2o39fLetcHS+rmlO1WBWxIX8cX7eh8Pt2fsgg/ IJqp1r3oIZSIAI3IvrQdCbGiawPChU9Hbz/rcsNcMXzJ/AWb51qACU9YLWHTUQs0pGIb ySvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=QMnMYHsobE30AYGk2IiXYt6+puvzansfKU0QRkuNzpI=; b=qBG8adK/NXEKZ9aW16KtLyYM2pKFt65qC135OQG5WZt3Lthkf82owvuZa6YN+Cp8Pq PtirQJ6VDTScBnfPY6UFClZZtdIqeUrVHt+YC9wxfrFHOOLDh22VezWtRQ6VEnkgGx/o RHgtYMszcX4y/uynnYnhEFR93EfMV9PFmUli2FHOTLRSnZlSK8+4C9hBph7d5uDL3D9Y Cd0eBmATTPvJ9/Ai6M/uIY9dKVVUj/y5GaiGXUxU4HV6MWHcck2rC4PD9Fk6gAenWoA6 ATOmoxe0NEyd5BfpNZ8bu4JKSErFK6vWHHeqPAKS+nW5MeyH3Y/nLFtM68YzzsZ69wDi 4OIQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dv23si27869679ejb.408.2021.01.04.04.39.54 for ; Mon, 04 Jan 2021 04:39:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726673AbhADMic (ORCPT ); Mon, 4 Jan 2021 07:38:32 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:10542 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726365AbhADMic (ORCPT ); Mon, 4 Jan 2021 07:38:32 -0500 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4D8Zr60rDKzMFBV; Mon, 4 Jan 2021 20:36:38 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.498.0; Mon, 4 Jan 2021 20:37:38 +0800 From: John Garry To: , CC: , , , , , John Garry Subject: [PATCH 2/2] scsi: hisi_sas: Expose HW queues for v2 hw Date: Mon, 4 Jan 2021 20:33:42 +0800 Message-ID: <1609763622-34119-3-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1609763622-34119-1-git-send-email-john.garry@huawei.com> References: <1609763622-34119-1-git-send-email-john.garry@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org As a performance enhancement, make the completion queue interrupts managed. In addition, in commit bf0beec0607d ("blk-mq: drain I/O when all CPUs in a hctx are offline"), CPU hotplug for MQ devices using managed interrupts is made safe. So expose HW queues to blk-mq to take advantage of this. Flag Scsi_host.host_tagset is also set to ensure that the HBA is not sent more commands than it can handle. However the driver still does not use request tag for IPTT as there are many HW bugs means that special rules apply for IPTT allocation. Signed-off-by: John Garry --- drivers/scsi/hisi_sas/hisi_sas.h | 4 ++ drivers/scsi/hisi_sas/hisi_sas_main.c | 11 +++++ drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 66 +++++++++++++++++++++----- 3 files changed, 68 insertions(+), 13 deletions(-) -- 2.26.2 diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h index 2b28dd405600..e821dd32dd28 100644 --- a/drivers/scsi/hisi_sas/hisi_sas.h +++ b/drivers/scsi/hisi_sas/hisi_sas.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -294,6 +295,7 @@ enum { struct hisi_sas_hw { int (*hw_init)(struct hisi_hba *hisi_hba); + int (*interrupt_preinit)(struct hisi_hba *hisi_hba); void (*setup_itct)(struct hisi_hba *hisi_hba, struct hisi_sas_device *device); int (*slot_index_alloc)(struct hisi_hba *hisi_hba, @@ -393,6 +395,8 @@ struct hisi_hba { u32 refclk_frequency_mhz; u8 sas_addr[SAS_ADDR_SIZE]; + int *irq_map; /* v2 hw */ + int n_phy; spinlock_t lock; struct semaphore sem; diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c index c34f60a600f1..ffb2b32fd35f 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_main.c +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c @@ -2628,6 +2628,13 @@ static struct Scsi_Host *hisi_sas_shost_alloc(struct platform_device *pdev, return NULL; } +static int hisi_sas_interrupt_preinit(struct hisi_hba *hisi_hba) +{ + if (hisi_hba->hw->interrupt_preinit) + return hisi_hba->hw->interrupt_preinit(hisi_hba); + return 0; +} + int hisi_sas_probe(struct platform_device *pdev, const struct hisi_sas_hw *hw) { @@ -2685,6 +2692,10 @@ int hisi_sas_probe(struct platform_device *pdev, sha->sas_port[i] = &hisi_hba->port[i].sas_port; } + rc = hisi_sas_interrupt_preinit(hisi_hba); + if (rc) + goto err_out_ha; + rc = scsi_add_host(shost, &pdev->dev); if (rc) goto err_out_ha; diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c index b57177b52fac..9adfdefef9ca 100644 --- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c +++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c @@ -3302,6 +3302,28 @@ static irq_handler_t fatal_interrupts[HISI_SAS_FATAL_INT_NR] = { fatal_axi_int_v2_hw }; +#define CQ0_IRQ_INDEX (96) + +static int hisi_sas_v2_interrupt_preinit(struct hisi_hba *hisi_hba) +{ + struct platform_device *pdev = hisi_hba->platform_dev; + struct Scsi_Host *shost = hisi_hba->shost; + struct irq_affinity desc = { + .pre_vectors = CQ0_IRQ_INDEX, + .post_vectors = 16, + }; + int resv = desc.pre_vectors + desc.post_vectors, minvec = resv + 1, nvec; + + nvec = devm_platform_get_irqs_affinity(pdev, &desc, minvec, 128, + &hisi_hba->irq_map); + if (nvec < 0) + return nvec; + + shost->nr_hw_queues = hisi_hba->cq_nvecs = nvec - resv; + + return 0; +} + /* * There is a limitation in the hip06 chipset that we need * to map in all mbigen interrupts, even if they are not used. @@ -3310,14 +3332,11 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba) { struct platform_device *pdev = hisi_hba->platform_dev; struct device *dev = &pdev->dev; - int irq, rc = 0, irq_map[128]; + int irq, rc = 0; int i, phy_no, fatal_no, queue_no; - for (i = 0; i < 128; i++) - irq_map[i] = platform_get_irq(pdev, i); - for (i = 0; i < HISI_SAS_PHY_INT_NR; i++) { - irq = irq_map[i + 1]; /* Phy up/down is irq1 */ + irq = hisi_hba->irq_map[i + 1]; /* Phy up/down is irq1 */ rc = devm_request_irq(dev, irq, phy_interrupts[i], 0, DRV_NAME " phy", hisi_hba); if (rc) { @@ -3331,7 +3350,7 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba) for (phy_no = 0; phy_no < hisi_hba->n_phy; phy_no++) { struct hisi_sas_phy *phy = &hisi_hba->phy[phy_no]; - irq = irq_map[phy_no + 72]; + irq = hisi_hba->irq_map[phy_no + 72]; rc = devm_request_irq(dev, irq, sata_int_v2_hw, 0, DRV_NAME " sata", phy); if (rc) { @@ -3343,7 +3362,7 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba) } for (fatal_no = 0; fatal_no < HISI_SAS_FATAL_INT_NR; fatal_no++) { - irq = irq_map[fatal_no + 81]; + irq = hisi_hba->irq_map[fatal_no + 81]; rc = devm_request_irq(dev, irq, fatal_interrupts[fatal_no], 0, DRV_NAME " fatal", hisi_hba); if (rc) { @@ -3354,24 +3373,22 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba) } } - for (queue_no = 0; queue_no < hisi_hba->queue_count; queue_no++) { + for (queue_no = 0; queue_no < hisi_hba->cq_nvecs; queue_no++) { struct hisi_sas_cq *cq = &hisi_hba->cq[queue_no]; - cq->irq_no = irq_map[queue_no + 96]; + cq->irq_no = hisi_hba->irq_map[queue_no + 96]; rc = devm_request_threaded_irq(dev, cq->irq_no, cq_interrupt_v2_hw, cq_thread_v2_hw, IRQF_ONESHOT, DRV_NAME " cq", cq); if (rc) { dev_err(dev, "irq init: could not request cq interrupt %d, rc=%d\n", - irq, rc); + cq->irq_no, rc); rc = -ENOENT; goto err_out; } + cq->irq_mask = irq_get_affinity_mask(cq->irq_no); } - - hisi_hba->cq_nvecs = hisi_hba->queue_count; - err_out: return rc; } @@ -3529,6 +3546,26 @@ static struct device_attribute *host_attrs_v2_hw[] = { NULL }; +static int map_queues_v2_hw(struct Scsi_Host *shost) +{ + struct hisi_hba *hisi_hba = shost_priv(shost); + struct blk_mq_queue_map *qmap = &shost->tag_set.map[HCTX_TYPE_DEFAULT]; + const struct cpumask *mask; + unsigned int queue, cpu; + + for (queue = 0; queue < qmap->nr_queues; queue++) { + mask = irq_get_affinity_mask(hisi_hba->irq_map[96 + queue]); + if (!mask) + continue; + + for_each_cpu(cpu, mask) + qmap->mq_map[cpu] = qmap->queue_offset + queue; + } + + return 0; + +} + static struct scsi_host_template sht_v2_hw = { .name = DRV_NAME, .proc_name = DRV_NAME, @@ -3553,10 +3590,13 @@ static struct scsi_host_template sht_v2_hw = { #endif .shost_attrs = host_attrs_v2_hw, .host_reset = hisi_sas_host_reset, + .map_queues = map_queues_v2_hw, + .host_tagset = 1, }; static const struct hisi_sas_hw hisi_sas_v2_hw = { .hw_init = hisi_sas_v2_init, + .interrupt_preinit = hisi_sas_v2_interrupt_preinit, .setup_itct = setup_itct_v2_hw, .slot_index_alloc = slot_index_alloc_quirk_v2_hw, .alloc_dev = alloc_dev_quirk_v2_hw,