From patchwork Thu Dec 3 02:07:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337416 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFBA3C64E7C for ; Thu, 3 Dec 2020 02:09:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4FE6B21D91 for ; Thu, 3 Dec 2020 02:09:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727872AbgLCCJC (ORCPT ); Wed, 2 Dec 2020 21:09:02 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:16990 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729411AbgLCCJC (ORCPT ); Wed, 2 Dec 2020 21:09:02 -0500 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B321hco159395; Wed, 2 Dec 2020 21:08:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=THr3PwzrisLGs197EgpbEA4rC/U8om6fT6y4XZnnpL0=; b=C16QrCKubYmpygmHlMZ0grJMXqpJFgSA4eO6LZWrq3AGpDV4G2mn9q5zZ8jr96lVdH9y 8QaUytZ5cBDdX1IMU6CwEEXge9UIPC46zk8M5/e7zfJZRtuLJW+eTg/eraDQzJaEVTvz n5aPLVZgcXM+tKTcmdurye1JEsHZgPWl6EKlZCuf0O9qnW/BeFLmGS3JY7v2SYikz7Ji U5J0Np1i5fK2DawWmzeNKfme6HU0t4pHlcUNywFfmUfxWiO0wrz+WK+3mnSgsWvOqAuS n/AnZRwiyXQftW/PzRUBj59/03QOL8ihzJJjQBBqNap+/ZN7TxmyJ+AkZsiP6tM4dM6i pg== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 3569tgx4ku-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:12 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uStT006206; Thu, 3 Dec 2020 02:08:11 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma03dal.us.ibm.com with ESMTP id 353e69pd7w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:11 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328AtF62390658 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:10 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0343D78063; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 77A847805C; Thu, 3 Dec 2020 02:08:09 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:09 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 01/18] ibmvfc: add vhost fields and defaults for MQ enablement Date: Wed, 2 Dec 2020 20:07:49 -0600 Message-Id: <20201203020806.14747-2-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 suspectscore=1 clxscore=1015 lowpriorityscore=0 adultscore=0 spamscore=0 impostorscore=0 bulkscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030006 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce several new vhost fields for managing MQ state of the adapter as well as initial defaults for MQ enablement. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 9 ++++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 13 +++++++++++-- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 42e4d35e0d35..f1d677a7423d 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -5161,12 +5161,13 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) } shost->transportt = ibmvfc_transport_template; - shost->can_queue = max_requests; + shost->can_queue = (max_requests / IBMVFC_SCSI_HW_QUEUES); shost->max_lun = max_lun; shost->max_id = max_targets; shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; + shost->nr_hw_queues = IBMVFC_SCSI_HW_QUEUES; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->sent); @@ -5178,6 +5179,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->partition_number = -1; vhost->log_level = log_level; vhost->task_set = 1; + + vhost->mq_enabled = IBMVFC_MQ; + vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->using_channels = 0; + vhost->do_enquiry = 1; + strcpy(vhost->partition_name, "UNKNOWN"); init_waitqueue_head(&vhost->work_wait_q); init_waitqueue_head(&vhost->init_wait_q); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 9d58cfd774d3..e095daada70e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,16 +41,21 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 +#define IBMVFC_MQ 0 +#define IBMVFC_SCSI_CHANNELS 0 +#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MIG_NO_SUB_TO_CRQ 0 +#define IBMVFC_MIG_NO_N_TO_M 0 /* * Ensure we have resources for ERP and initialization: - * 1 for ERP * 1 for initialization * 1 for NPIV Logout * 2 for BSG passthru * 2 for each discovery thread + * 1 ERP for each possible HW Queue */ -#define IBMVFC_NUM_INTERNAL_REQ (1 + 1 + 1 + 2 + (disc_threads * 2)) +#define IBMVFC_NUM_INTERNAL_REQ (1 + 1 + 2 + (disc_threads * 2) + IBMVFC_SCSI_HW_QUEUES) #define IBMVFC_MAD_SUCCESS 0x00 #define IBMVFC_MAD_NOT_SUPPORTED 0xF1 @@ -826,6 +831,10 @@ struct ibmvfc_host { int delay_init; int scan_complete; int logged_in; + int mq_enabled; + int using_channels; + int do_enquiry; + int client_scsi_channels; int aborting_passthru; int events_to_log; #define IBMVFC_AE_LINKUP 0x0001 From patchwork Thu Dec 3 02:07:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337410 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB6E2C8303C for ; Thu, 3 Dec 2020 02:09:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A1A1E221FB for ; Thu, 3 Dec 2020 02:09:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729598AbgLCCJj (ORCPT ); Wed, 2 Dec 2020 21:09:39 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:49810 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728279AbgLCCJB (ORCPT ); Wed, 2 Dec 2020 21:09:01 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322kER095952; Wed, 2 Dec 2020 21:08:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=FikaVMSWv9fzh3+QqH1htxIk/scNAtSYHyhJCHaSiXA=; b=FxqiK1f1dHlPpF5SniKkC5nugqEFuOnj3tNk4h1HmjUPO/tjRgNiVnno345a6tZQi0Fv /CnoLE6XCliqrTgUr6veQtt1u9011PT/0kKgUSOOJO2tSdq2vXBtHJ/XBfiTNFnK7gfH IL/NBn2NaSqSvgNwDgCm94Ac9cCegX7hPAEQOILgF3dHwQcgwC3ACGm5uET7NJVbVWW1 cSyzHJIjIAcjBnVfz8yUruEQ09pX2TIcnoEGTCJBHXHYNwgScRMTJ91z7uNBuU5zEwGb s2sPNHscNITmhZV0yRZeAJPmlogPPoHhV9fd60EMlOlghWtkJximV+uCmKA4nWIRbX8v Gg== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxc9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:13 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327Wme010529; Thu, 3 Dec 2020 02:08:12 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01dal.us.ibm.com with ESMTP id 355rf7px9q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:12 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328A5g11338290 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:10 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B294A78063; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D69678064; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 02/18] ibmvfc: define hcall wrapper for registering a Sub-CRQ Date: Wed, 2 Dec 2020 20:07:50 -0600 Message-Id: <20201203020806.14747-3-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=1 mlxlogscore=999 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Sub-CRQs are registred with firmware via a hypercall. Abstract that interface into a simpler helper function. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index f1d677a7423d..64674054dbae 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -138,6 +138,20 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *); static const char *unknown_error = "unknown error"; +static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba, + unsigned long length, unsigned long *cookie, + unsigned long *irq) +{ + unsigned long retbuf[PLPAR_HCALL_BUFSIZE]; + long rc; + + rc = plpar_hcall(H_REG_SUB_CRQ, retbuf, unit_address, ioba, length); + *cookie = retbuf[0]; + *irq = retbuf[1]; + + return rc; +} + static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags) { u64 host_caps = be64_to_cpu(vhost->login_buf->resp.capabilities); From patchwork Thu Dec 3 02:07:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AD78C8303B for ; Thu, 3 Dec 2020 02:09:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 38AF7221EB for ; Thu, 3 Dec 2020 02:09:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729612AbgLCCJm (ORCPT ); Wed, 2 Dec 2020 21:09:42 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:48938 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728252AbgLCCJB (ORCPT ); Wed, 2 Dec 2020 21:09:01 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322kET095952; Wed, 2 Dec 2020 21:08:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=quLqGqXAl/PPgl+TuyjYAEjbjck4OUuu3Qk2jmO0lrg=; b=HfTpzJrch9k/+CR54cx6q7JB0qQwTKPrSfs+cZOrQTtV762n2SsjbI0Xv01XDUtJXR7L ATk6jsf0xEHd402U08XUANGteVtKMOottZqDIXb8QRuQ6ziLK0JccB06EdRQaonYIgZ1 IhLHZh4AyGEMXd5+UaTSxYg8G20hd6CjaUM1ffvfpq4N+uatfeamBnK83XSpjJWw2lqD /MlvAgLSf9K1y/Bk6CH4Efk3NbK8ukqny+ZkI/olCQKJo8OYobRYZkjPtmfP7tfnTXA7 uKg3ledqaZYQF2DkM51CukUCuAisejdpNx/vvbil0j4OlIFAHEyil/eg+xsEz5gOw6VS LQ== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxcg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:14 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uLTO013319; Thu, 3 Dec 2020 02:08:12 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfvc7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:12 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328BQ221102922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:11 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6561178064; Thu, 3 Dec 2020 02:08:11 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CD0EE7805F; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:10 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 03/18] ibmvfc: add Subordinate CRQ definitions Date: Wed, 2 Dec 2020 20:07:51 -0600 Message-Id: <20201203020806.14747-4-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=1 mlxlogscore=999 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Subordinate Command Response Queues (Sub CRQ) are used in conjunction with the primary CRQ when more than one queue is needed by the virtual IO adapter. Recent phyp firmware versions support Sub CRQ's with ibmvfc adapters. This feature is a prerequisite for supporting multiple hardware backed submission queues in the vfc adapter. The Sub CRQ command element differs from the standard CRQ in that it is 32bytes long as opposed to 16bytes for the latter. Despite this extra 16bytes the ibmvfc protocol will use the original CRQ command element mapped to the first 16bytes of the Sub CRQ element initially. Add definitions for the Sub CRQ command element and queue. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index e095daada70e..b3cd35cbf067 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -656,6 +656,29 @@ struct ibmvfc_crq_queue { dma_addr_t msg_token; }; +struct ibmvfc_sub_crq { + struct ibmvfc_crq crq; + __be64 reserved[2]; +} __packed __aligned(8); + +struct ibmvfc_sub_queue { + struct ibmvfc_sub_crq *msgs; + dma_addr_t msg_token; + int size, cur; + struct ibmvfc_host *vhost; + unsigned long cookie; + unsigned long vios_cookie; + unsigned long hw_irq; + unsigned long irq; + unsigned long hwq_id; + char name[32]; +}; + +struct ibmvfc_scsi_channels { + struct ibmvfc_sub_queue *scrqs; + unsigned int active_queues; +}; + enum ibmvfc_ae_link_state { IBMVFC_AE_LS_LINK_UP = 0x01, IBMVFC_AE_LS_LINK_BOUNCED = 0x02, From patchwork Thu Dec 3 02:07:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337408 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE8D3C83013 for ; Thu, 3 Dec 2020 02:10:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F90122201 for ; Thu, 3 Dec 2020 02:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729609AbgLCCJm (ORCPT ); Wed, 2 Dec 2020 21:09:42 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:13160 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728151AbgLCCJB (ORCPT ); Wed, 2 Dec 2020 21:09:01 -0500 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B326YKE065479; Wed, 2 Dec 2020 21:08:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=2Uh9hTMUc0YdAyAUP0RhYH8A17ttsv2M0nBTdSYt2zo=; b=WKTh/3A2LvWOQ6/ruEvyr10tj0H1Ta6aCR5S6/TpWQkjTtSsuL+nZXhrbtH18j846zxB FpwsranvkRXQhYUymNx6kthvj2nPlmslg43qE8n02aotve+1XYlxnPWrE8fEju/qC4KK 4xNzSb4Hnh2XJMxb60O2wCH6cxauQU/NP5QQ4w6pjnkSmN7LsJfhTgReRkZigNk5lKFL X7/K99jnm1wTYx0sdfmGexlxeZ9SA0XUgz8njFYEr9HAmhp8kabdGx8KajIESssd0DZA 8PM/5D3JLKNVw3DuTQpwT74eaJw46WXvwVjiFJEr8G6T6SBgEFo77DanbOSVUXbdePFH LQ== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jg86uh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:14 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31xdPP013019; Thu, 3 Dec 2020 02:08:13 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma02wdc.us.ibm.com with ESMTP id 356cbem9py-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:13 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328CpU22086060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:12 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 179C678063; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8CD767805C; Thu, 3 Dec 2020 02:08:11 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:11 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 04/18] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Date: Wed, 2 Dec 2020 20:07:52 -0600 Message-Id: <20201203020806.14747-5-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 lowpriorityscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 impostorscore=0 phishscore=0 suspectscore=3 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allocate a set of Sub-CRQs in advance. During channel setup the client and VIOS negotiate the number of queues the VIOS supports and the number that the client desires to request. Its possible that the final channel resources allocated is less than requested, but the client is still responsible for sending handles for every queue it is hoping for. Also, provide deallocation cleanup routines. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 129 +++++++++++++++++++++++++++++++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 130 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 64674054dbae..f879be666c84 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -793,6 +793,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) unsigned long flags; struct vio_dev *vdev = to_vio_dev(vhost->dev); struct ibmvfc_crq_queue *crq = &vhost->crq; + struct ibmvfc_sub_queue *scrq; + int i; /* Close the CRQ */ do { @@ -809,6 +811,14 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) memset(crq->msgs, 0, PAGE_SIZE); crq->cur = 0; + if (vhost->scsi_scrqs.scrqs) { + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + scrq = &vhost->scsi_scrqs.scrqs[i]; + memset(scrq->msgs, 0, PAGE_SIZE); + scrq->cur = 0; + } + } + /* And re-open it again */ rc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, crq->msg_token, PAGE_SIZE); @@ -4983,6 +4993,118 @@ static int ibmvfc_init_crq(struct ibmvfc_host *vhost) return retrc; } +static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, + int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_sub_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + int rc = -ENOMEM; + + ENTER; + + scrq->msgs = (struct ibmvfc_sub_crq *)get_zeroed_page(GFP_KERNEL); + if (!scrq->msgs) + return rc; + + scrq->size = PAGE_SIZE / sizeof(*scrq->msgs); + scrq->msg_token = dma_map_single(dev, scrq->msgs, PAGE_SIZE, + DMA_BIDIRECTIONAL); + + if (dma_mapping_error(dev, scrq->msg_token)) + goto dma_map_failed; + + rc = h_reg_sub_crq(vdev->unit_address, scrq->msg_token, PAGE_SIZE, + &scrq->cookie, &scrq->hw_irq); + + if (rc) { + dev_warn(dev, "Error registering sub-crq: %d\n", rc); + if (rc == H_PARAMETER) + dev_warn_once(dev, "Firmware may not support MQ\n"); + goto reg_failed; + } + + scrq->hwq_id = index; + scrq->vhost = vhost; + + LEAVE; + return 0; + +reg_failed: + dma_unmap_single(dev, scrq->msg_token, PAGE_SIZE, DMA_BIDIRECTIONAL); +dma_map_failed: + free_page((unsigned long)scrq->msgs); + LEAVE; + return rc; +} + +static void ibmvfc_deregister_scsi_channel(struct ibmvfc_host *vhost, int index) +{ + struct device *dev = vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + struct ibmvfc_sub_queue *scrq = &vhost->scsi_scrqs.scrqs[index]; + long rc; + + ENTER; + + do { + rc = plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, + scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); + + if (rc) + dev_err(dev, "Failed to free sub-crq[%d]: rc=%ld\n", index, rc); + + dma_unmap_single(dev, scrq->msg_token, PAGE_SIZE, DMA_BIDIRECTIONAL); + free_page((unsigned long)scrq->msgs); + LEAVE; +} + +static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) +{ + int i, j; + + ENTER; + + vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + sizeof(*vhost->scsi_scrqs.scrqs), + GFP_KERNEL); + if (!vhost->scsi_scrqs.scrqs) + return -1; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + if (ibmvfc_register_scsi_channel(vhost, i)) { + for (j = i; j > 0; j--) + ibmvfc_deregister_scsi_channel(vhost, j - 1); + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; + return -1; + } + } + + LEAVE; + return 0; +} + +static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) +{ + int i; + + ENTER; + if (!vhost->scsi_scrqs.scrqs) + return; + + for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + ibmvfc_deregister_scsi_channel(vhost, i); + + kfree(vhost->scsi_scrqs.scrqs); + vhost->scsi_scrqs.scrqs = NULL; + vhost->scsi_scrqs.active_queues = 0; + LEAVE; +} + /** * ibmvfc_free_mem - Free memory for vhost * @vhost: ibmvfc host struct @@ -5239,6 +5361,12 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) goto remove_shost; } + if (vhost->mq_enabled) { + rc = ibmvfc_init_sub_crqs(vhost); + if (rc) + dev_warn(dev, "Failed to allocate Sub-CRQs. rc=%d\n", rc); + } + if (shost_to_fc_host(shost)->rqst_q) blk_queue_max_segments(shost_to_fc_host(shost)->rqst_q, 1); dev_set_drvdata(dev, vhost); @@ -5296,6 +5424,7 @@ static int ibmvfc_remove(struct vio_dev *vdev) ibmvfc_purge_requests(vhost, DID_ERROR); spin_unlock_irqrestore(vhost->host->host_lock, flags); ibmvfc_free_event_pool(vhost); + ibmvfc_release_sub_crqs(vhost); ibmvfc_free_mem(vhost); spin_lock(&ibmvfc_driver_lock); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index b3cd35cbf067..986ce4530382 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -838,6 +838,7 @@ struct ibmvfc_host { mempool_t *tgt_pool; struct ibmvfc_crq_queue crq; struct ibmvfc_async_crq_queue async_crq; + struct ibmvfc_scsi_channels scsi_scrqs; struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; From patchwork Thu Dec 3 02:07:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5CF0C83036 for ; Thu, 3 Dec 2020 02:09:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C5E422203 for ; Thu, 3 Dec 2020 02:09:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729415AbgLCCJB (ORCPT ); Wed, 2 Dec 2020 21:09:01 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11392 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728142AbgLCCJA (ORCPT ); Wed, 2 Dec 2020 21:09:00 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322lbo095985; Wed, 2 Dec 2020 21:08:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=0vrJkpxV7J6wUU6nHBy8wxjwXBtmUbHss6o/M2/SEuI=; b=GRCHzgZED+MHc7NVpt8I+4MfvrqoLpnealIRqV5G8YuuwbxFlfL0JoocBUK/y8K2jbt+ Qt/8Bw2ms0jHnINmTVVHgra4LPP27u19o4ddOK8fSOhAFhSllxG1RG6w7/Yp2IjjbS0k 4e8PCyDjFnQtSz5LL/KUQozRRoZdpfdH6r/SMXstiKf9kbxm2LqYGUjOZuN3PBFJfH5O remkjn/KixdVebJZVriU9PD6kH6Q3iD+KoErubcvTecj7KHkgaE311fWp4D234CTl6GA jH0dVWC1vO0iFnvvJIeRqJVJHKOcFLoPJD2+p33rdFJ0upfOPSFzRbzZsbpeijNlOUDl Ow== Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxdc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:15 -0500 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327YJZ004616; Thu, 3 Dec 2020 02:08:14 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma04wdc.us.ibm.com with ESMTP id 354ysup6px-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:14 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328CJm25559308 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:12 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C88AF78066; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 31E4178060; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 05/18] ibmvfc: add Sub-CRQ IRQ enable/disable routine Date: Wed, 2 Dec 2020 20:07:53 -0600 Message-Id: <20201203020806.14747-6-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=1 mlxlogscore=766 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Each Sub-CRQ has its own interrupt. A hypercall is required to toggle the IRQ state. Provide the necessary mechanism via a helper function. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index f879be666c84..e082935f56cf 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3361,6 +3361,26 @@ static void ibmvfc_tasklet(void *data) spin_unlock_irqrestore(vhost->host->host_lock, flags); } +static int ibmvfc_toggle_scrq_irq(struct ibmvfc_sub_queue *scrq, int enable) +{ + struct device *dev = scrq->vhost->dev; + struct vio_dev *vdev = to_vio_dev(dev); + unsigned long rc; + int irq_action = H_ENABLE_VIO_INTERRUPT; + + if (!enable) + irq_action = H_DISABLE_VIO_INTERRUPT; + + rc = plpar_hcall_norets(H_VIOCTL, vdev->unit_address, irq_action, + scrq->hw_irq, 0, 0); + + if (rc) + dev_err(dev, "Couldn't %s sub-crq[%lu] irq. rc=%ld\n", + enable ? "enable" : "disable", scrq->hwq_id, rc); + + return rc; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Dec 3 02:07:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F1F9C71156 for ; Thu, 3 Dec 2020 02:10:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4690D221FB for ; Thu, 3 Dec 2020 02:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729297AbgLCCJA (ORCPT ); Wed, 2 Dec 2020 21:09:00 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44658 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727790AbgLCCJA (ORCPT ); Wed, 2 Dec 2020 21:09:00 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322lUd096065; Wed, 2 Dec 2020 21:08:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=1RKZ2SdB/Y/n99fWxar2L0vt9zLjUEVpvzulfL68jLk=; b=X8eBvpAMBlm7CO9WCuVmUDgROjQwUUUgsRmPPL1WdxmHBlwJ1VqBgrfAUayXDjA663KJ cC0MlfNlAW33VsnNG7+pk3M599PxUcoI/oVZzwqhEWAS2Fy02Y03y0GMH1N3DEV7HFAR G96lM0y5zyesH1Wn7w7WSc/1Eniq4MYdH7KSLTk1i1Tuo+JadX5Lf3ZY+kBIuhvIAhdP TsTkC9aIK97W7J2gSDFQeox2kkVSlmpaXrGb8WQ5jNKizluALqW1P+KpfuetW19cAOPe TtjBl9EiYdMqxHwQWwZCKnZ+gEyk1Z6fSQ7zpCM21GGep0PggbSfsmkxLottBQMu/phg iQ== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxdj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:16 -0500 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327SU9013205; Thu, 3 Dec 2020 02:08:14 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma03wdc.us.ibm.com with ESMTP id 353e69df1g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:14 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Dv820447530 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:13 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 654C07805E; Thu, 3 Dec 2020 02:08:13 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E35CE78068; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:12 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 06/18] ibmvfc: add handlers to drain and complete Sub-CRQ responses Date: Wed, 2 Dec 2020 20:07:54 -0600 Message-Id: <20201203020806.14747-7-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=3 mlxlogscore=999 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The logic for iterating over the Sub-CRQ responses is similiar to that of the primary CRQ. Add the necessary handlers for processing those responses. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 80 ++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index e082935f56cf..b61ae1df21e5 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3381,6 +3381,86 @@ static int ibmvfc_toggle_scrq_irq(struct ibmvfc_sub_queue *scrq, int enable) return rc; } +static void ibmvfc_handle_scrq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost) +{ + struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba); + unsigned long flags; + + switch (crq->valid) { + case IBMVFC_CRQ_CMD_RSP: + break; + case IBMVFC_CRQ_XPORT_EVENT: + return; + default: + dev_err(vhost->dev, "Got and invalid message type 0x%02x\n", crq->valid); + return; + } + + /* The only kind of payload CRQs we should get are responses to + * things we send. Make sure this response is to something we + * actually sent + */ + if (unlikely(!ibmvfc_valid_event(&vhost->pool, evt))) { + dev_err(vhost->dev, "Returned correlation_token 0x%08llx is invalid!\n", + crq->ioba); + return; + } + + if (unlikely(atomic_read(&evt->free))) { + dev_err(vhost->dev, "Received duplicate correlation_token 0x%08llx!\n", + crq->ioba); + return; + } + + del_timer(&evt->timer); + list_del(&evt->queue); + ibmvfc_trc_end(evt); + spin_unlock_irqrestore(vhost->host->host_lock, flags); + evt->done(evt); + spin_lock_irqsave(vhost->host->host_lock, flags); +} + +static struct ibmvfc_crq *ibmvfc_next_scrq(struct ibmvfc_sub_queue *scrq) +{ + struct ibmvfc_crq *crq; + + crq = &scrq->msgs[scrq->cur].crq; + if (crq->valid & 0x80) { + if (++scrq->cur == scrq->size) + scrq->cur = 0; + rmb(); + } else + crq = NULL; + + return crq; +} + +static void ibmvfc_drain_sub_crq(struct ibmvfc_sub_queue *scrq) +{ + struct ibmvfc_crq *crq; + unsigned long flags; + int done = 0; + + spin_lock_irqsave(scrq->vhost->host->host_lock, flags); + while (!done) { + while ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_handle_scrq(crq, scrq->vhost); + crq->valid = 0; + wmb(); + } + + ibmvfc_toggle_scrq_irq(scrq, 1); + if ((crq = ibmvfc_next_scrq(scrq)) != NULL) { + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_handle_scrq(crq, scrq->vhost); + crq->valid = 0; + wmb(); + } else + done = 1; + } + spin_unlock_irqrestore(scrq->vhost->host->host_lock, flags); +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Dec 3 02:07:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F92BC83035 for ; Thu, 3 Dec 2020 02:09:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 152A022201 for ; Thu, 3 Dec 2020 02:09:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729583AbgLCCJg (ORCPT ); Wed, 2 Dec 2020 21:09:36 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:21532 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729366AbgLCCJB (ORCPT ); Wed, 2 Dec 2020 21:09:01 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B3250j7128371; Wed, 2 Dec 2020 21:08:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=WvC3IHNQsBmGR4/ZDYjF+he46T9h4kaAaSVMnc33dgs=; b=G+hMoZObQAvc4FldF4d8uW7HoO0xu3KmuqPyaKZsfsO5TXXTKwtZnhT6dTY+oT0MrzGA dxdiW/+6YPsy/b4gBQEC7UB/T0aSj0TKALqW6AUUriB9gnt2EFW5e5P3zc1gwlOIhqJs GOnxoaHC8ah7dSe+5EnqqX3Ua8gzCSmF0sc31EwN+vZ1q/5boRRMUSDD6Atw7rB/BicL w0xs9NA7LzxO4htbM/zQG8XbHkQLpeDr2wNwZZ0Xa0r6nKV26C6jt6FscMDKIi9tvQau JGZpxtaYlT8sjj+2HoXx2QHMTxAHChS5Ws7hYsp74VFwZsOvgyGvKgg+uLmWQh9vxcNT sQ== Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 356p2kh3h7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:16 -0500 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327U5t029816; Thu, 3 Dec 2020 02:08:15 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma02dal.us.ibm.com with ESMTP id 3569xueq80-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:15 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B3284Ad11535052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:04 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2B0D878060; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8978D7805C; Thu, 3 Dec 2020 02:08:13 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:13 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 07/18] ibmvfc: define Sub-CRQ interrupt handler routine Date: Wed, 2 Dec 2020 20:07:55 -0600 Message-Id: <20201203020806.14747-8-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=1 lowpriorityscore=0 mlxlogscore=916 malwarescore=0 clxscore=1015 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Simple handler that calls Sub-CRQ drain routine directly. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index b61ae1df21e5..649268996a5c 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -3461,6 +3461,16 @@ static void ibmvfc_drain_sub_crq(struct ibmvfc_sub_queue *scrq) spin_unlock_irqrestore(scrq->vhost->host->host_lock, flags); } +static irqreturn_t ibmvfc_interrupt_scsi(int irq, void *scrq_instance) +{ + struct ibmvfc_sub_queue *scrq = (struct ibmvfc_sub_queue *)scrq_instance; + + ibmvfc_toggle_scrq_irq(scrq, 0); + ibmvfc_drain_sub_crq(scrq); + + return IRQ_HANDLED; +} + /** * ibmvfc_init_tgt - Set the next init job step for the target * @tgt: ibmvfc target struct From patchwork Thu Dec 3 02:07:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7439BC83024 for ; Thu, 3 Dec 2020 02:09:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3EFC821D91 for ; Thu, 3 Dec 2020 02:09:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727599AbgLCCJE (ORCPT ); Wed, 2 Dec 2020 21:09:04 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:63250 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729461AbgLCCJD (ORCPT ); Wed, 2 Dec 2020 21:09:03 -0500 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B321kdW044458; Wed, 2 Dec 2020 21:08:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=9R+EPmfXOYd059WiPsY59u7q+OrEXF3u9tkNl3psk2o=; b=PbqTstXlEL0uHoWdKroR5Gp+6VZQZ9kAxATCydPNsJm4UdfVx6hN++ehYfR1tGrC8hXt wIdsjewe2Eunilz1MKz0lju9wqFzx+UU3SEdV9fPrp0S044UnIyvg1RLB3FcRGDJkixV TSdUEI+2UapcFuX0G3MG56PoWZW1MkE1iw53MrDcyb8miCuhpK7YNESWa9bglrNzVBew B959JN77pbZuVKM87bGF7m98xMJd0jA/AOxjCDddsPG5f+G26IUOL1LX8fgdUqUPw3jy BUCgTFp7aSM5zmC5xJhkjekHLFBw82NaXb4qn01V0x4RsdGhvJi0Acou4QstKMeLUXJG CQ== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 356jdx73nd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:16 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uOU2013344; Thu, 3 Dec 2020 02:08:16 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfvc80-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:16 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B3285Ru18350810 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:05 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D1E7B7805F; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 45AB07805C; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 08/18] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Date: Wed, 2 Dec 2020 20:07:56 -0600 Message-Id: <20201203020806.14747-9-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 phishscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=3 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030006 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Create an irq mapping for the hw_irq number provided from phyp firmware. Request an irq assigned our Sub-CRQ interrupt handler. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 649268996a5c..6b299df786fc 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -5134,12 +5134,34 @@ static int ibmvfc_register_scsi_channel(struct ibmvfc_host *vhost, goto reg_failed; } + scrq->irq = irq_create_mapping(NULL, scrq->hw_irq); + + if (!scrq->irq) { + rc = -EINVAL; + dev_err(dev, "Error mapping sub-crq[%d] irq\n", index); + goto irq_failed; + } + + snprintf(scrq->name, sizeof(scrq->name), "ibmvfc-%x-scsi%d", + vdev->unit_address, index); + rc = request_irq(scrq->irq, ibmvfc_interrupt_scsi, 0, scrq->name, scrq); + + if (rc) { + dev_err(dev, "Couldn't register sub-crq[%d] irq\n", index); + irq_dispose_mapping(scrq->irq); + goto irq_failed; + } + scrq->hwq_id = index; scrq->vhost = vhost; LEAVE; return 0; +irq_failed: + do { + plpar_hcall_norets(H_FREE_SUB_CRQ, vdev->unit_address, scrq->cookie); + } while (rc == H_BUSY || H_IS_LONG_BUSY(rc)); reg_failed: dma_unmap_single(dev, scrq->msg_token, PAGE_SIZE, DMA_BIDIRECTIONAL); dma_map_failed: From patchwork Thu Dec 3 02:07:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B3D0C8302B for ; Thu, 3 Dec 2020 02:09:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2EDD6221EB for ; Thu, 3 Dec 2020 02:09:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387560AbgLCCJZ (ORCPT ); Wed, 2 Dec 2020 21:09:25 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:60358 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729484AbgLCCJD (ORCPT ); Wed, 2 Dec 2020 21:09:03 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322kkq095936; Wed, 2 Dec 2020 21:08:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=PrWHp6v1nGVxzxhSWuM+ug2bmLfl1sSykYOEHPe8fKw=; b=BwB6DiiYiOKtkhZk1Pg9bSDFDJi11piMmCYzQD8bB8VNUhOG2SC94qXikRJxh9wdVbe9 X2xOQ31p6jMil3VQ6DfEzjXdCJZUFTJh+YMumoZ7EycXRcwsWDJUV8KlPjst3X6BHcUt bHvDOWPYVyV6AO/03FiqxW2WyExSa6OAsMQeYx8eHAdlowZtpbdISsuJNOyVdn7F+zq6 AcgqGcBV4tmvbcetqIFEy4saRIwCq1BSwb/Ztwxoj9ev4plDc8/BhbRqiS7KqtFFpgcF ElPBhXfBYZr9EWcHlR0kXdQN4y/YbXeHLZP0nyn1MrJ0crOsbTN5XseWd1z8pFfc7jBO RA== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxfm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:18 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327Xcd010551; Thu, 3 Dec 2020 02:08:17 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma01dal.us.ibm.com with ESMTP id 355rf7pxah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:17 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B3288JO36438716 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:09 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 83C2178064; Thu, 3 Dec 2020 02:08:15 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC4C978060; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:14 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 09/18] ibmvfc: implement channel enquiry and setup commands Date: Wed, 2 Dec 2020 20:07:57 -0600 Message-Id: <20201203020806.14747-10-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=4 mlxlogscore=999 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org New NPIV_ENQUIRY_CHANNEL and NPIV_SETUP_CHANNEL management datagrams (MADs) were defined in a previous patchset. If the client advertises a desire to use channels and the partner VIOS is channel capable then the client must proceed with channel enquiry to determine the maximum number of channels the VIOS is capable of providing, and registering SubCRQs via channel setup with the VIOS immediately following NPIV Login. This handshaking should not be performed for subsequent NPIV Logins unless the CRQ connection has been reset. Implement these two new MADs and issue them following a successful NPIV login where the VIOS has set the SUPPORT_CHANNELS capability bit in the NPIV Login response. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 135 ++++++++++++++++++++++++++++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 3 + 2 files changed, 136 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 6b299df786fc..d584a4ce0682 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -806,6 +806,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) spin_lock_irqsave(vhost->host->host_lock, flags); vhost->state = IBMVFC_NO_CRQ; vhost->logged_in = 0; + vhost->do_enquiry = 1; + vhost->using_channels = 0; /* Clean out the queue */ memset(crq->msgs, 0, PAGE_SIZE); @@ -4476,6 +4478,118 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); } +static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + ibmvfc_free_event(evt); + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + vhost->do_enquiry = 0; + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Setup failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + return; + default: + dev_err(vhost->dev, "Invalid Channel Setup response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + return; + } + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); +} + +static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_setup_mad *mad; + struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; + struct ibmvfc_event *evt = ibmvfc_get_event(vhost); + + memset(setup_buf, 0, sizeof(*setup_buf)); + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + + ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_setup; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_SETUP); + mad->common.length = cpu_to_be16(sizeof(*mad)); + mad->buffer.va = cpu_to_be64(vhost->channel_setup_dma); + mad->buffer.len = cpu_to_be32(sizeof(*vhost->channel_setup_buf)); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Sent channel setup\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DOWN); +} + +static void ibmvfc_channel_enquiry_done(struct ibmvfc_event *evt) +{ + struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_enquiry *rsp = &evt->xfer_iu->channel_enquiry; + u32 mad_status = be16_to_cpu(rsp->common.status); + int level = IBMVFC_DEFAULT_LOG_LEVEL; + + switch (mad_status) { + case IBMVFC_MAD_SUCCESS: + ibmvfc_dbg(vhost, "Channel Enquiry succeeded\n"); + vhost->max_vios_scsi_channels = be32_to_cpu(rsp->num_scsi_subq_channels); + ibmvfc_free_event(evt); + break; + case IBMVFC_MAD_FAILED: + level += ibmvfc_retry_host_init(vhost); + ibmvfc_log(vhost, level, "Channel Enquiry failed\n"); + fallthrough; + case IBMVFC_MAD_DRIVER_FAILED: + ibmvfc_free_event(evt); + return; + default: + dev_err(vhost->dev, "Invalid Channel Enquiry response: 0x%x\n", + mad_status); + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); + ibmvfc_free_event(evt); + return; + } + + ibmvfc_channel_setup(vhost); +} + +static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) +{ + struct ibmvfc_channel_enquiry *mad; + struct ibmvfc_event *evt = ibmvfc_get_event(vhost); + + ibmvfc_init_event(evt, ibmvfc_channel_enquiry_done, IBMVFC_MAD_FORMAT); + mad = &evt->iu.channel_enquiry; + memset(mad, 0, sizeof(*mad)); + mad->common.version = cpu_to_be32(1); + mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); + mad->common.length = cpu_to_be16(sizeof(*mad)); + + if (IBMVFC_MIG_NO_SUB_TO_CRQ) + mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); + if (IBMVFC_MIG_NO_N_TO_M) + mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); + + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); + + if (!ibmvfc_send_event(evt, vhost, default_timeout)) + ibmvfc_dbg(vhost, "Send channel enquiry\n"); + else + ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD); +} + /** * ibmvfc_npiv_login_done - Completion handler for NPIV Login * @evt: ibmvfc event struct @@ -4557,8 +4671,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) vhost->host->can_queue = be32_to_cpu(rsp->max_cmds) - IBMVFC_NUM_INTERNAL_REQ; vhost->host->max_sectors = npiv_max_sectors; - ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); - wake_up(&vhost->work_wait_q); + + if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS) && vhost->do_enquiry) { + ibmvfc_channel_enquiry(vhost); + } else { + vhost->do_enquiry = 0; + ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY); + wake_up(&vhost->work_wait_q); + } } /** @@ -5331,9 +5451,20 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost) goto free_trace; } + vhost->channel_setup_buf = dma_alloc_coherent(dev, sizeof(*vhost->channel_setup_buf), + &vhost->channel_setup_dma, + GFP_KERNEL); + + if (!vhost->channel_setup_buf) { + dev_err(dev, "Couldn't allocate Channel Setup buffer\n"); + goto free_tgt_pool; + } + LEAVE; return 0; +free_tgt_pool: + mempool_destroy(vhost->tgt_pool); free_trace: kfree(vhost->trace); free_disc_buffer: diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 986ce4530382..dff26dbd912c 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -842,10 +842,13 @@ struct ibmvfc_host { struct ibmvfc_npiv_login login_info; union ibmvfc_npiv_login_data *login_buf; dma_addr_t login_buf_dma; + struct ibmvfc_channel_setup *channel_setup_buf; + dma_addr_t channel_setup_dma; int disc_buf_sz; int log_level; struct ibmvfc_discover_targets_entry *disc_buf; struct mutex passthru_mutex; + int max_vios_scsi_channels; int task_set; int init_retries; int discovery_threads; From patchwork Thu Dec 3 02:07:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337412 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D50CC83019 for ; Thu, 3 Dec 2020 02:09:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E417721D91 for ; Thu, 3 Dec 2020 02:09:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387460AbgLCCJU (ORCPT ); Wed, 2 Dec 2020 21:09:20 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:37836 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729411AbgLCCJE (ORCPT ); Wed, 2 Dec 2020 21:09:04 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B3252dY128550; Wed, 2 Dec 2020 21:08:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Ox50fkjae/OHqq1pLBa+IgDlo9WflP2edAwjFagubcg=; b=oqgpdHZayC3BrmEO49VF3NFfnKNWnaPcFjZtWooWz7j4Mnml0Xcwml+bkSaSuKsNspGT rc25Qjf4rXHaMPqwbSb7MExj1KTL9bepalksq8Vqmc/V4V1uPtJXQqqGbcZm2uVGX+EW P0yb9/ydAYoLTp0CkYFCQB3b1I32zHnmYqyXHQM6sCgrPlXB0CJHxr9ImOoJQgBmQog3 XFOHlXaTkojExPwfQBHmYKxCIWX+TlE4IpGvEpFqLnUFKVJDQoeJBZ7+5KPo6OI2Tj25 w1sm0I37h3mZAWwmhw3S7gyFwb4S0lAt/BCeN9ZEpY2JsF8aUL6P48c8uQXxJWKNTSij 7Q== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356p2kh3jp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:18 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327Xcf010551; Thu, 3 Dec 2020 02:08:18 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01dal.us.ibm.com with ESMTP id 355rf7pxam-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:17 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Ga516581324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:16 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3517678060; Thu, 3 Dec 2020 02:08:16 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9DB967805F; Thu, 3 Dec 2020 02:08:15 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:15 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 10/18] ibmvfc: advertise client support for using hardware channels Date: Wed, 2 Dec 2020 20:07:58 -0600 Message-Id: <20201203020806.14747-11-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=1 lowpriorityscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Previous patches have plumbed the necessary Sub-CRQ interface and channel negotiation MADs to fully channelized hardware queues. Advertise client support via NPIV Login capability IBMVFC_CAN_USE_CHANNELS when the client bits have MQ enabled via vhost->mq_enabled, or when channels were already in use during a subsequent NPIV Login. The later is required because channel support is only renegotiated after a CRQ pair is broken. Simple NPIV Logout/Logins require the client to continue to advertise the channel capability until the CRQ pair between the client is broken. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d584a4ce0682..0eb91ac86d96 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1282,6 +1282,10 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) login_info->max_cmds = cpu_to_be32(max_requests + IBMVFC_NUM_INTERNAL_REQ); login_info->capabilities = cpu_to_be64(IBMVFC_CAN_MIGRATE | IBMVFC_CAN_SEND_VF_WWPN); + + if (vhost->mq_enabled || vhost->using_channels) + login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS); + login_info->async.va = cpu_to_be64(vhost->async_crq.msg_token); login_info->async.len = cpu_to_be32(vhost->async_crq.size * sizeof(*vhost->async_crq.msgs)); strncpy(login_info->partition_name, vhost->partition_name, IBMVFC_MAX_NAME); From patchwork Thu Dec 3 02:07:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 909F8C64E7C for ; Thu, 3 Dec 2020 02:09:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49804221EB for ; Thu, 3 Dec 2020 02:09:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729508AbgLCCJF (ORCPT ); Wed, 2 Dec 2020 21:09:05 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:1494 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729492AbgLCCJE (ORCPT ); Wed, 2 Dec 2020 21:09:04 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B323DPa075533; Wed, 2 Dec 2020 21:08:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=z6egoUa+qQtR2sJZG4f7jZZtFPKGyk8whFHKiL+TM4A=; b=oI/zXjAnq4UVGk1keO5jQrDRJWRD1Xj102FY2HvmKS0u/GLdnOlv5cuNhYnd0fUfACte 5XQfcgkJ8vwiBv6MNVH07fxc51HjS+LHz4M4M+qDL1n1JlUafh7FiO0UlxoAB1EHTL// PIiGvD71HLEux1UIFRAuz15nJQGRHlsfraL6q/y6mUqC/tGCeRfnoEhjxZt0h6q+XROu aW6DuMCYg8wx0vRvVcrgw6M1bdCJGtFVM2kZlD+21HYGZ96FTFqJdLVyt6e3GPwy4Rbr d0OS10NfT2sPbqzBCLMfEZI4tAoQGE3AeMZXfYc38gRPGuzDkIpK/UOwCXXb1zn+D5Tn pA== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jdgexa6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:19 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327VCg010528; Thu, 3 Dec 2020 02:08:18 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01dal.us.ibm.com with ESMTP id 355rf7pxaq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:18 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328G2g11338322 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:17 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB90C78066; Thu, 3 Dec 2020 02:08:16 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4F83C78064; Thu, 3 Dec 2020 02:08:16 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:16 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 11/18] ibmvfc: set and track hw queue in ibmvfc_event struct Date: Wed, 2 Dec 2020 20:07:59 -0600 Message-Id: <20201203020806.14747-12-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 phishscore=0 clxscore=1015 suspectscore=1 mlxscore=0 priorityscore=1501 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Extract the hwq id from a SCSI command and store it in the ibmvfc_event structure to identify which Sub-CRQ to send the command down when channels are being utilized. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 5 +++++ drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index 0eb91ac86d96..b51ae17883b7 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -1397,6 +1397,7 @@ static void ibmvfc_init_event(struct ibmvfc_event *evt, evt->crq.format = format; evt->done = done; evt->eh_comp = NULL; + evt->hwq = 0; } /** @@ -1748,6 +1749,8 @@ static int ibmvfc_queuecommand_lck(struct scsi_cmnd *cmnd, struct ibmvfc_cmd *vfc_cmd; struct ibmvfc_fcp_cmd_iu *iu; struct ibmvfc_event *evt; + u32 tag_and_hwq = blk_mq_unique_tag(cmnd->request); + u16 hwq = blk_mq_unique_tag_to_hwq(tag_and_hwq); int rc; if (unlikely((rc = fc_remote_port_chkready(rport))) || @@ -1775,6 +1778,8 @@ static int ibmvfc_queuecommand_lck(struct scsi_cmnd *cmnd, } vfc_cmd->correlation = cpu_to_be64(evt); + if (vhost->using_channels) + evt->hwq = hwq % vhost->scsi_scrqs.active_queues; if (likely(!(rc = ibmvfc_map_sg_data(cmnd, evt, vfc_cmd, vhost->dev)))) return ibmvfc_send_event(evt, vhost, 0); diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index dff26dbd912c..e0ffb0416223 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -781,6 +781,7 @@ struct ibmvfc_event { struct completion comp; struct completion *eh_comp; struct timer_list timer; + u16 hwq; }; /* a pool of event structs for use */ From patchwork Thu Dec 3 02:08:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7CBFC8300F for ; Thu, 3 Dec 2020 02:09:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7582621D91 for ; Thu, 3 Dec 2020 02:09:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729533AbgLCCJH (ORCPT ); Wed, 2 Dec 2020 21:09:07 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:43622 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729516AbgLCCJH (ORCPT ); Wed, 2 Dec 2020 21:09:07 -0500 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B321ids044296; Wed, 2 Dec 2020 21:08:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=hJZ276vi5JSiqzv/GyFJ3+qhFdPyz+aRaWI7hvpz5zY=; b=WEBsB40BVZCu5rtrpbrl3jredLdbu/YEhckpvAZKRjGnM4qmINrhr24nOHnTkjZRKjnJ DJZjCXMrX+hQ9ZZ8fw/ohXf+tUSTjl5Xrp9CJxmcG7wI2ZG82FTsdKnhYNJSG7OQTaY8 Z5rP0RShPRtPmO2CHRqR/b3akSnFYaSbMg7tOLDvb+knOak0LYgRso99P6u8o7/Lc3Vt l+rmXLXbLEg34/OVlXARtEVuya6nI9mRTkZqKJd2U3u4GUoN/5O3jeJ/iJ4YkeKCLrBh ksWdfHvnHvOlbWHPBlN2IySfYBTDmnDdkhgsHVJVlawUzwC4KWC/3pDOM6SzXj3454W7 rg== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0b-001b2d01.pphosted.com with ESMTP id 356jdx73pp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:19 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uTs2006224; Thu, 3 Dec 2020 02:08:19 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma03dal.us.ibm.com with ESMTP id 353e69pd99-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:19 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Hwp22610222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:17 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8DC817805C; Thu, 3 Dec 2020 02:08:17 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 01F747806A; Thu, 3 Dec 2020 02:08:17 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:16 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 12/18] ibmvfc: send commands down HW Sub-CRQ when channelized Date: Wed, 2 Dec 2020 20:08:00 -0600 Message-Id: <20201203020806.14747-13-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 phishscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=1 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030006 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When the client has negotiated the use of channels all vfcFrames are required to go down a Sub-CRQ channel or it is a protocoal violation. If the adapter state is channelized submit vfcFrames to the appropriate Sub-CRQ via the h_send_sub_crq() helper. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index b51ae17883b7..d94db73d4a15 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -701,6 +701,15 @@ static int ibmvfc_send_crq(struct ibmvfc_host *vhost, u64 word1, u64 word2) return plpar_hcall_norets(H_SEND_CRQ, vdev->unit_address, word1, word2); } +static int ibmvfc_send_sub_crq(struct ibmvfc_host *vhost, u64 cookie, u64 word1, + u64 word2, u64 word3, u64 word4) +{ + struct vio_dev *vdev = to_vio_dev(vhost->dev); + + return plpar_hcall_norets(H_SEND_SUB_CRQ, vdev->unit_address, cookie, + word1, word2, word3, word4); +} + /** * ibmvfc_send_crq_init - Send a CRQ init message * @vhost: ibmvfc host struct @@ -1513,15 +1522,19 @@ static int ibmvfc_send_event(struct ibmvfc_event *evt, struct ibmvfc_host *vhost, unsigned long timeout) { __be64 *crq_as_u64 = (__be64 *) &evt->crq; + int channel_cmd = 0; int rc; /* Copy the IU into the transfer area */ *evt->xfer_iu = evt->iu; - if (evt->crq.format == IBMVFC_CMD_FORMAT) + if (evt->crq.format == IBMVFC_CMD_FORMAT) { evt->xfer_iu->cmd.tag = cpu_to_be64((u64)evt); - else if (evt->crq.format == IBMVFC_MAD_FORMAT) + channel_cmd = 1; + } else if (evt->crq.format == IBMVFC_MAD_FORMAT) { evt->xfer_iu->mad_common.tag = cpu_to_be64((u64)evt); - else + if (evt->xfer_iu->mad_common.opcode == IBMVFC_TMF_MAD) + channel_cmd = 1; + } else BUG(); list_add_tail(&evt->queue, &vhost->sent); @@ -1534,8 +1547,17 @@ static int ibmvfc_send_event(struct ibmvfc_event *evt, mb(); - if ((rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), - be64_to_cpu(crq_as_u64[1])))) { + if (vhost->using_channels && channel_cmd) + rc = ibmvfc_send_sub_crq(vhost, + vhost->scsi_scrqs.scrqs[evt->hwq].vios_cookie, + be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1]), + 0, 0); + else + rc = ibmvfc_send_crq(vhost, be64_to_cpu(crq_as_u64[0]), + be64_to_cpu(crq_as_u64[1])); + + if (rc) { list_del(&evt->queue); del_timer(&evt->timer); From patchwork Thu Dec 3 02:08:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A416C83021 for ; Thu, 3 Dec 2020 02:09:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5479221FB for ; Thu, 3 Dec 2020 02:09:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729572AbgLCCJR (ORCPT ); Wed, 2 Dec 2020 21:09:17 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:37176 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729496AbgLCCJF (ORCPT ); Wed, 2 Dec 2020 21:09:05 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B323DqE075470; Wed, 2 Dec 2020 21:08:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=An21lTSeedpbOJBRRtdb3N0VXEp9CNxvAycyR7i/Nyo=; b=HbM84/12UsCuQXu4lQcLokOSTfVjzOWSJq5XRuA0HolOAQl6XrMOBB+bXPZp/zZec1Km XvpSFraxasIxGCI8Ez84VM58V2/GbCPJxorsYIy5cCKnNx0HGMiXzqKO7YBOsTiM72Lk eb743Xu90kZ0lmyq8IG5KyP9sXtZoKMYmdcRjFVT11sYSyQPuVhQZ+/Tdt8Xal/KAnkG zHBXHBqe2bHR1y+QwZ9U32Jb2LF8OqDgUSJ0Inb2sNiAfeB6tHTVaNXih7xqLXeF5VfX 2/8tteh1tG87hAAGRsdUr/Roz8VFMPAScQjciG0Uu8PYGtl6ob+XEZ7eWbyyS9dpPl9d gg== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jdgexb5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:21 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327XAx010548; Thu, 3 Dec 2020 02:08:20 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma01dal.us.ibm.com with ESMTP id 355rf7pxb0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:19 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Itq24510844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:18 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 405F47805E; Thu, 3 Dec 2020 02:08:18 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A908178063; Thu, 3 Dec 2020 02:08:17 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:17 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 13/18] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Date: Wed, 2 Dec 2020 20:08:01 -0600 Message-Id: <20201203020806.14747-14-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 phishscore=0 clxscore=1015 suspectscore=3 mlxscore=0 priorityscore=1501 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org If the ibmvfc client adapter requests channels it must submit a number of Sub-CRQ handles matching the number of channels being requested. The VIOS in its response will overwrite the actual number of channel resources allocated which may be less than what was requested. The client then must store the VIOS Sub-CRQ handle for each queue. This VIOS handle is needed as a parameter with h_send_sub_crq(). Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d94db73d4a15..d65de320252e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -4512,15 +4512,35 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost) static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) { struct ibmvfc_host *vhost = evt->vhost; + struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); int level = IBMVFC_DEFAULT_LOG_LEVEL; + int flags, active_queues, i; ibmvfc_free_event(evt); switch (mad_status) { case IBMVFC_MAD_SUCCESS: ibmvfc_dbg(vhost, "Channel Setup succeded\n"); + flags = be32_to_cpu(setup->flags); vhost->do_enquiry = 0; + active_queues = be32_to_cpu(setup->num_scsi_subq_channels); + scrqs->active_queues = active_queues; + + if (flags & IBMVFC_CHANNELS_CANCELED) { + ibmvfc_dbg(vhost, "Channels Canceled\n"); + vhost->using_channels = 0; + } else { + if (active_queues) + vhost->using_channels = 1; + for (i = 0; i < active_queues; i++) + scrqs->scrqs[i].vios_cookie = + be64_to_cpu(setup->channel_handles[i]); + + ibmvfc_dbg(vhost, "Using %u channels\n", + vhost->scsi_scrqs.active_queues); + } break; case IBMVFC_MAD_FAILED: level += ibmvfc_retry_host_init(vhost); @@ -4544,9 +4564,19 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) struct ibmvfc_channel_setup_mad *mad; struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf; struct ibmvfc_event *evt = ibmvfc_get_event(vhost); + struct ibmvfc_scsi_channels *scrqs = &vhost->scsi_scrqs; + unsigned int num_channels = + min(vhost->client_scsi_channels, vhost->max_vios_scsi_channels); + int i; memset(setup_buf, 0, sizeof(*setup_buf)); - setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + if (num_channels == 0) + setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS); + else { + setup_buf->num_scsi_subq_channels = cpu_to_be32(num_channels); + for (i = 0; i < num_channels; i++) + setup_buf->channel_handles[i] = cpu_to_be64(scrqs->scrqs[i].cookie); + } ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); mad = &evt->iu.channel_setup; From patchwork Thu Dec 3 02:08:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5B6AC71156 for ; Thu, 3 Dec 2020 02:09:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 77C35221FB for ; Thu, 3 Dec 2020 02:09:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729492AbgLCCJG (ORCPT ); Wed, 2 Dec 2020 21:09:06 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:12058 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729511AbgLCCJF (ORCPT ); Wed, 2 Dec 2020 21:09:05 -0500 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B326XRT065425; Wed, 2 Dec 2020 21:08:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=6PX7ikJAOSRj9lPSw5dbkhWeYbfqM4LI5jXxaMrblW8=; b=OgYJIKRvY8Kzwrlx9DeysQ263bPNkdDciqRComXAlYlQ3a9Yys688KDPod712xt/lLk2 +j8zZV8zlxfbr2ZglgpblYoj9//3gC85NWvToYif4b+Wyt6TP8s5IoC19aFPkGXnSZ8+ Ofah0EeyzmDh5iCMEodsnPU/IXKoe+XwVA5Of80IIZsb2+5WgFYtzP222YEr8Xa6bnjo c6D27rSEqTZs7TuqB/B8uZWkM9ueaeUqwpuPz2wXgPH05tbD+oklZSB2J2NiExA0Dwfy Y9EZ/5UVoS8IBsFon+uf9/7th70U3bGEkV3odLuqazYiXJcG46lHiSSGtuDxXglGRweg Kw== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jg86ukr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:21 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31xjDt013051; Thu, 3 Dec 2020 02:08:20 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma02wdc.us.ibm.com with ESMTP id 356cbem9qw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:20 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328JLC22675906 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:19 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F3D527805F; Thu, 3 Dec 2020 02:08:18 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6734B7805E; Thu, 3 Dec 2020 02:08:18 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:18 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 14/18] ibmvfc: add cancel mad initialization helper Date: Wed, 2 Dec 2020 20:08:02 -0600 Message-Id: <20201203020806.14747-15-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 lowpriorityscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 impostorscore=0 phishscore=0 suspectscore=1 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add a helper routine for initializing a Cancel MAD. This will be useful for a channelized client that needs to send a Cancel commands down every channel commands were sent for a particular LUN. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 67 ++++++++++++++++++++-------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index d65de320252e..ec3db5a6baf3 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2286,6 +2286,44 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, return SUCCESS; } +static struct ibmvfc_event *ibmvfc_init_tmf(struct ibmvfc_host *vhost, + struct scsi_device *sdev, + int type) +{ + struct scsi_target *starget = scsi_target(sdev); + struct fc_rport *rport = starget_to_rport(starget); + struct ibmvfc_event *evt; + struct ibmvfc_tmf *tmf; + + evt = ibmvfc_get_event(vhost); + ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); + + tmf = &evt->iu.tmf; + memset(tmf, 0, sizeof(*tmf)); + if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { + tmf->common.version = cpu_to_be32(2); + tmf->target_wwpn = cpu_to_be64(rport->port_name); + } else { + tmf->common.version = cpu_to_be32(1); + } + tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); + tmf->common.length = cpu_to_be16(sizeof(*tmf)); + tmf->scsi_id = cpu_to_be64(rport->port_id); + int_to_scsilun(sdev->lun, &tmf->lun); + if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) + type &= ~IBMVFC_TMF_SUPPRESS_ABTS; + if (vhost->state == IBMVFC_ACTIVE) + tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); + else + tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); + tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); + tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); + + init_completion(&evt->comp); + + return evt; +} + /** * ibmvfc_cancel_all - Cancel all outstanding commands to the device * @sdev: scsi device to cancel commands @@ -2300,9 +2338,6 @@ static int ibmvfc_wait_for_ops(struct ibmvfc_host *vhost, void *device, static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); - struct scsi_target *starget = scsi_target(sdev); - struct fc_rport *rport = starget_to_rport(starget); - struct ibmvfc_tmf *tmf; struct ibmvfc_event *evt, *found_evt; union ibmvfc_iu rsp; int rsp_rc = -EBUSY; @@ -2327,32 +2362,8 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) } if (vhost->logged_in) { - evt = ibmvfc_get_event(vhost); - ibmvfc_init_event(evt, ibmvfc_sync_completion, IBMVFC_MAD_FORMAT); - - tmf = &evt->iu.tmf; - memset(tmf, 0, sizeof(*tmf)); - if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) { - tmf->common.version = cpu_to_be32(2); - tmf->target_wwpn = cpu_to_be64(rport->port_name); - } else { - tmf->common.version = cpu_to_be32(1); - } - tmf->common.opcode = cpu_to_be32(IBMVFC_TMF_MAD); - tmf->common.length = cpu_to_be16(sizeof(*tmf)); - tmf->scsi_id = cpu_to_be64(rport->port_id); - int_to_scsilun(sdev->lun, &tmf->lun); - if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS)) - type &= ~IBMVFC_TMF_SUPPRESS_ABTS; - if (vhost->state == IBMVFC_ACTIVE) - tmf->flags = cpu_to_be32((type | IBMVFC_TMF_LUA_VALID)); - else - tmf->flags = cpu_to_be32(((type & IBMVFC_TMF_SUPPRESS_ABTS) | IBMVFC_TMF_LUA_VALID)); - tmf->cancel_key = cpu_to_be32((unsigned long)sdev->hostdata); - tmf->my_cancel_key = cpu_to_be32((unsigned long)starget->hostdata); - + evt = ibmvfc_init_tmf(vhost, sdev, type); evt->sync_iu = &rsp; - init_completion(&evt->comp); rsp_rc = ibmvfc_send_event(evt, vhost, default_timeout); } From patchwork Thu Dec 3 02:08:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92342C83016 for ; Thu, 3 Dec 2020 02:09:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48695221EB for ; Thu, 3 Dec 2020 02:09:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729541AbgLCCJH (ORCPT ); Wed, 2 Dec 2020 21:09:07 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:11590 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729518AbgLCCJG (ORCPT ); Wed, 2 Dec 2020 21:09:06 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322lbr095985; Wed, 2 Dec 2020 21:08:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=oKqIBsDi6ATCu+2VKWLOJBnqTvd96VekI27xyK9Lick=; b=fUuY/exmKzBjvFusj96QMozZRqgrDLswHUixlFlKkQ/Fi+ENYn/jpjRD7L24t719WySs O9P2sn9Jvg75rf/TmTt9tJ9VUhLZtPRNED1xR4+k9m9FAVTfAA9eYwZzAhHatbzfEYi6 CKRippjkPc/im7V/b2ADgdcyRsRuNYRu5puNYSHNM3lopZZ9iGwRrEJJUjrzYlnFLNOY rALvTBoWyhjECTmMng8KX9NbQ08D5rWGcc8esL27vblAhF1HRAUYhjpw9QEhg6xRa+/Y vfQLNA0CZ8DeQsNxe724b5PQ1k3y/SBUfYeFlBQcEWKbQ35DnWvIS99V0JDDdjGuGPrh +Q== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jfcxxhs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:22 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uL3W013325; Thu, 3 Dec 2020 02:08:21 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfvc8k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:21 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328JVt21102944 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:19 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A2B117805F; Thu, 3 Dec 2020 02:08:19 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1A4457805E; Thu, 3 Dec 2020 02:08:19 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:19 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 15/18] ibmvfc: send Cancel MAD down each hw scsi channel Date: Wed, 2 Dec 2020 20:08:03 -0600 Message-Id: <20201203020806.14747-16-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 suspectscore=3 mlxlogscore=999 spamscore=0 malwarescore=0 mlxscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In general the client needs to send Cancel MADs and task management commands down the same channel as the command(s) intended to cancel or abort. The client assigns cancel keys per LUN and thus must send a Cancel down each channel commands were submitted for that LUN. Further, the client then must wait for those cancel completions prior to submitting a LUN RESET or ABORT TASK SET. Add a cancel event pointer and cancel rsp iu storage to the ibmvfc_sub_queue struct such that the cancel routine can assign a cancel event to each applicable queue. When in legacy CRQ mode we fake treating it as a subqueue by using a subqueue struct allocated on the stack. Wait for completion of each submitted cancel. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 104 ++++++++++++++++++++++----------- drivers/scsi/ibmvscsi/ibmvfc.h | 38 ++++++------ 2 files changed, 90 insertions(+), 52 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index ec3db5a6baf3..e353b9e88104 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2339,67 +2339,103 @@ static int ibmvfc_cancel_all(struct scsi_device *sdev, int type) { struct ibmvfc_host *vhost = shost_priv(sdev->host); struct ibmvfc_event *evt, *found_evt; - union ibmvfc_iu rsp; - int rsp_rc = -EBUSY; + struct ibmvfc_sub_queue *scrqs; + struct ibmvfc_sub_queue legacy_crq; + int rsp_rc = 0; unsigned long flags; u16 status; + int cancel_cnt = 0; + int num_hwq; + int ret = 0; + int i; ENTER; spin_lock_irqsave(vhost->host->host_lock, flags); - found_evt = NULL; - list_for_each_entry(evt, &vhost->sent, queue) { - if (evt->cmnd && evt->cmnd->device == sdev) { - found_evt = evt; + if (vhost->using_channels && vhost->scsi_scrqs.active_queues) { + num_hwq = vhost->scsi_scrqs.active_queues; + scrqs = vhost->scsi_scrqs.scrqs; + } else { + /* Use ibmvfc_sub_queue on the stack to fake legacy CRQ as a subqueue */ + num_hwq = 1; + scrqs = &legacy_crq; + } + + for (i = 0; i < num_hwq; i++) { + scrqs[i].cancel_event = NULL; + found_evt = NULL; + list_for_each_entry(evt, &vhost->sent, queue) { + if (evt->cmnd && evt->cmnd->device == sdev && evt->hwq == i) { + found_evt = evt; + cancel_cnt++; + break; + } + } + + if (!found_evt) + continue; + + if (vhost->logged_in) { + scrqs[i].cancel_event = ibmvfc_init_tmf(vhost, sdev, type); + scrqs[i].cancel_event->hwq = i; + scrqs[i].cancel_event->sync_iu = &scrqs[i].cancel_rsp; + rsp_rc = ibmvfc_send_event(scrqs[i].cancel_event, vhost, default_timeout); + if (rsp_rc) + break; + } else { + rsp_rc = -EBUSY; break; } } - if (!found_evt) { + spin_unlock_irqrestore(vhost->host->host_lock, flags); + + if (!cancel_cnt) { if (vhost->log_level > IBMVFC_DEFAULT_LOG_LEVEL) sdev_printk(KERN_INFO, sdev, "No events found to cancel\n"); - spin_unlock_irqrestore(vhost->host->host_lock, flags); return 0; } - if (vhost->logged_in) { - evt = ibmvfc_init_tmf(vhost, sdev, type); - evt->sync_iu = &rsp; - rsp_rc = ibmvfc_send_event(evt, vhost, default_timeout); - } - - spin_unlock_irqrestore(vhost->host->host_lock, flags); - if (rsp_rc != 0) { sdev_printk(KERN_ERR, sdev, "Failed to send cancel event. rc=%d\n", rsp_rc); /* If failure is received, the host adapter is most likely going through reset, return success so the caller will wait for the command being cancelled to get returned */ - return 0; + goto free_events; } sdev_printk(KERN_INFO, sdev, "Cancelling outstanding commands.\n"); - wait_for_completion(&evt->comp); - status = be16_to_cpu(rsp.mad_common.status); - spin_lock_irqsave(vhost->host->host_lock, flags); - ibmvfc_free_event(evt); - spin_unlock_irqrestore(vhost->host->host_lock, flags); + for (i = 0; i < num_hwq; i++) { + if (!scrqs[i].cancel_event) + continue; - if (status != IBMVFC_MAD_SUCCESS) { - sdev_printk(KERN_WARNING, sdev, "Cancel failed with rc=%x\n", status); - switch (status) { - case IBMVFC_MAD_DRIVER_FAILED: - case IBMVFC_MAD_CRQ_ERROR: - /* Host adapter most likely going through reset, return success to - the caller will wait for the command being cancelled to get returned */ - return 0; - default: - return -EIO; - }; + wait_for_completion(&scrqs[i].cancel_event->comp); + status = be16_to_cpu(scrqs[i].cancel_rsp.mad_common.status); + + if (status != IBMVFC_MAD_SUCCESS) { + sdev_printk(KERN_WARNING, sdev, "Cancel failed with rc=%x\n", status); + switch (status) { + case IBMVFC_MAD_DRIVER_FAILED: + case IBMVFC_MAD_CRQ_ERROR: + /* Host adapter most likely going through reset, return success to + the caller will wait for the command being cancelled to get returned */ + goto free_events; + default: + ret = -EIO; + goto free_events; + }; + } } sdev_printk(KERN_INFO, sdev, "Successfully cancelled outstanding commands\n"); - return 0; +free_events: + spin_lock_irqsave(vhost->host->host_lock, flags); + for (i = 0; i < num_hwq; i++) + if (scrqs[i].cancel_event) + ibmvfc_free_event(scrqs[i].cancel_event); + spin_unlock_irqrestore(vhost->host->host_lock, flags); + + return ret; } /** diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index e0ffb0416223..980eb9afe93a 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -584,6 +584,24 @@ struct ibmvfc_connection_info { __be64 reserved[16]; } __packed __aligned(8); +union ibmvfc_iu { + struct ibmvfc_mad_common mad_common; + struct ibmvfc_npiv_login_mad npiv_login; + struct ibmvfc_npiv_logout_mad npiv_logout; + struct ibmvfc_discover_targets discover_targets; + struct ibmvfc_port_login plogi; + struct ibmvfc_process_login prli; + struct ibmvfc_move_login move_login; + struct ibmvfc_query_tgt query_tgt; + struct ibmvfc_implicit_logout implicit_logout; + struct ibmvfc_tmf tmf; + struct ibmvfc_cmd cmd; + struct ibmvfc_passthru_mad passthru; + struct ibmvfc_channel_enquiry channel_enquiry; + struct ibmvfc_channel_setup_mad channel_setup; + struct ibmvfc_connection_info connection_info; +} __packed __aligned(8); + struct ibmvfc_trace_start_entry { u32 xfer_len; } __packed; @@ -666,6 +684,8 @@ struct ibmvfc_sub_queue { dma_addr_t msg_token; int size, cur; struct ibmvfc_host *vhost; + struct ibmvfc_event *cancel_event; + union ibmvfc_iu cancel_rsp; unsigned long cookie; unsigned long vios_cookie; unsigned long hw_irq; @@ -712,24 +732,6 @@ struct ibmvfc_async_crq_queue { dma_addr_t msg_token; }; -union ibmvfc_iu { - struct ibmvfc_mad_common mad_common; - struct ibmvfc_npiv_login_mad npiv_login; - struct ibmvfc_npiv_logout_mad npiv_logout; - struct ibmvfc_discover_targets discover_targets; - struct ibmvfc_port_login plogi; - struct ibmvfc_process_login prli; - struct ibmvfc_move_login move_login; - struct ibmvfc_query_tgt query_tgt; - struct ibmvfc_implicit_logout implicit_logout; - struct ibmvfc_tmf tmf; - struct ibmvfc_cmd cmd; - struct ibmvfc_passthru_mad passthru; - struct ibmvfc_channel_enquiry channel_enquiry; - struct ibmvfc_channel_setup_mad channel_setup; - struct ibmvfc_connection_info connection_info; -} __packed __aligned(8); - enum ibmvfc_target_action { IBMVFC_TGT_ACTION_NONE = 0, IBMVFC_TGT_ACTION_INIT, From patchwork Thu Dec 3 02:08:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 338186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00F38C64E7C for ; Thu, 3 Dec 2020 02:10:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DE2C21D7A for ; Thu, 3 Dec 2020 02:10:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729658AbgLCCKY (ORCPT ); Wed, 2 Dec 2020 21:10:24 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:6580 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727681AbgLCCKY (ORCPT ); Wed, 2 Dec 2020 21:10:24 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322Xxm185149; Wed, 2 Dec 2020 21:09:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=zs9ZClk1wGF1XY64i1xm/zly2L9990L1G1St6BXBZ0E=; b=JQFN0NZ026gPths/85oRQPu/hx4Sc9F6x+GuQ8ELhueYyglrrNuUOIP1TFa6xGDGLo8V jF/AUDOp5NcGT0oY6NFOWEkrDcj8eI4st/5fFuNVK4BAjiNh7aJC9/3hzbFX7F8in3F+ 1qirGfFVhit7ztL9cYux9MFHyHk3jgU3c8geMSfd8cHQquh016tkfYSEpTqiHfHRdfp4 wOmw/3OSfj0JmIaM/SlB8agBPQxkCJEeM4SHAHf6VSj3OuyC8tPl6Yg5WzrcVNPoCIZA V2XOpbeC8gH/1zGd5hl0gtsNbIIFE4KnqnILXzO/Nvr9dhomeU3A+U0VL4YDQX6wJlU9 Vg== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 356jdgyb8a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:09:37 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uMgQ013334; Thu, 3 Dec 2020 02:09:36 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfvcjp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:09:36 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Aev2425408 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:10 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5DF6F7805F; Thu, 3 Dec 2020 02:08:20 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BD07878060; Thu, 3 Dec 2020 02:08:19 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:19 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler , Brian King Subject: [PATCH v3 16/18] ibmvfc: enable MQ and set reasonable defaults Date: Wed, 2 Dec 2020 20:08:04 -0600 Message-Id: <20201203020806.14747-17-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 priorityscore=1501 clxscore=1015 mlxlogscore=855 impostorscore=0 suspectscore=1 spamscore=0 mlxscore=0 bulkscore=0 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030006 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Turn on MQ by default and set sane values for the upper limit on hw queues for the scsi host, and number of hw scsi channels to request from the partner VIOS. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h index 980eb9afe93a..93c234a5512d 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.h +++ b/drivers/scsi/ibmvscsi/ibmvfc.h @@ -41,9 +41,9 @@ #define IBMVFC_DEFAULT_LOG_LEVEL 2 #define IBMVFC_MAX_CDB_LEN 16 #define IBMVFC_CLS3_ERROR 0 -#define IBMVFC_MQ 0 -#define IBMVFC_SCSI_CHANNELS 0 -#define IBMVFC_SCSI_HW_QUEUES 1 +#define IBMVFC_MQ 1 +#define IBMVFC_SCSI_CHANNELS 8 +#define IBMVFC_SCSI_HW_QUEUES 16 #define IBMVFC_MIG_NO_SUB_TO_CRQ 0 #define IBMVFC_MIG_NO_N_TO_M 0 From patchwork Thu Dec 3 02:08:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337414 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41C30C8301A for ; Thu, 3 Dec 2020 02:09:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C0C4221EB for ; Thu, 3 Dec 2020 02:09:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729562AbgLCCJK (ORCPT ); Wed, 2 Dec 2020 21:09:10 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:22406 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729548AbgLCCJI (ORCPT ); Wed, 2 Dec 2020 21:09:08 -0500 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B3250jC128371; Wed, 2 Dec 2020 21:08:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=DPj0YSggplOvuiVrgSY90VVLutSqNS/gveNO1fE1x98=; b=kkMLOCFlfmRoZSr/XiD8jrSYWGvp4iZTimyP4pRnl5fJkOi8+4gLNDB4qgYMkT9e7Zcr WWgPVoOzOFB3yHNJZL9rj2ko4Uk25Nf+0P6hMWREVYDy7baknJ0jrTzMsCX0ujWso/1r nedsRBY/ItEOUWwgqgetvznjKV7im8IZjy2sI3C1k3q7YIDHV/r/F909a+A/EzakK+Sq 0RSvYa9/+yQqZAZqOGDFzsEB828dyRpyJ5Mb6tKcdRZJslmHEwd+DFlKsolyq/tNOOoh w4Y55SWR9pUPgWQkzgNjrUvS4nKW0yCgO6Y+Ho0XqsroDTmul+gzHgsza9/uVl3SDcTa Cw== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 356p2kh3nh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:08:23 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B327Wmk010529; Thu, 3 Dec 2020 02:08:22 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma01dal.us.ibm.com with ESMTP id 355rf7pxbg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:08:22 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328Eet22807166 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:14 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 031207805F; Thu, 3 Dec 2020 02:08:21 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7910D78068; Thu, 3 Dec 2020 02:08:20 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:20 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 17/18] ibmvfc: provide modules parameters for MQ settings Date: Wed, 2 Dec 2020 20:08:05 -0600 Message-Id: <20201203020806.14747-18-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=1 lowpriorityscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add the various module parameter toggles for adjusting the MQ characteristics at boot/load time as well as a device attribute for changing the client scsi channel request amount. Signed-off-by: Tyrel Datwyler Reviewed-by: Brian King --- drivers/scsi/ibmvscsi/ibmvfc.c | 76 +++++++++++++++++++++++++++++----- 1 file changed, 66 insertions(+), 10 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index e353b9e88104..e499599662ec 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -40,6 +40,12 @@ static unsigned int disc_threads = IBMVFC_MAX_DISC_THREADS; static unsigned int ibmvfc_debug = IBMVFC_DEBUG; static unsigned int log_level = IBMVFC_DEFAULT_LOG_LEVEL; static unsigned int cls3_error = IBMVFC_CLS3_ERROR; +static unsigned int mq_enabled = IBMVFC_MQ; +static unsigned int nr_scsi_hw_queues = IBMVFC_SCSI_HW_QUEUES; +static unsigned int nr_scsi_channels = IBMVFC_SCSI_CHANNELS; +static unsigned int mig_channels_only = IBMVFC_MIG_NO_SUB_TO_CRQ; +static unsigned int mig_no_less_channels = IBMVFC_MIG_NO_N_TO_M; + static LIST_HEAD(ibmvfc_head); static DEFINE_SPINLOCK(ibmvfc_driver_lock); static struct scsi_transport_template *ibmvfc_transport_template; @@ -49,6 +55,22 @@ MODULE_AUTHOR("Brian King "); MODULE_LICENSE("GPL"); MODULE_VERSION(IBMVFC_DRIVER_VERSION); +module_param_named(mq, mq_enabled, uint, S_IRUGO); +MODULE_PARM_DESC(mq, "Enable multiqueue support. " + "[Default=" __stringify(IBMVFC_MQ) "]"); +module_param_named(scsi_host_queues, nr_scsi_hw_queues, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_host_queues, "Number of SCSI Host submission queues. " + "[Default=" __stringify(IBMVFC_SCSI_HW_QUEUES) "]"); +module_param_named(scsi_hw_channels, nr_scsi_channels, uint, S_IRUGO); +MODULE_PARM_DESC(scsi_hw_channels, "Number of hw scsi channels to request. " + "[Default=" __stringify(IBMVFC_SCSI_CHANNELS) "]"); +module_param_named(mig_channels_only, mig_channels_only, uint, S_IRUGO); +MODULE_PARM_DESC(mig_channels_only, "Prevent migration to non-channelized system. " + "[Default=" __stringify(IBMVFC_MIG_NO_SUB_TO_CRQ) "]"); +module_param_named(mig_no_less_channels, mig_no_less_channels, uint, S_IRUGO); +MODULE_PARM_DESC(mig_no_less_channels, "Prevent migration to system with less channels. " + "[Default=" __stringify(IBMVFC_MIG_NO_N_TO_M) "]"); + module_param_named(init_timeout, init_timeout, uint, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(init_timeout, "Initialization timeout in seconds. " "[Default=" __stringify(IBMVFC_INIT_TIMEOUT) "]"); @@ -823,7 +845,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) crq->cur = 0; if (vhost->scsi_scrqs.scrqs) { - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { scrq = &vhost->scsi_scrqs.scrqs[i]; memset(scrq->msgs, 0, PAGE_SIZE); scrq->cur = 0; @@ -3234,6 +3256,37 @@ static ssize_t ibmvfc_store_log_level(struct device *dev, return strlen(buf); } +static ssize_t ibmvfc_show_scsi_channels(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + int len; + + spin_lock_irqsave(shost->host_lock, flags); + len = snprintf(buf, PAGE_SIZE, "%d\n", vhost->client_scsi_channels); + spin_unlock_irqrestore(shost->host_lock, flags); + return len; +} + +static ssize_t ibmvfc_store_scsi_channels(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct Scsi_Host *shost = class_to_shost(dev); + struct ibmvfc_host *vhost = shost_priv(shost); + unsigned long flags = 0; + unsigned int channels; + + spin_lock_irqsave(shost->host_lock, flags); + channels = simple_strtoul(buf, NULL, 10); + vhost->client_scsi_channels = min(channels, nr_scsi_hw_queues); + ibmvfc_hard_reset_host(vhost); + spin_unlock_irqrestore(shost->host_lock, flags); + return strlen(buf); +} + static DEVICE_ATTR(partition_name, S_IRUGO, ibmvfc_show_host_partition_name, NULL); static DEVICE_ATTR(device_name, S_IRUGO, ibmvfc_show_host_device_name, NULL); static DEVICE_ATTR(port_loc_code, S_IRUGO, ibmvfc_show_host_loc_code, NULL); @@ -3242,6 +3295,8 @@ static DEVICE_ATTR(npiv_version, S_IRUGO, ibmvfc_show_host_npiv_version, NULL); static DEVICE_ATTR(capabilities, S_IRUGO, ibmvfc_show_host_capabilities, NULL); static DEVICE_ATTR(log_level, S_IRUGO | S_IWUSR, ibmvfc_show_log_level, ibmvfc_store_log_level); +static DEVICE_ATTR(nr_scsi_channels, S_IRUGO | S_IWUSR, + ibmvfc_show_scsi_channels, ibmvfc_store_scsi_channels); #ifdef CONFIG_SCSI_IBMVFC_TRACE /** @@ -3298,6 +3353,7 @@ static struct device_attribute *ibmvfc_attrs[] = { &dev_attr_npiv_version, &dev_attr_capabilities, &dev_attr_log_level, + &dev_attr_nr_scsi_channels, NULL }; @@ -4685,9 +4741,9 @@ static void ibmvfc_channel_enquiry(struct ibmvfc_host *vhost) mad->common.opcode = cpu_to_be32(IBMVFC_CHANNEL_ENQUIRY); mad->common.length = cpu_to_be16(sizeof(*mad)); - if (IBMVFC_MIG_NO_SUB_TO_CRQ) + if (mig_channels_only) mad->flags |= cpu_to_be32(IBMVFC_NO_CHANNELS_TO_CRQ_SUPPORT); - if (IBMVFC_MIG_NO_N_TO_M) + if (mig_no_less_channels) mad->flags |= cpu_to_be32(IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT); ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT); @@ -5426,13 +5482,13 @@ static int ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) ENTER; - vhost->scsi_scrqs.scrqs = kcalloc(IBMVFC_SCSI_HW_QUEUES, + vhost->scsi_scrqs.scrqs = kcalloc(nr_scsi_hw_queues, sizeof(*vhost->scsi_scrqs.scrqs), GFP_KERNEL); if (!vhost->scsi_scrqs.scrqs) return -1; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) { + for (i = 0; i < nr_scsi_hw_queues; i++) { if (ibmvfc_register_scsi_channel(vhost, i)) { for (j = i; j > 0; j--) ibmvfc_deregister_scsi_channel(vhost, j - 1); @@ -5456,7 +5512,7 @@ static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) if (!vhost->scsi_scrqs.scrqs) return; - for (i = 0; i < IBMVFC_SCSI_HW_QUEUES; i++) + for (i = 0; i < nr_scsi_hw_queues; i++) ibmvfc_deregister_scsi_channel(vhost, i); kfree(vhost->scsi_scrqs.scrqs); @@ -5668,13 +5724,13 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) } shost->transportt = ibmvfc_transport_template; - shost->can_queue = (max_requests / IBMVFC_SCSI_HW_QUEUES); + shost->can_queue = (max_requests / nr_scsi_hw_queues); shost->max_lun = max_lun; shost->max_id = max_targets; shost->max_sectors = IBMVFC_MAX_SECTORS; shost->max_cmd_len = IBMVFC_MAX_CDB_LEN; shost->unique_id = shost->host_no; - shost->nr_hw_queues = IBMVFC_SCSI_HW_QUEUES; + shost->nr_hw_queues = nr_scsi_hw_queues; vhost = shost_priv(shost); INIT_LIST_HEAD(&vhost->sent); @@ -5687,8 +5743,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->log_level = log_level; vhost->task_set = 1; - vhost->mq_enabled = IBMVFC_MQ; - vhost->client_scsi_channels = IBMVFC_SCSI_CHANNELS; + vhost->mq_enabled = mq_enabled; + vhost->client_scsi_channels = min(nr_scsi_hw_queues, nr_scsi_channels); vhost->using_channels = 0; vhost->do_enquiry = 1; From patchwork Thu Dec 3 02:08:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tyrel Datwyler X-Patchwork-Id: 337407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E476C71155 for ; Thu, 3 Dec 2020 02:10:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CD0B21D7A for ; Thu, 3 Dec 2020 02:10:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729669AbgLCCK1 (ORCPT ); Wed, 2 Dec 2020 21:10:27 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:18352 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727681AbgLCCK0 (ORCPT ); Wed, 2 Dec 2020 21:10:26 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B322rE9113478; Wed, 2 Dec 2020 21:09:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=ylRBnpuamzDBA63iOoDUHX6UnApFo/wltbO4CWNulm0=; b=rIobVNc/9RG7zHBDQVyDbihx20iLw6q9OCghIAVIYlJZ6Dk5LT8TOoZSlMS/JukXfHZd /Opm3Sn/hreOqY7B4yoaIJVvE6EI8iVkCnwclGXAVOjzZVPIvx+eT2urGJMm3bHCnzof 7X7mCk6/RfcY95fOC4hBUiy2mmQFnulSOKobgXHyWINBoSuvsoyl4YQJb5/4HQ8L2rxc E5Wbb3+IEzsRdvmxaMginRMCM08BhE6DRZQkiFvCNd9b/kxvIH64YCEpI8qEEr2/puAN GCQhCQq+8EcJRLG8cxrXMekzwOx+tvBZIYlYKGkCyZsOQxxPi98cwe9Cae78lzxKmRvF 3A== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 356jdqf009-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 21:09:39 -0500 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B31uL48013325; Thu, 3 Dec 2020 02:09:38 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma01wdc.us.ibm.com with ESMTP id 355vrfvcjw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 03 Dec 2020 02:09:38 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B328FOo32047844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 3 Dec 2020 02:08:15 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B038678060; Thu, 3 Dec 2020 02:08:21 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 275F078063; Thu, 3 Dec 2020 02:08:21 +0000 (GMT) Received: from vios4361.aus.stglabs.ibm.com (unknown [9.3.43.61]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 3 Dec 2020 02:08:21 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v3 18/18] ibmvfc: drop host lock when completing commands in CRQ Date: Wed, 2 Dec 2020 20:08:06 -0600 Message-Id: <20201203020806.14747-19-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201203020806.14747-1-tyreld@linux.ibm.com> References: <20201203020806.14747-1-tyreld@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_14:2020-11-30,2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=1 bulkscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 malwarescore=0 mlxscore=0 impostorscore=0 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012030009 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The legacy CRQ holds the host lock the even while completing commands. This presents a problem when in legacy single queue mode and nr_hw_queues is greater than one since calling scsi_done() introduces the potential for deadlock. If nr_hw_queues is greater than one drop the hostlock in the legacy CRQ path when completing a command. Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c index e499599662ec..e2200bdff2be 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc.c +++ b/drivers/scsi/ibmvscsi/ibmvfc.c @@ -2969,6 +2969,7 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost) { long rc; struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba); + unsigned long flags; switch (crq->valid) { case IBMVFC_CRQ_INIT_RSP: @@ -3039,7 +3040,12 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost) del_timer(&evt->timer); list_del(&evt->queue); ibmvfc_trc_end(evt); - evt->done(evt); + if (nr_scsi_hw_queues > 1) { + spin_unlock_irqrestore(vhost->host->host_lock, flags); + evt->done(evt); + spin_lock_irqsave(vhost->host->host_lock, flags); + } else + evt->done(evt); } /**