From patchwork Sat Apr 4 10:48:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5541DC25B07 for ; Wed, 3 Aug 2022 16:29:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238246AbiHCQ3V (ORCPT ); Wed, 3 Aug 2022 12:29:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237658AbiHCQ3R (ORCPT ); Wed, 3 Aug 2022 12:29:17 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 932A7DEBD; Wed, 3 Aug 2022 09:29:15 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 3D9CE412C8; Wed, 3 Aug 2022 16:29:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544152; x=1661358553; bh=cWTZkdj1OTtRNc40YGJ3Y4Qr8 RkuXngG7Uzgs5cBdAU=; b=RPxPpAqD8+Xhi9nW/8xU2UxAM4vH038ReFTn+da4Z 5lHX2Wzx42p+8L2nRnU23VMo5tPCMYb0ih7F1of8iv+tojFFXjfxNSUXiK16HW5E WBz73u2xF7D9w4zoSMcOjOewk4q+DzXtT1ph9ys7nFmybt4qM9zO2ss1+CtXtpxh lo= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AOzRwOfmC1AA; Wed, 3 Aug 2022 19:29:12 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id B8FA341283; Wed, 3 Aug 2022 19:29:11 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:11 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:10 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 01/48] scsi: target/core: Add a way to hide a port group Date: Sat, 4 Apr 2020 13:48:52 +0300 Message-ID: <20220803162857.27770-2-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov Default target port group is always returned in the list of port groups, even if the behaviour is unwanted, i.e. it has no members and non-default port groups are primary port groups. A new port group attribute - "hidden" can be used to hide empty port groups with no ports in REPORT TARGET PORT GROUPS, including default target port group: echo 1 > $DEVICE/alua/default_tg_pt_gp/hidden Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_alua.c | 8 ++++++++ drivers/target/target_core_configfs.c | 29 +++++++++++++++++++++++++++ include/target/target_core_base.h | 1 + 3 files changed, 38 insertions(+) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index 58df0145e8d0..9d1a8238a4c6 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -164,6 +164,8 @@ target_emulate_report_target_port_groups(struct se_cmd *cmd) spin_lock(&dev->t10_alua.tg_pt_gps_lock); list_for_each_entry(tg_pt_gp, &dev->t10_alua.tg_pt_gps_list, tg_pt_gp_list) { + if (tg_pt_gp->tg_pt_gp_hidden && !tg_pt_gp->tg_pt_gp_members) + continue; /* * Check if the Target port group and Target port descriptor list * based on tg_pt_gp_members count will fit into the response payload. @@ -308,6 +310,12 @@ target_emulate_set_target_port_groups(struct se_cmd *cmd) rc = TCM_UNSUPPORTED_SCSI_OPCODE; goto out; } + if (l_tg_pt_gp->tg_pt_gp_hidden && !tg_pt_gp->tg_pt_gp_members) { + spin_unlock(&l_lun->lun_tg_pt_gp_lock); + pr_err("Unable to change state for hidden port group with no members\n"); + rc = TCM_INVALID_CDB_FIELD; + goto out; + } valid_states = l_tg_pt_gp->tg_pt_gp_alua_supported_states; rcu_read_unlock(); diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 416514c5c7ac..7c0e42e782de 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -3029,6 +3029,33 @@ static ssize_t target_tg_pt_gp_preferred_store(struct config_item *item, return core_alua_store_preferred_bit(to_tg_pt_gp(item), page, count); } +static ssize_t target_tg_pt_gp_hidden_show(struct config_item *item, + char *page) +{ + return sprintf(page, "%d\n", to_tg_pt_gp(item)->tg_pt_gp_hidden); +} + +static ssize_t target_tg_pt_gp_hidden_store(struct config_item *item, + const char *page, size_t count) +{ + struct t10_alua_tg_pt_gp *tg_pt_gp = to_tg_pt_gp(item); + int tmp, ret; + + ret = kstrtoint(page, 0, &tmp); + if (ret < 0) { + pr_err("Unable to extract hidden flag: %d\n", ret); + return ret; + } + + if (tmp != 0 && tmp != 1) { + pr_err("Illegal value for hidden flag: %d\n", tmp); + return -EINVAL; + } + tg_pt_gp->tg_pt_gp_hidden = tmp; + + return count; +} + static ssize_t target_tg_pt_gp_tg_pt_gp_id_show(struct config_item *item, char *page) { @@ -3115,6 +3142,7 @@ CONFIGFS_ATTR(target_tg_pt_gp_, alua_support_standby); CONFIGFS_ATTR(target_tg_pt_gp_, alua_support_active_optimized); CONFIGFS_ATTR(target_tg_pt_gp_, alua_support_active_nonoptimized); CONFIGFS_ATTR(target_tg_pt_gp_, alua_write_metadata); +CONFIGFS_ATTR(target_tg_pt_gp_, hidden); CONFIGFS_ATTR(target_tg_pt_gp_, nonop_delay_msecs); CONFIGFS_ATTR(target_tg_pt_gp_, trans_delay_msecs); CONFIGFS_ATTR(target_tg_pt_gp_, implicit_trans_secs); @@ -3138,6 +3166,7 @@ static struct configfs_attribute *target_core_alua_tg_pt_gp_attrs[] = { &target_tg_pt_gp_attr_trans_delay_msecs, &target_tg_pt_gp_attr_implicit_trans_secs, &target_tg_pt_gp_attr_preferred, + &target_tg_pt_gp_attr_hidden, &target_tg_pt_gp_attr_tg_pt_gp_id, &target_tg_pt_gp_attr_members, NULL, diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index c2b36f7d917d..f607ff74c368 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -297,6 +297,7 @@ struct t10_alua_tg_pt_gp { int tg_pt_gp_trans_delay_msecs; int tg_pt_gp_implicit_trans_secs; int tg_pt_gp_pref; + int tg_pt_gp_hidden; int tg_pt_gp_write_metadata; u32 tg_pt_gp_members; int tg_pt_gp_alua_access_state; From patchwork Mon Apr 20 00:18:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65755C19F2D for ; Wed, 3 Aug 2022 16:29:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238218AbiHCQ3S (ORCPT ); Wed, 3 Aug 2022 12:29:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236336AbiHCQ3Q (ORCPT ); Wed, 3 Aug 2022 12:29:16 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8050F6437; Wed, 3 Aug 2022 09:29:14 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id E3E754128A; Wed, 3 Aug 2022 16:29:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544151; x=1661358552; bh=gD2QuUT4XkhOvnrXXvyNPQCEo eXbpe6JDx+hN39zLh0=; b=su+7cP8utoawnu68/3SDKeDoouTqOguRSFmqpdEpv h+xt5wy3hoP+XQ+EHjOARTzJFKVEhzCY7rUsHexfBSAz0jijtN3vudsK/7RFwKo/ auxIbvSqweFY4/5bQ+BJZf5+Rlq9j8PSagh0nEfCZwfCL1+3jzUOq2IJozS8CeZY 00= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oFbw9kXJYdlJ; Wed, 3 Aug 2022 19:29:11 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id B2C264127C; Wed, 3 Aug 2022 19:29:11 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:11 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:10 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Konstantin Vinogradov , "Ivan Efremov" , Dmitry Bogdanov Subject: [RFC PATCH 02/48] scsi: target/core: Set MULTIP bit for se_device with multiple ports Date: Mon, 20 Apr 2020 03:18:45 +0300 Message-ID: <20220803162857.27770-3-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov SAM-5 4.8.3 (SCSI target device with multiple SCSI ports structure) obligates to set MULTIP bit when there's multiple SCSI target ports: > Each device server shall indicate the presence of multiple SCSI target > ports by setting the MULTIP bit to one in its standard INQUIRY data > (see SPC-4). The change sets MULTIP bit automatically to indicate the presence of multiple SCSI target ports within standard inquiry response data if there are multiple target ports in all target port groups of the se_device. Signed-off-by: Roman Bolshakov Signed-off-by: Konstantin Vinogradov Signed-off-by: Ivan Efremov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_spc.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/target/target_core_spc.c b/drivers/target/target_core_spc.c index c14441c89bed..252d704264ed 100644 --- a/drivers/target/target_core_spc.c +++ b/drivers/target/target_core_spc.c @@ -75,6 +75,8 @@ spc_emulate_inquiry_std(struct se_cmd *cmd, unsigned char *buf) struct se_portal_group *tpg = lun->lun_tpg; struct se_device *dev = cmd->se_dev; struct se_session *sess = cmd->se_sess; + struct t10_alua_tg_pt_gp *tg_pt_gp; + u64 dev_ports = 0; /* Set RMB (removable media) for tape devices */ if (dev->transport->get_device_type(dev) == TYPE_TAPE) @@ -115,6 +117,21 @@ spc_emulate_inquiry_std(struct se_cmd *cmd, unsigned char *buf) buf[5] |= 0x1; } + spin_lock(&dev->t10_alua.tg_pt_gps_lock); + list_for_each_entry(tg_pt_gp, &dev->t10_alua.tg_pt_gps_list, + tg_pt_gp_list) { + dev_ports += tg_pt_gp->tg_pt_gp_members; + /* Exact number of ports does not matter for MULTIP bit */ + if (dev_ports > 1) + break; + } + spin_unlock(&dev->t10_alua.tg_pt_gps_lock); + /* + * Set MULTIP bit to indicate presence of multiple SCSI target ports + */ + if (dev_ports > 1) + buf[6] |= 0x10; + buf[7] = 0x2; /* CmdQue=1 */ /* From patchwork Mon Apr 20 17:20:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF509C19F29 for ; Wed, 3 Aug 2022 16:29:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236808AbiHCQ3W (ORCPT ); Wed, 3 Aug 2022 12:29:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236679AbiHCQ3R (ORCPT ); Wed, 3 Aug 2022 12:29:17 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0470BDEDB; Wed, 3 Aug 2022 09:29:16 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id A790F412C5; Wed, 3 Aug 2022 16:29:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544153; x=1661358554; bh=M7bIgwjGB8IpWO1nTeQabcybK tqluBRod/zJ/cf+cGs=; b=JfwhYYZA8IZzRQkY3uDO4k1K6B8lvzywYvosJg0vo 0W0p5/IipvDYozuhj+buH1DTQCX9Z+GIfbGTHq0D67TDmyCMTswrwbLMPZqbRi7/ sIO43oLOmCtLkQiCvqQ+fAhXXOrZ+BfLP7Qoy7VfcCSO3cZBGQXYkgnZddSucJdW c0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ij2NwWhHly_U; Wed, 3 Aug 2022 19:29:13 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id BDD5741284; Wed, 3 Aug 2022 19:29:11 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:11 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:10 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 03/48] scsi: target/core: Add cleanup sequence in core_tpg_register() Date: Mon, 20 Apr 2020 20:20:16 +0300 Message-ID: <20220803162857.27770-4-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov It does not change any functionality but allows to introduce more steps in the cleanup sequence without code duplication later. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_tpg.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index 736847c933e5..f0d38d77edcc 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -484,10 +484,8 @@ int core_tpg_register( ret = core_tpg_add_lun(se_tpg, se_tpg->tpg_virt_lun0, true, g_lun0_dev); - if (ret < 0) { - kfree(se_tpg->tpg_virt_lun0); - return ret; - } + if (ret < 0) + goto out_free_lun0; } pr_debug("TARGET_CORE[%s]: Allocated portal_group for endpoint: %s, " @@ -497,6 +495,10 @@ int core_tpg_register( se_tpg->proto_id, se_tpg->se_tpg_tfo->tpg_get_tag(se_tpg)); return 0; + +out_free_lun0: + kfree(se_tpg->tpg_virt_lun0); + return ret; } EXPORT_SYMBOL(core_tpg_register); From patchwork Mon Apr 20 17:57:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE067C19F28 for ; Wed, 3 Aug 2022 16:29:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238249AbiHCQ3X (ORCPT ); Wed, 3 Aug 2022 12:29:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235210AbiHCQ3S (ORCPT ); Wed, 3 Aug 2022 12:29:18 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0648167D4; Wed, 3 Aug 2022 09:29:16 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9DFC6412D1; Wed, 3 Aug 2022 16:29:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544154; x=1661358555; bh=BKueMVILuHGTa7Cku0CCwxAzc UvksIUmExMUkgy0exQ=; b=St6b8nTek8BsBYwqKBwq7l9dd0DfmB71YahYeR1uj 1zswPr6dYqKwJMiK7+Jju/HAFzruiqh1n4xjpoTt8CNFnUZJNpoqq/IY1nz8MI8E f3LeUcqhwB7wJKc23pPD865L+t5/1PvNeF8ZL/LxbL01sow7q6YRbMGtU33mvEA3 Ys= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ODHGVaPRVC8s; Wed, 3 Aug 2022 19:29:14 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1799541287; Wed, 3 Aug 2022 19:29:12 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:12 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:11 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 04/48] scsi: target/core: Add RTPI field to target port Date: Mon, 20 Apr 2020 20:57:55 +0300 Message-ID: <20220803162857.27770-5-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov SAM-5 4.6.5.2 (Relative Port Identifier attribute) defines the attribute as unique across SCSI target ports. The change introduces RTPI attribute to se_portal group. The value is auto-incremented and unique across all SCSI target ports. It also limits number of SCSI target ports to 65535. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_tpg.c | 78 +++++++++++++++++++++++++++++-- include/target/target_core_base.h | 4 ++ 2 files changed, 77 insertions(+), 5 deletions(-) diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index f0d38d77edcc..1b7ab98bbd04 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -31,6 +31,10 @@ #include "target_core_ua.h" extern struct se_device *g_lun0_dev; +static u16 g_tpg_count; +static u16 g_tpg_rtpi_counter = 1; +static LIST_HEAD(g_tpg_list); +static DEFINE_SPINLOCK(g_tpg_lock); /* __core_tpg_get_initiator_node_acl(): * @@ -439,6 +443,57 @@ static void core_tpg_lun_ref_release(struct percpu_ref *ref) complete(&lun->lun_shutdown_comp); } +static int core_tpg_register_rtpi(struct se_portal_group *se_tpg) +{ + struct se_portal_group *tpg; + + /* + * Allocate the next RELATIVE TARGET PORT IDENTIFIER. + * Here is the table from SPC-4 4.3.4: + * + * Table 34 -- Relative target port identifier values + * + * Value Description + * 0h Reserved + * 1h Relative port 1, historically known as port A + * 2h Relative port 2, historically known as port B + * 3h to FFFFh Relative port 3 through 65 535 + */ + spin_lock(&g_tpg_lock); + + if (g_tpg_count == 0xffff) { + spin_unlock(&g_tpg_lock); + pr_warn("Reached g_tpg_count == 0xffff\n"); + return -ENOSPC; + } +again: + se_tpg->tpg_rtpi = g_tpg_rtpi_counter++; + if (!se_tpg->tpg_rtpi) + goto again; + + list_for_each_entry(tpg, &g_tpg_list, tpg_list) { + /* + * Make sure RELATIVE TARGET PORT IDENTIFIER is unique + * for 16-bit wrap.. + */ + if (se_tpg->tpg_rtpi == tpg->tpg_rtpi) + goto again; + } + list_add(&se_tpg->tpg_list, &g_tpg_list); + g_tpg_count++; + spin_unlock(&g_tpg_lock); + + return 0; +} + +static void core_tpg_deregister_rtpi(struct se_portal_group *se_tpg) +{ + spin_lock(&g_tpg_lock); + list_del(&se_tpg->tpg_list); + g_tpg_count--; + spin_unlock(&g_tpg_lock); +} + /* Does not change se_wwn->priv. */ int core_tpg_register( struct se_wwn *se_wwn, @@ -471,6 +526,7 @@ int core_tpg_register( se_tpg->proto_id = proto_id; se_tpg->se_tpg_wwn = se_wwn; atomic_set(&se_tpg->tpg_pr_ref_count, 0); + INIT_LIST_HEAD(&se_tpg->tpg_list); INIT_LIST_HEAD(&se_tpg->acl_node_list); INIT_LIST_HEAD(&se_tpg->tpg_sess_list); spin_lock_init(&se_tpg->session_lock); @@ -478,9 +534,15 @@ int core_tpg_register( mutex_init(&se_tpg->acl_node_mutex); if (se_tpg->proto_id >= 0) { + ret = core_tpg_register_rtpi(se_tpg); + if (ret < 0) + return ret; + se_tpg->tpg_virt_lun0 = core_tpg_alloc_lun(se_tpg, 0); - if (IS_ERR(se_tpg->tpg_virt_lun0)) - return PTR_ERR(se_tpg->tpg_virt_lun0); + if (IS_ERR(se_tpg->tpg_virt_lun0)) { + ret = PTR_ERR(se_tpg->tpg_virt_lun0); + goto out_deregister_rtpi; + } ret = core_tpg_add_lun(se_tpg, se_tpg->tpg_virt_lun0, true, g_lun0_dev); @@ -488,16 +550,20 @@ int core_tpg_register( goto out_free_lun0; } - pr_debug("TARGET_CORE[%s]: Allocated portal_group for endpoint: %s, " - "Proto: %d, Portal Tag: %u\n", se_tpg->se_tpg_tfo->fabric_name, + pr_debug("TARGET_CORE[%s]: Allocated portal_group for endpoint: %s, Proto: %d, Portal Tag: %u, RTPI: %#2x\n", + se_tpg->se_tpg_tfo->fabric_name, se_tpg->se_tpg_tfo->tpg_get_wwn(se_tpg) ? se_tpg->se_tpg_tfo->tpg_get_wwn(se_tpg) : NULL, - se_tpg->proto_id, se_tpg->se_tpg_tfo->tpg_get_tag(se_tpg)); + se_tpg->proto_id, + se_tpg->se_tpg_tfo->tpg_get_tag(se_tpg), + se_tpg->tpg_rtpi); return 0; out_free_lun0: kfree(se_tpg->tpg_virt_lun0); +out_deregister_rtpi: + core_tpg_deregister_rtpi(se_tpg); return ret; } EXPORT_SYMBOL(core_tpg_register); @@ -537,6 +603,8 @@ int core_tpg_deregister(struct se_portal_group *se_tpg) kfree_rcu(se_tpg->tpg_virt_lun0, rcu_head); } + core_tpg_deregister_rtpi(se_tpg); + return 0; } EXPORT_SYMBOL(core_tpg_deregister); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index f607ff74c368..72fe94afc7a5 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -904,6 +904,8 @@ struct se_portal_group { */ int proto_id; bool enabled; + /* RELATIVE TARGET PORT IDENTIFIER */ + u16 tpg_rtpi; /* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */ atomic_t tpg_pr_ref_count; /* Spinlock for adding/removing ACLed Nodes */ @@ -911,6 +913,8 @@ struct se_portal_group { /* Spinlock for adding/removing sessions */ spinlock_t session_lock; struct mutex tpg_lun_mutex; + /* List of all SCSI target ports */ + struct list_head tpg_list; /* linked list for initiator ACL list */ struct list_head acl_node_list; struct hlist_head tpg_lun_hlist; From patchwork Sat Dec 21 23:45:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E63D1C19F28 for ; Wed, 3 Aug 2022 16:29:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238273AbiHCQ30 (ORCPT ); Wed, 3 Aug 2022 12:29:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236336AbiHCQ3T (ORCPT ); Wed, 3 Aug 2022 12:29:19 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B76FB7F1; Wed, 3 Aug 2022 09:29:18 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 0D171412CC; Wed, 3 Aug 2022 16:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544154; x=1661358555; bh=bTn0+8wdAkk+C7HrrnZK+MAoT gpu4GHz8yPpg+OGVZ8=; b=Rkei+HlEHT4cH4zJ3aSoEPmxZO6DX3mElG2ZNTHDu soC/Is42wMve+qXKlUFs45rNcCiVN0Xn/qZZjNKNM9lM9TK+rifvQcLZ9rT1j5N8 OYU04NqVAkD6ZA02Xpjp3Yoxm6EtLWqYs/8BpgdfRCa8478oelPHZdFq0D49HXs6 Nw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6a6ew5FqCpt7; Wed, 3 Aug 2022 19:29:14 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 5A6F041291; Wed, 3 Aug 2022 19:29:12 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:12 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:11 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 05/48] scsi: target/core: Use RTPI from target port Date: Sun, 22 Dec 2019 02:45:33 +0300 Message-ID: <20220803162857.27770-6-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov Replace all references to RTPI from LUN field to se_portal_group field. It introduces consitent reporting of RTPI for all LUNs and all target ports. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_alua.c | 4 ++-- drivers/target/target_core_device.c | 2 +- drivers/target/target_core_pr.c | 8 ++++---- drivers/target/target_core_spc.c | 2 +- drivers/target/target_core_stat.c | 6 +++--- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index 9d1a8238a4c6..774750715ba8 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -224,7 +224,7 @@ target_emulate_report_target_port_groups(struct se_cmd *cmd) /* * Set RELATIVE TARGET PORT IDENTIFIER */ - put_unaligned_be16(lun->lun_rtpi, &buf[off]); + put_unaligned_be16(lun->lun_tpg->tpg_rtpi, &buf[off]); off += 2; rd_len += 4; } @@ -404,7 +404,7 @@ target_emulate_set_target_port_groups(struct se_cmd *cmd) spin_lock(&dev->se_port_lock); list_for_each_entry(lun, &dev->dev_sep_list, lun_dev_link) { - if (lun->lun_rtpi != rtpi) + if (lun->lun_tpg->tpg_rtpi != rtpi) continue; // XXX: racy unlock diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 086ac9c9343c..bbef6b869ee9 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -223,7 +223,7 @@ struct se_dev_entry *core_get_se_deve_from_rtpi( tpg->se_tpg_tfo->fabric_name); continue; } - if (lun->lun_rtpi != rtpi) + if (lun->lun_tpg->tpg_rtpi != rtpi) continue; kref_get(&deve->pr_kref); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 3829b61b56c1..668670b1414a 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -663,7 +663,7 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration( } pr_reg->pr_res_mapped_lun = mapped_lun; pr_reg->pr_aptpl_target_lun = lun->unpacked_lun; - pr_reg->tg_pt_sep_rtpi = lun->lun_rtpi; + pr_reg->tg_pt_sep_rtpi = lun->lun_tpg->tpg_rtpi; pr_reg->pr_res_key = sa_res_key; pr_reg->pr_reg_all_tg_pt = all_tg_pt; pr_reg->pr_reg_aptpl = aptpl; @@ -969,7 +969,7 @@ static int __core_scsi3_check_aptpl_registration( rcu_read_unlock(); pr_reg->pr_reg_nacl = nacl; - pr_reg->tg_pt_sep_rtpi = lun->lun_rtpi; + pr_reg->tg_pt_sep_rtpi = lun->lun_tpg->tpg_rtpi; list_del(&pr_reg->pr_reg_aptpl_list); spin_unlock(&pr_tmpl->aptpl_reg_lock); /* @@ -1577,7 +1577,7 @@ core_scsi3_decode_spec_i_port( */ if (tmp_tpg->proto_id != proto_ident) continue; - dest_rtpi = tmp_lun->lun_rtpi; + dest_rtpi = tmp_lun->lun_tpg->tpg_rtpi; iport_ptr = NULL; i_str = target_parse_pr_out_transport_id(tmp_tpg, @@ -3221,7 +3221,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, spin_lock(&dev->se_port_lock); list_for_each_entry(tmp_lun, &dev->dev_sep_list, lun_dev_link) { - if (tmp_lun->lun_rtpi != rtpi) + if (tmp_lun->lun_tpg->tpg_rtpi != rtpi) continue; dest_se_tpg = tmp_lun->lun_tpg; dest_tf_ops = dest_se_tpg->se_tpg_tfo; diff --git a/drivers/target/target_core_spc.c b/drivers/target/target_core_spc.c index 252d704264ed..be18a72d9f8a 100644 --- a/drivers/target/target_core_spc.c +++ b/drivers/target/target_core_spc.c @@ -332,7 +332,7 @@ spc_emulate_evpd_83(struct se_cmd *cmd, unsigned char *buf) /* Skip over Obsolete field in RTPI payload * in Table 472 */ off += 2; - put_unaligned_be16(lun->lun_rtpi, &buf[off]); + put_unaligned_be16(lun->lun_tpg->tpg_rtpi, &buf[off]); off += 2; len += 8; /* Header size + Designation descriptor */ /* diff --git a/drivers/target/target_core_stat.c b/drivers/target/target_core_stat.c index 62d15bcc3d93..8fd6efa2b978 100644 --- a/drivers/target/target_core_stat.c +++ b/drivers/target/target_core_stat.c @@ -455,7 +455,7 @@ static ssize_t target_stat_port_indx_show(struct config_item *item, char *page) rcu_read_lock(); dev = rcu_dereference(lun->lun_se_dev); if (dev) - ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_rtpi); + ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_tpg->tpg_rtpi); rcu_read_unlock(); return ret; } @@ -561,7 +561,7 @@ static ssize_t target_stat_tgt_port_indx_show(struct config_item *item, rcu_read_lock(); dev = rcu_dereference(lun->lun_se_dev); if (dev) - ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_rtpi); + ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_tpg->tpg_rtpi); rcu_read_unlock(); return ret; } @@ -579,7 +579,7 @@ static ssize_t target_stat_tgt_port_name_show(struct config_item *item, if (dev) ret = snprintf(page, PAGE_SIZE, "%sPort#%u\n", tpg->se_tpg_tfo->fabric_name, - lun->lun_rtpi); + lun->lun_tpg->tpg_rtpi); rcu_read_unlock(); return ret; } From patchwork Sat Dec 21 23:49:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F1B9C3F6B0 for ; Wed, 3 Aug 2022 16:29:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238360AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236503AbiHCQ3T (ORCPT ); Wed, 3 Aug 2022 12:29:19 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E4A3DEBD; Wed, 3 Aug 2022 09:29:18 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id E4574412D8; Wed, 3 Aug 2022 16:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544155; x=1661358556; bh=uDcexxKfPmb+UsKJ436OueHFa f0oq73Ey9mzgbzMB5Q=; b=aL6ojNbNJatsA8l+779hB9xCPH1pcTlObwD6ezNYb JmgGZv5VfsGx2RWrOmsQKK2X3c2vZxGzt7hw5pL8IgRwQvwn+rocvKNwpwqeb1QO OJQoGjfOYtWDuDg613QdDWdDWx+OgJB3r0rWvzhRzKe8WDkD8k9zu7YOWtbMWxAt P0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N8i05648f5uN; Wed, 3 Aug 2022 19:29:15 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id B5E8241292; Wed, 3 Aug 2022 19:29:12 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:12 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:12 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 06/48] scsi: target/core: Drop device-based RTPI Date: Sun, 22 Dec 2019 02:49:48 +0300 Message-ID: <20220803162857.27770-7-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov The code is not needed since target port-based RTPI allocation replaced it. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_device.c | 41 --------------------------- drivers/target/target_core_internal.h | 1 - drivers/target/target_core_tpg.c | 6 ---- include/target/target_core_base.h | 4 --- 4 files changed, 52 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index bbef6b869ee9..40a582b084d7 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -470,47 +470,6 @@ void core_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg) mutex_unlock(&tpg->acl_node_mutex); } -int core_alloc_rtpi(struct se_lun *lun, struct se_device *dev) -{ - struct se_lun *tmp; - - spin_lock(&dev->se_port_lock); - if (dev->export_count == 0x0000ffff) { - pr_warn("Reached dev->dev_port_count ==" - " 0x0000ffff\n"); - spin_unlock(&dev->se_port_lock); - return -ENOSPC; - } -again: - /* - * Allocate the next RELATIVE TARGET PORT IDENTIFIER for this struct se_device - * Here is the table from spc4r17 section 7.7.3.8. - * - * Table 473 -- RELATIVE TARGET PORT IDENTIFIER field - * - * Code Description - * 0h Reserved - * 1h Relative port 1, historically known as port A - * 2h Relative port 2, historically known as port B - * 3h to FFFFh Relative port 3 through 65 535 - */ - lun->lun_rtpi = dev->dev_rpti_counter++; - if (!lun->lun_rtpi) - goto again; - - list_for_each_entry(tmp, &dev->dev_sep_list, lun_dev_link) { - /* - * Make sure RELATIVE TARGET PORT IDENTIFIER is unique - * for 16-bit wrap.. - */ - if (lun->lun_rtpi == tmp->lun_rtpi) - goto again; - } - spin_unlock(&dev->se_port_lock); - - return 0; -} - static void se_release_vpd_for_dev(struct se_device *dev) { struct t10_vpd *vpd, *vpd_tmp; diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index a889a6237d9c..7dcc62749edd 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -59,7 +59,6 @@ struct target_fabric_configfs { extern struct t10_alua_lu_gp *default_lu_gp; /* target_core_device.c */ -int core_alloc_rtpi(struct se_lun *lun, struct se_device *dev); struct se_dev_entry *core_get_se_deve_from_rtpi(struct se_node_acl *, u16); void target_pr_kref_release(struct kref *); void core_free_device_list_for_node(struct se_node_acl *, diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index 1b7ab98bbd04..502bd85bf75c 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -648,10 +648,6 @@ int core_tpg_add_lun( if (ret < 0) goto out; - ret = core_alloc_rtpi(lun, dev); - if (ret) - goto out_kill_ref; - if (!(dev->transport_flags & TRANSPORT_FLAG_PASSTHROUGH_ALUA) && !(dev->se_hba->hba_flags & HBA_FLAGS_INTERNAL_USE)) target_attach_tg_pt_gp(lun, dev->t10_alua.default_tg_pt_gp); @@ -675,8 +671,6 @@ int core_tpg_add_lun( return 0; -out_kill_ref: - percpu_ref_exit(&lun->lun_ref); out: return ret; } diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 72fe94afc7a5..6ed484683ab6 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -734,8 +734,6 @@ struct se_lun { bool lun_access_ro; u32 lun_index; - /* RELATIVE TARGET PORT IDENTIFER */ - u16 lun_rtpi; atomic_t lun_acl_count; struct se_device __rcu *lun_se_dev; @@ -787,8 +785,6 @@ struct se_device_queue { }; struct se_device { - /* RELATIVE TARGET PORT IDENTIFER Counter */ - u16 dev_rpti_counter; /* Used for SAM Task Attribute ordering */ u32 dev_cur_ordered_id; u32 dev_flags; From patchwork Sat Dec 21 00:53:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC77BC25B08 for ; Wed, 3 Aug 2022 16:29:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238379AbiHCQ3g (ORCPT ); Wed, 3 Aug 2022 12:29:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238236AbiHCQ3T (ORCPT ); Wed, 3 Aug 2022 12:29:19 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF368167D4; Wed, 3 Aug 2022 09:29:18 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 61443412C6; Wed, 3 Aug 2022 16:29:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544156; x=1661358557; bh=xtaI47DiYCSiYCtfR0l+q74xA h2uT5t2R4YiDZGtSe0=; b=JfhglayCYIJ4/fpQwgrhLvbx9cE9BKNJkTFMP8/5T mvfCRAmb6yJqtQv56wOMzEJf9kAM593ArfNDHqcvno19zWCywZLWK7gphaNCNtcM z04/TZCQiY4VxI3KuIarkh+m5rT3HDe0oYXxE/QJwsofyc+nyn45O/FjQClkDia2 +Q= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3S_keEhZiq14; Wed, 3 Aug 2022 19:29:16 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 0066A4127C; Wed, 3 Aug 2022 19:29:12 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:13 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:12 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 07/48] scsi: target/core: Add common port attributes Date: Sat, 21 Dec 2019 03:53:23 +0300 Message-ID: <20220803162857.27770-8-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov All SCSI target port attributes (tpgN/attribs/attrname) are defined and implemented in fabric modules in existing implementation. The change introduces a way to have common tpg attribs in configfs for all fabrics without code duplication. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 9 ++++- drivers/target/target_core_fabric_configfs.c | 39 +++++++++++++++++++- drivers/target/target_core_internal.h | 1 + drivers/target/target_core_tpg.c | 4 ++ 4 files changed, 50 insertions(+), 3 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 7c0e42e782de..40b9808738d2 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -464,13 +464,19 @@ int target_register_template(const struct target_core_fabric_ops *fo) INIT_LIST_HEAD(&tf->tf_list); atomic_set(&tf->tf_access_cnt, 0); tf->tf_ops = fo; - target_fabric_setup_cits(tf); + ret = target_fabric_setup_cits(tf); + if (ret) + goto out; mutex_lock(&g_tf_lock); list_add_tail(&tf->tf_list, &g_tf_list); mutex_unlock(&g_tf_lock); return 0; + +out: + kfree(tf); + return ret; } EXPORT_SYMBOL(target_register_template); @@ -491,6 +497,7 @@ void target_unregister_template(const struct target_core_fabric_ops *fo) */ rcu_barrier(); kfree(t->tf_tpg_base_cit.ct_attrs); + kfree(t->tf_tpg_attrib_cit.ct_attrs); kfree(t); return; } diff --git a/drivers/target/target_core_fabric_configfs.c b/drivers/target/target_core_fabric_configfs.c index 95a88f6224cd..a34b5db4eec5 100644 --- a/drivers/target/target_core_fabric_configfs.c +++ b/drivers/target/target_core_fabric_configfs.c @@ -795,7 +795,38 @@ TF_CIT_SETUP(tpg_lun, NULL, &target_fabric_lun_group_ops, NULL); /* End of tfc_tpg_lun_cit */ -TF_CIT_SETUP_DRV(tpg_attrib, NULL, NULL); +static int +target_fabric_setup_tpg_attrib_cit(struct target_fabric_configfs *tf) +{ + int i, k, len = 0; + struct config_item_type *cit = &tf->tf_tpg_attrib_cit; + struct configfs_attribute **attrs; + + for (i = 0; core_tpg_attrib_attrs[i]; i++) + len += sizeof(struct configfs_attribute *); + if (tf->tf_ops->tfc_tpg_attrib_attrs) + for (i = 0; tf->tf_ops->tfc_tpg_attrib_attrs[i]; i++) + len += sizeof(struct configfs_attribute *); + len += sizeof(struct configfs_attribute *); + + attrs = kzalloc(len, GFP_KERNEL); + if (!attrs) + return -ENOMEM; + + for (i = 0; core_tpg_attrib_attrs[i]; i++) + attrs[i] = core_tpg_attrib_attrs[i]; + if (tf->tf_ops->tfc_tpg_attrib_attrs) + for (k = 0; tf->tf_ops->tfc_tpg_attrib_attrs[k]; k++, i++) + attrs[i] = tf->tf_ops->tfc_tpg_attrib_attrs[k]; + attrs[i] = NULL; + + cit->ct_attrs = attrs; + cit->ct_owner = tf->tf_ops->module; + pr_debug("Setup generic tpg_attrib\n"); + + return 0; +} + TF_CIT_SETUP_DRV(tpg_auth, NULL, NULL); TF_CIT_SETUP_DRV(tpg_param, NULL, NULL); @@ -1113,7 +1144,11 @@ int target_fabric_setup_cits(struct target_fabric_configfs *tf) target_fabric_setup_tpg_lun_cit(tf); target_fabric_setup_tpg_np_cit(tf); target_fabric_setup_tpg_np_base_cit(tf); - target_fabric_setup_tpg_attrib_cit(tf); + + ret = target_fabric_setup_tpg_attrib_cit(tf); + if (ret) + return ret; + target_fabric_setup_tpg_auth_cit(tf); target_fabric_setup_tpg_param_cit(tf); target_fabric_setup_tpg_nacl_cit(tf); diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 7dcc62749edd..d662cdc9a04c 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -117,6 +117,7 @@ int core_tmr_lun_reset(struct se_device *, struct se_tmr_req *, /* target_core_tpg.c */ extern struct se_device *g_lun0_dev; +extern struct configfs_attribute *core_tpg_attrib_attrs[]; struct se_node_acl *__core_tpg_get_initiator_node_acl(struct se_portal_group *tpg, const char *); diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index 502bd85bf75c..61fad44cf205 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -714,3 +714,7 @@ void core_tpg_remove_lun( percpu_ref_exit(&lun->lun_ref); } + +struct configfs_attribute *core_tpg_attrib_attrs[] = { + NULL, +}; From patchwork Tue Apr 21 14:00:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69737C3F6B0 for ; Wed, 3 Aug 2022 16:29:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238393AbiHCQ3i (ORCPT ); Wed, 3 Aug 2022 12:29:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238237AbiHCQ3U (ORCPT ); Wed, 3 Aug 2022 12:29:20 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8325F2610A; Wed, 3 Aug 2022 09:29:19 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 32693412D6; Wed, 3 Aug 2022 16:29:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544156; x=1661358557; bh=t0gWfskAxIMNQ8FvZn20aDt2R TIXHrsWhyzZRVYohBM=; b=hO+Wf+v6vX+Bb2CxTSA67euN/dcOnRFX49/ZLQu/e FQKNKLA15IE+CHE3mpxzZEWtTfKQOa44khhkhCkTzVxoO+K5UBcc1NZK6D+SQmwS hCOXmraWliypU+Wk9pMqkb6izUJmS0cHoyatMgQrPMkR3/nadperh//2uQqH0x0Y fM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q4ZF-NnPDRzR; Wed, 3 Aug 2022 19:29:16 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 539154129F; Wed, 3 Aug 2022 19:29:13 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:13 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:12 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Roman Bolshakov , Dmitry Bogdanov Subject: [RFC PATCH 08/48] scsi: target/core: Add RTPI attribute for target port Date: Tue, 21 Apr 2020 17:00:11 +0300 Message-ID: <20220803162857.27770-9-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Roman Bolshakov RELATIVE TARGET PORT IDENTIFIER can be read and configured via configfs: $ echo 0x10 > $TARGET/tpgt_N/attrib/rtpi RTPI has to be unique among all target ports, otherwise the write fails. Configuration of a new RTPI value triggers INQUIRY DATA HAS CHANGED unit attention as defined in SPC-4 "6.6.1 INQUIRY command introduction": > If INQUIRY data changes for any reason, the device server shall > establish a unit attention condition for the initiator port associated > with every I_T nexus (see SAM-5), with the additional sense code set to > INQUIRY DATA HAS CHANGED. Signed-off-by: Roman Bolshakov Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_tpg.c | 70 ++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index 61fad44cf205..a409cf1c4ca0 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -715,6 +715,76 @@ void core_tpg_remove_lun( percpu_ref_exit(&lun->lun_ref); } +/* Set a Unit Attention on all I_T nexuses related to the port */ +static void core_tpg_ua(struct se_portal_group *se_tpg, u8 asc, u8 ascq) +{ + struct se_lun *lun; + struct se_dev_entry *se_deve; + + mutex_lock(&se_tpg->tpg_lun_mutex); + hlist_for_each_entry_rcu(lun, &se_tpg->tpg_lun_hlist, link) { + spin_lock(&lun->lun_deve_lock); + list_for_each_entry(se_deve, &lun->lun_deve_list, lun_link) + core_scsi3_ua_allocate(se_deve, asc, ascq); + spin_unlock(&lun->lun_deve_lock); + } + mutex_unlock(&se_tpg->tpg_lun_mutex); +} + +static ssize_t core_tpg_rtpi_show(struct config_item *item, char *page) +{ + struct se_portal_group *se_tpg = attrib_to_tpg(item); + + return sprintf(page, "%#x\n", se_tpg->tpg_rtpi); +} + +static ssize_t core_tpg_rtpi_store(struct config_item *item, + const char *page, size_t count) +{ + struct se_portal_group *se_tpg = attrib_to_tpg(item); + struct se_portal_group *tpg; + bool rtpi_changed = false; + u16 val; + int ret; + + ret = kstrtou16(page, 0, &val); + if (ret < 0) + return ret; + if (val == 0) + return -EINVAL; + + /* RTPI shouldn't conflict with values of existing ports */ + spin_lock(&g_tpg_lock); + + list_for_each_entry(tpg, &g_tpg_list, tpg_list) { + if (se_tpg != tpg && val == tpg->tpg_rtpi) { + spin_unlock(&g_tpg_lock); + pr_err("TARGET_CORE[%s]->TPG[%u] - RTPI %#x conflicts with TARGET_CORE[%s]->TPG[%u]\n", + se_tpg->se_tpg_tfo->fabric_name, + se_tpg->se_tpg_tfo->tpg_get_tag(tpg), + val, + tpg->se_tpg_tfo->fabric_name, + tpg->se_tpg_tfo->tpg_get_tag(tpg)); + return -EINVAL; + } + } + + if (se_tpg->tpg_rtpi != val) { + se_tpg->tpg_rtpi = val; + rtpi_changed = true; + } + spin_unlock(&g_tpg_lock); + + if (rtpi_changed) + core_tpg_ua(se_tpg, 0x3f, ASCQ_3FH_INQUIRY_DATA_HAS_CHANGED); + ret = count; + + return ret; +} + +CONFIGFS_ATTR(core_tpg_, rtpi); + struct configfs_attribute *core_tpg_attrib_attrs[] = { + &core_tpg_attr_rtpi, NULL, }; From patchwork Thu Feb 25 12:18:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77941C19F2C for ; Wed, 3 Aug 2022 16:29:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236097AbiHCQ3j (ORCPT ); Wed, 3 Aug 2022 12:29:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238261AbiHCQ3X (ORCPT ); Wed, 3 Aug 2022 12:29:23 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5407541D32; Wed, 3 Aug 2022 09:29:20 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id B9E44412DD; Wed, 3 Aug 2022 16:29:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544157; x=1661358558; bh=F8zwbwFQerInqTJkM47gn2DFi lWIUw9B3wyqG8xTPDM=; b=HES+aTLitYbxxoXdsh206sV6bg2cjkXlOoFQO0ZA8 lrmBe0LOAk1/d53IgC9VCJtHNqe6c1h61GMCF0fOA2wSpR0zVLAyXRE/whdkfFKO pht+TL+aNLp9YX3t5Koi0F5T+2dwEcZqAr0967V4bqpLicAO5oAm/OyBaJfyFnDu SM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L-f30nbTlgaw; Wed, 3 Aug 2022 19:29:17 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id E625341239; Wed, 3 Aug 2022 19:29:13 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:14 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:12 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 09/48] target: core: check RTPI uniquity for enabled TPG Date: Thu, 25 Feb 2021 15:18:00 +0300 Message-ID: <20220803162857.27770-10-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Garantee uniquity of RTPI only for enabled target port groups. Allow any RPTI for disabled tpg until it is enabled. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_fabric_configfs.c | 29 +++++++++++++- drivers/target/target_core_internal.h | 2 + drivers/target/target_core_tpg.c | 40 +++++++++++++------- 3 files changed, 56 insertions(+), 15 deletions(-) diff --git a/drivers/target/target_core_fabric_configfs.c b/drivers/target/target_core_fabric_configfs.c index a34b5db4eec5..fc1b8f54fb54 100644 --- a/drivers/target/target_core_fabric_configfs.c +++ b/drivers/target/target_core_fabric_configfs.c @@ -857,6 +857,7 @@ static ssize_t target_fabric_tpg_base_enable_store(struct config_item *item, size_t count) { struct se_portal_group *se_tpg = to_tpg(item); + struct se_portal_group *tpg; int ret; bool op; @@ -867,11 +868,37 @@ static ssize_t target_fabric_tpg_base_enable_store(struct config_item *item, if (se_tpg->enabled == op) return count; + spin_lock(&g_tpg_lock); + + if (op) { + tpg = core_get_tpg_by_rtpi(se_tpg->tpg_rtpi); + if (tpg) { + spin_unlock(&g_tpg_lock); + + pr_err("TARGET_CORE[%s]->TPG[%u] - RTPI %#x conflicts with TARGET_CORE[%s]->TPG[%u]\n", + se_tpg->se_tpg_tfo->fabric_name, + se_tpg->se_tpg_tfo->tpg_get_tag(tpg), + se_tpg->tpg_rtpi, + tpg->se_tpg_tfo->fabric_name, + tpg->se_tpg_tfo->tpg_get_tag(tpg)); + return -EINVAL; + } + } + + se_tpg->enabled |= 0x10; /* transient state */ + spin_unlock(&g_tpg_lock); + ret = se_tpg->se_tpg_tfo->fabric_enable_tpg(se_tpg, op); - if (ret) + + spin_lock(&g_tpg_lock); + if (ret < 0) { + se_tpg->enabled &= ~0x10; /* clear transient state */ + spin_unlock(&g_tpg_lock); return ret; + } se_tpg->enabled = op; + spin_unlock(&g_tpg_lock); return count; } diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index d662cdc9a04c..d4ace697edb0 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -116,6 +116,7 @@ int core_tmr_lun_reset(struct se_device *, struct se_tmr_req *, struct list_head *, struct se_cmd *); /* target_core_tpg.c */ +extern struct spinlock g_tpg_lock; extern struct se_device *g_lun0_dev; extern struct configfs_attribute *core_tpg_attrib_attrs[]; @@ -131,6 +132,7 @@ void core_tpg_remove_lun(struct se_portal_group *, struct se_lun *); struct se_node_acl *core_tpg_add_initiator_node_acl(struct se_portal_group *tpg, const char *initiatorname); void core_tpg_del_initiator_node_acl(struct se_node_acl *acl); +struct se_portal_group *core_get_tpg_by_rtpi(u16 rtpi); /* target_core_transport.c */ extern struct kmem_cache *se_tmr_req_cache; diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index a409cf1c4ca0..5a81d592530f 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -34,7 +34,7 @@ extern struct se_device *g_lun0_dev; static u16 g_tpg_count; static u16 g_tpg_rtpi_counter = 1; static LIST_HEAD(g_tpg_list); -static DEFINE_SPINLOCK(g_tpg_lock); +DEFINE_SPINLOCK(g_tpg_lock); /* __core_tpg_get_initiator_node_acl(): * @@ -476,7 +476,7 @@ static int core_tpg_register_rtpi(struct se_portal_group *se_tpg) * Make sure RELATIVE TARGET PORT IDENTIFIER is unique * for 16-bit wrap.. */ - if (se_tpg->tpg_rtpi == tpg->tpg_rtpi) + if (tpg->enabled && se_tpg->tpg_rtpi == tpg->tpg_rtpi) goto again; } list_add(&se_tpg->tpg_list, &g_tpg_list); @@ -738,12 +738,26 @@ static ssize_t core_tpg_rtpi_show(struct config_item *item, char *page) return sprintf(page, "%#x\n", se_tpg->tpg_rtpi); } +struct se_portal_group * +core_get_tpg_by_rtpi(u16 rtpi) +{ + struct se_portal_group *tpg; + + lockdep_assert_held(&g_tpg_lock); + + list_for_each_entry(tpg, &g_tpg_list, tpg_list) { + if (tpg->enabled && rtpi == tpg->tpg_rtpi) + return tpg; + } + + return NULL; +} + static ssize_t core_tpg_rtpi_store(struct config_item *item, const char *page, size_t count) { struct se_portal_group *se_tpg = attrib_to_tpg(item); struct se_portal_group *tpg; - bool rtpi_changed = false; u16 val; int ret; @@ -753,11 +767,14 @@ static ssize_t core_tpg_rtpi_store(struct config_item *item, if (val == 0) return -EINVAL; - /* RTPI shouldn't conflict with values of existing ports */ + if (se_tpg->tpg_rtpi == val) + return count; + spin_lock(&g_tpg_lock); - list_for_each_entry(tpg, &g_tpg_list, tpg_list) { - if (se_tpg != tpg && val == tpg->tpg_rtpi) { + if (se_tpg->enabled) { + tpg = core_get_tpg_by_rtpi(val); + if (tpg) { spin_unlock(&g_tpg_lock); pr_err("TARGET_CORE[%s]->TPG[%u] - RTPI %#x conflicts with TARGET_CORE[%s]->TPG[%u]\n", se_tpg->se_tpg_tfo->fabric_name, @@ -769,17 +786,12 @@ static ssize_t core_tpg_rtpi_store(struct config_item *item, } } - if (se_tpg->tpg_rtpi != val) { - se_tpg->tpg_rtpi = val; - rtpi_changed = true; - } + se_tpg->tpg_rtpi = val; spin_unlock(&g_tpg_lock); - if (rtpi_changed) - core_tpg_ua(se_tpg, 0x3f, ASCQ_3FH_INQUIRY_DATA_HAS_CHANGED); - ret = count; + core_tpg_ua(se_tpg, 0x3f, ASCQ_3FH_INQUIRY_DATA_HAS_CHANGED); - return ret; + return count; } CONFIGFS_ATTR(core_tpg_, rtpi); From patchwork Thu Apr 30 14:16:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1734BC19F28 for ; Wed, 3 Aug 2022 16:29:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238383AbiHCQ3g (ORCPT ); Wed, 3 Aug 2022 12:29:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235210AbiHCQ3Y (ORCPT ); Wed, 3 Aug 2022 12:29:24 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4CB443E59; Wed, 3 Aug 2022 09:29:20 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 7992F412CC; Wed, 3 Aug 2022 16:29:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544158; x=1661358559; bh=nHf1+JiaUlppLZuGjnVcLvxaH EGVObBWMJ9YhuHNlac=; b=R+Hkl2vTFgIlHjOY+i3/cSnQ7obMPAFu6yngWfLU0 rDHhTfN0TodtEYaCWJpyV/eTA22vo9QL1I/7BlMm2NExOfgiZU5KcIB1zeum6TRs XAyZ0Z0bHTLJz+35DjkO7+GAb7hT8yBhGo1Xf+Tqs3DInt1beuWYOjUuBnCuNc64 1k= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XTGavMJdcEV5; Wed, 3 Aug 2022 19:29:18 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 3F458412C9; Wed, 3 Aug 2022 19:29:14 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:14 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:13 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Konstantin Shelekhin , Dmitry Bogdanov Subject: [RFC PATCH 10/48] scsi: target/core: Unlock PR generation bump Date: Thu, 30 Apr 2020 17:16:53 +0300 Message-ID: <20220803162857.27770-11-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org From: Konstantin Shelekhin Get rid of spinlock and bump PR generation atomically. This is required to prevent deadlock later on. Signed-off-by: Konstantin Shelekhin Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 3 ++- drivers/target/target_core_pr.c | 25 ++++++------------------- include/target/target_core_base.h | 2 +- 3 files changed, 9 insertions(+), 21 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 40b9808738d2..610de251994e 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1814,7 +1814,8 @@ static ssize_t target_pr_res_pr_all_tgt_pts_show(struct config_item *item, static ssize_t target_pr_res_pr_generation_show(struct config_item *item, char *page) { - return sprintf(page, "0x%08x\n", pr_to_dev(item)->t10_pr.pr_generation); + return sprintf(page, "0x%08x\n", + atomic_read(&pr_to_dev(item)->t10_pr.pr_generation)); } diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 668670b1414a..8f3d920d2590 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -594,8 +594,6 @@ target_scsi3_pr_reservation_check(struct se_cmd *cmd) static u32 core_scsi3_pr_generation(struct se_device *dev) { - u32 prg; - /* * PRGeneration field shall contain the value of a 32-bit wrapping * counter mainted by the device server. @@ -605,11 +603,7 @@ static u32 core_scsi3_pr_generation(struct se_device *dev) * * See spc4r17 section 6.3.12 READ_KEYS service action */ - spin_lock(&dev->dev_reservation_lock); - prg = dev->t10_pr.pr_generation++; - spin_unlock(&dev->dev_reservation_lock); - - return prg; + return atomic_inc_return(&dev->t10_pr.pr_generation); } static struct t10_pr_registration *__core_scsi3_do_alloc_registration( @@ -1057,15 +1051,8 @@ static void __core_scsi3_add_registration( /* * Increment PRgeneration counter for struct se_device upon a successful * REGISTER, see spc4r17 section 6.3.2 READ_KEYS service action - * - * Also, when register_move = 1 for PROUT REGISTER_AND_MOVE service - * action, the struct se_device->dev_reservation_lock will already be held, - * so we do not call core_scsi3_pr_generation() which grabs the lock - * for the REGISTER. */ - pr_reg->pr_res_generation = (register_move) ? - dev->t10_pr.pr_generation++ : - core_scsi3_pr_generation(dev); + pr_reg->pr_res_generation = core_scsi3_pr_generation(dev); spin_lock(&pr_tmpl->registration_lock); list_add_tail(&pr_reg->pr_reg_list, &pr_tmpl->registration_list); @@ -3481,7 +3468,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, * Increment PRGeneration for existing registrations.. */ if (!new_reg) - dest_pr_reg->pr_res_generation = pr_tmpl->pr_generation++; + dest_pr_reg->pr_res_generation = core_scsi3_pr_generation(dev); spin_unlock(&dev->dev_reservation_lock); pr_debug("SPC-3 PR [%s] Service Action: REGISTER_AND_MOVE" @@ -3703,7 +3690,7 @@ core_scsi3_pri_read_keys(struct se_cmd *cmd) if (!buf) return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; - put_unaligned_be32(dev->t10_pr.pr_generation, buf); + put_unaligned_be32(atomic_read(&dev->t10_pr.pr_generation), buf); spin_lock(&dev->t10_pr.registration_lock); list_for_each_entry(pr_reg, &dev->t10_pr.registration_list, @@ -3758,7 +3745,7 @@ core_scsi3_pri_read_reservation(struct se_cmd *cmd) if (!buf) return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; - put_unaligned_be32(dev->t10_pr.pr_generation, &buf[0]); + put_unaligned_be32(atomic_read(&dev->t10_pr.pr_generation), &buf[0]); spin_lock(&dev->dev_reservation_lock); pr_reg = dev->dev_pr_res_holder; @@ -3900,7 +3887,7 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) if (!buf) return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; - put_unaligned_be32(dev->t10_pr.pr_generation, &buf[0]); + put_unaligned_be32(atomic_read(&dev->t10_pr.pr_generation), &buf[0]); spin_lock(&dev->dev_reservation_lock); if (dev->dev_pr_res_holder) { diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 6ed484683ab6..7e13db647faa 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -385,7 +385,7 @@ struct t10_reservation { * for SCSI device */ int pr_aptpl_active; #define PR_APTPL_BUF_LEN 262144 - u32 pr_generation; + atomic_t pr_generation; spinlock_t registration_lock; spinlock_t aptpl_reg_lock; /* From patchwork Fri Mar 4 10:38:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595470 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F7DFC19F2D for ; Wed, 3 Aug 2022 16:29:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238349AbiHCQ3m (ORCPT ); Wed, 3 Aug 2022 12:29:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238262AbiHCQ3Y (ORCPT ); Wed, 3 Aug 2022 12:29:24 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 333EF2181E; Wed, 3 Aug 2022 09:29:21 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id C3DF3412DF; Wed, 3 Aug 2022 16:29:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544158; x=1661358559; bh=cPS+Mb67ptHTuG8erQcA1iuBz UlO5/SOtnFYXDOKr1c=; b=EC6jalnwN0hspbq2oI5FrzQHeucqBezph24dNJJWQ pRJQ/4ahDIyyjDxxRnQXMFG8RbBzY9e29hGRB0tUrC0UJC5nChxaSB1wpM48MXg9 /4/ufF39fq0zPYWsQjSvM9qJzjIdU2BkExFP28lrxqCClAaZk+7y5eQpsE2rB+aI R0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lv7QzbJ-8Sxh; Wed, 3 Aug 2022 19:29:18 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 9511041283; Wed, 3 Aug 2022 19:29:14 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:14 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:13 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 11/48] target: core: fix preempt and abort for allreg res Date: Fri, 4 Mar 2022 13:38:07 +0300 Message-ID: <20220803162857.27770-12-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Match a key only if SARK is not zero according to SPC-4 and the comment above the code: If an all registrants persistent reservation is present and the SERVICE ACTION RESERVATION KEY field is set to zero, then all registrations shall be removed except for that of the I_T nexus that is being used for the PERSISTENT RESERVE OUT command; Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 8f3d920d2590..c278ff721da3 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -3020,7 +3020,7 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, if (calling_it_nexus) continue; - if (pr_reg->pr_res_key != sa_res_key) + if ((sa_res_key) && (pr_reg->pr_res_key != sa_res_key)) continue; pr_reg_nacl = pr_reg->pr_reg_nacl; From patchwork Wed Mar 2 07:13:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24685C19F29 for ; Wed, 3 Aug 2022 16:29:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238378AbiHCQ3f (ORCPT ); Wed, 3 Aug 2022 12:29:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233932AbiHCQ3d (ORCPT ); Wed, 3 Aug 2022 12:29:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E422F459B6; Wed, 3 Aug 2022 09:29:21 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 864AD412E3; Wed, 3 Aug 2022 16:29:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544159; x=1661358560; bh=gV3Bft/U4CnMbfHDO5HaUCMRP MMl6olr1lhlIr5XPZE=; b=JzR5lTOeThtoqKgYVPFN2qR3rNv7v1TBIAGcwScGm r/Ru7FSQ6tfnkBUWPuOX2SLB7V/pbnNbyO2OQa27lDObQ2VlucgHZsW+3qWUw3iu m+A2rIa8p8M/RBGndVf/5y8dfTTjzC0o8b7kAp4mHg4bfjbfEPTxspCiLtkn+PTD Pw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Uyb7NvTxePDZ; Wed, 3 Aug 2022 19:29:19 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id DFD8141284; Wed, 3 Aug 2022 19:29:14 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:14 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:14 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 12/48] target: core: fix memory leak in preempt_and_abort Date: Wed, 2 Mar 2022 10:13:09 +0300 Message-ID: <20220803162857.27770-13-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Always release preempt_and_abort_list to avoid memory leak of t10_pr_registration objects in it. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index c278ff721da3..0b3a22f51c5e 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -2954,11 +2954,12 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, __core_scsi3_complete_pro_preempt(dev, pr_reg_n, (preempt_type == PREEMPT_AND_ABORT) ? &preempt_and_abort_list : NULL, type, scope, preempt_type); - - if (preempt_type == PREEMPT_AND_ABORT) - core_scsi3_release_preempt_and_abort( - &preempt_and_abort_list, pr_reg_n); } + + if (preempt_type == PREEMPT_AND_ABORT) + core_scsi3_release_preempt_and_abort( + &preempt_and_abort_list, pr_reg_n); + spin_unlock(&dev->dev_reservation_lock); if (pr_tmpl->pr_aptpl_active) From patchwork Fri Mar 4 10:44:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC742C3F6B0 for ; Wed, 3 Aug 2022 16:29:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238363AbiHCQ3o (ORCPT ); Wed, 3 Aug 2022 12:29:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236336AbiHCQ3d (ORCPT ); Wed, 3 Aug 2022 12:29:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D88346D9A; Wed, 3 Aug 2022 09:29:22 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id E6487412E2; Wed, 3 Aug 2022 16:29:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544159; x=1661358560; bh=IMSXmV/3YvcEX2ISgaseaYdnq ViOOm6e8B0STsuY//U=; b=btlmbsEGR1xYMw9PwR4oZ+HgBPg/ZYJh+Fheql6xD atpqhY5oZTjF3sX4sXKUR8UxvGYG5XQtDcoJ856GHXyYp8RDY5iZtpO68MpYL3yR sHmY2EIULqmbID04+HHWLlyyKXLJERgjr/Kp0UXh5TWHVXodE9O13eGm1/MouBTK c8= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K7OxfYh3tbHs; Wed, 3 Aug 2022 19:29:19 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 3141C412CD; Wed, 3 Aug 2022 19:29:15 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:15 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:14 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 13/48] target: core: abort all preempted regs if requested Date: Fri, 4 Mar 2022 13:44:33 +0300 Message-ID: <20220803162857.27770-14-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org According to SPC the aborting shall always happened. SPC-4: 5.12.11.2.6 Preempting and aborting If the SERVICE ACTION RESERVATION KEY field is set to zero and an all registrants persistent reservation is present, the device server shall abort all commands for all registered I_T nexuses. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 0b3a22f51c5e..b943f8f379cf 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -2956,9 +2956,23 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, type, scope, preempt_type); } - if (preempt_type == PREEMPT_AND_ABORT) + /* + * SPC-4 5.12.11.2.6 Preempting and aborting + * The actions described in this subclause shall be performed + * for all I_T nexuses that are registered with the non-zero + * SERVICE ACTION RESERVATION KEY value, without regard for + * whether the preempted I_T nexuses hold the persistent + * reservation. If the SERVICE ACTION RESERVATION KEY field is + * set to zero and an all registrants persistent reservation is + * present, the device server shall abort all commands for all + * registered I_T nexuses. + */ + if (preempt_type == PREEMPT_AND_ABORT) { + core_tmr_lun_reset(dev, NULL, &preempt_and_abort_list, + cmd); core_scsi3_release_preempt_and_abort( &preempt_and_abort_list, pr_reg_n); + } spin_unlock(&dev->dev_reservation_lock); From patchwork Mon Dec 6 10:56:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB4B9C25B06 for ; Wed, 3 Aug 2022 16:29:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238375AbiHCQ3p (ORCPT ); Wed, 3 Aug 2022 12:29:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238281AbiHCQ3d (ORCPT ); Wed, 3 Aug 2022 12:29:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09260474F4; Wed, 3 Aug 2022 09:29:23 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9FDEF412E6; Wed, 3 Aug 2022 16:29:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544160; x=1661358561; bh=mTFyth72aJekuzC4/KtI7crQh GNVVupsPSsf1cnXZMo=; b=bXbx1Cm9LvfdRKBc9sTBS1xu9dG2qF4cX9p/frhiZ N3ND0G6zGjx/9wOMttnRLMFVjgvGEhZXVC/Cd+B8Tclb0QATAku0XcfuE0x29wsd OkbJ6oTihaExhDP/oQfXPnBKgt/TixS4ZwklIjETjltgQ5FfYKTF9/tP9me8saG/ uU= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YrG1w72k9eqd; Wed, 3 Aug 2022 19:29:20 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 77D64412D0; Wed, 3 Aug 2022 19:29:15 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:15 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:14 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 14/48] target: core: new key must be used for moved PR Date: Mon, 6 Dec 2021 13:56:28 +0300 Message-ID: <20220803162857.27770-15-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org According to SPC4 5.12.8: e) Retain the reservation key specified in the SERVICE ACTION RESERVATION KEY field and associated information; But currently sa_res_key is only used for the not existing I_T nexus. The patch adds a changing of the key for the existing I_T nexus the PR moved to. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index b943f8f379cf..844b971827e6 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -3438,8 +3438,6 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, * transport protocols where port names are not required; * d) Register the reservation key specified in the SERVICE ACTION * RESERVATION KEY field; - * e) Retain the reservation key specified in the SERVICE ACTION - * RESERVATION KEY field and associated information; * * Also, It is not an error for a REGISTER AND MOVE service action to * register an I_T nexus that is already registered with the same @@ -3462,6 +3460,12 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, dest_pr_reg = __core_scsi3_locate_pr_reg(dev, dest_node_acl, iport_ptr); new_reg = 1; + } else { + /* + * e) Retain the reservation key specified in the SERVICE ACTION + * RESERVATION KEY field and associated information; + */ + dest_pr_reg->pr_res_key = sa_res_key; } /* * f) Release the persistent reservation for the persistent reservation From patchwork Wed Dec 1 15:42:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDB79C25B08 for ; Wed, 3 Aug 2022 16:29:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237358AbiHCQ3k (ORCPT ); Wed, 3 Aug 2022 12:29:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238277AbiHCQ3d (ORCPT ); Wed, 3 Aug 2022 12:29:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C19F459B1; Wed, 3 Aug 2022 09:29:23 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 110BC412E7; Wed, 3 Aug 2022 16:29:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544160; x=1661358561; bh=QveIwksVWIhConjr1nbizK7pq av5dltAECoHVmy3pb8=; b=fsnZunNt6fU312rkfEvoZPuzP+upYa+jFO5PWxaLP awH2qPJIDcXiPxVhtDllMDvGzimwY2L7YhbtDZkEPRU7NX94KmhBRIW+zxO47Chc GvD8+fhLOt5ys81jorn7aBGTlCkiVn2XK+rP4IN0EItms680LqTZ9GQ/0USSkMce XA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V-KO2i5Rnnbk; Wed, 3 Aug 2022 19:29:20 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id CDF0B41287; Wed, 3 Aug 2022 19:29:15 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:15 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:15 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 15/48] target: core: remove unused variable in se_dev_entry Date: Wed, 1 Dec 2021 18:42:40 +0300 Message-ID: <20220803162857.27770-16-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org deve->pr_res_key is never set, and se_cmd->pr_res_key is set in the checks inside __target_execute_cmd Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_device.c | 2 -- include/target/target_core_base.h | 4 ---- 2 files changed, 6 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 40a582b084d7..c0e4fb7b95d6 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -83,7 +83,6 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd) } se_cmd->se_lun = se_lun; - se_cmd->pr_res_key = deve->pr_res_key; se_cmd->se_cmd_flags |= SCF_SE_LUN_CMD; se_cmd->lun_ref_active = true; } @@ -160,7 +159,6 @@ int transport_lookup_tmr_lun(struct se_cmd *se_cmd) } se_cmd->se_lun = se_lun; - se_cmd->pr_res_key = deve->pr_res_key; se_cmd->se_cmd_flags |= SCF_SE_LUN_CMD; se_cmd->lun_ref_active = true; } diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 7e13db647faa..91c068525c02 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -379,8 +379,6 @@ struct t10_pr_registration { }; struct t10_reservation { - /* Reservation effects all target ports */ - int pr_all_tg_pt; /* Activate Persistence across Target Power Loss enabled * for SCSI device */ int pr_aptpl_active; @@ -398,7 +396,6 @@ struct t10_reservation { * a single *pr_res_holder of the reservation, but all * registrations are considered reservation holders. */ - struct se_node_acl *pr_res_holder; struct list_head registration_list; struct list_head aptpl_reg_list; }; @@ -656,7 +653,6 @@ struct se_lun_acl { struct se_dev_entry { u64 mapped_lun; - u64 pr_res_key; u64 creation_time; bool lun_access_ro; u32 attach_count; From patchwork Fri Dec 24 09:45:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC377C19F29 for ; Wed, 3 Aug 2022 16:29:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238444AbiHCQ3t (ORCPT ); Wed, 3 Aug 2022 12:29:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238315AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46AB447B9A; Wed, 3 Aug 2022 09:29:26 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id EE1CE412FB; Wed, 3 Aug 2022 16:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544161; x=1661358562; bh=2DK1C7WLPz8HKq9ML7gZnd9y2 UWaKq7ryGTobmH+KRc=; b=ZKiUaaCWkBxG1LL/e7CBZj0i0oNFZU3O72m2oQn2r 0w2Ld+PPj/yRwc1g5oJhps+KZxZGQDHG/1/f118MAdcJYYeMYWbTgDk4rR3pTRHS Qhk2YdZ+k/Xl0aoDpZgM651wvMsUP5HDb1htEUl85xfF9hOeNTT1yHpBJIaLZREt WU= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FjStNOJsWKGH; Wed, 3 Aug 2022 19:29:21 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 48FAC412D2; Wed, 3 Aug 2022 19:29:16 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:16 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:15 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 16/48] target: core: undepend PR registrant of nacl Date: Fri, 24 Dec 2021 12:45:48 +0300 Message-ID: <20220803162857.27770-17-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Make registration info be independent of nacl. Store and use an information about initiatror name and target port inside registration object. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 14 +- drivers/target/target_core_device.c | 7 - drivers/target/target_core_fabric_lib.c | 50 ++++---- drivers/target/target_core_internal.h | 9 +- drivers/target/target_core_pr.c | 164 ++++++++---------------- drivers/target/target_core_pr.h | 2 - include/target/target_core_base.h | 1 + 7 files changed, 88 insertions(+), 159 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 610de251994e..6e654edc4399 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1735,7 +1735,6 @@ static struct se_device *pr_to_dev(struct config_item *item) static ssize_t target_core_dev_pr_show_spc3_res(struct se_device *dev, char *page) { - struct se_node_acl *se_nacl; struct t10_pr_registration *pr_reg; char i_buf[PR_REG_ISID_ID_LEN] = { }; @@ -1743,12 +1742,11 @@ static ssize_t target_core_dev_pr_show_spc3_res(struct se_device *dev, if (!pr_reg) return sprintf(page, "No SPC-3 Reservation holder\n"); - se_nacl = pr_reg->pr_reg_nacl; core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); return sprintf(page, "SPC-3 Reservation: %s Initiator: %s%s\n", - se_nacl->se_tpg->se_tpg_tfo->fabric_name, - se_nacl->initiatorname, i_buf); + pr_reg->se_tpg->se_tpg_tfo->fabric_name, + pr_reg->pr_iport, i_buf); } static ssize_t target_core_dev_pr_show_spc2_res(struct se_device *dev, @@ -1823,7 +1821,6 @@ static ssize_t target_pr_res_pr_holder_tg_port_show(struct config_item *item, char *page) { struct se_device *dev = pr_to_dev(item); - struct se_node_acl *se_nacl; struct se_portal_group *se_tpg; struct t10_pr_registration *pr_reg; const struct target_core_fabric_ops *tfo; @@ -1836,8 +1833,7 @@ static ssize_t target_pr_res_pr_holder_tg_port_show(struct config_item *item, goto out_unlock; } - se_nacl = pr_reg->pr_reg_nacl; - se_tpg = se_nacl->se_tpg; + se_tpg = pr_reg->se_tpg; tfo = se_tpg->se_tpg_tfo; len += sprintf(page+len, "SPC-3 Reservation: %s" @@ -1874,12 +1870,12 @@ static ssize_t target_pr_res_pr_registered_i_pts_show(struct config_item *item, memset(buf, 0, 384); memset(i_buf, 0, PR_REG_ISID_ID_LEN); - tfo = pr_reg->pr_reg_nacl->se_tpg->se_tpg_tfo; + tfo = pr_reg->se_tpg->se_tpg_tfo; core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); sprintf(buf, "%s Node: %s%s Key: 0x%016Lx PRgen: 0x%08x\n", tfo->fabric_name, - pr_reg->pr_reg_nacl->initiatorname, i_buf, pr_reg->pr_res_key, + pr_reg->pr_iport, i_buf, pr_reg->pr_res_key, pr_reg->pr_res_generation); if (len + strlen(buf) >= PAGE_SIZE) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index c0e4fb7b95d6..a4c31f3112ce 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -391,12 +391,6 @@ void core_disable_device_list_for_node( struct se_node_acl *nacl, struct se_portal_group *tpg) { - /* - * rcu_dereference_raw protected by se_lun->lun_group symlink - * reference to se_device->dev_group. - */ - struct se_device *dev = rcu_dereference_raw(lun->lun_se_dev); - lockdep_assert_held(&nacl->lun_entry_mutex); /* @@ -437,7 +431,6 @@ void core_disable_device_list_for_node( kfree_rcu(orig, rcu_head); - core_scsi3_free_pr_reg_from_nacl(dev, nacl); target_luns_data_has_changed(nacl, NULL, false); } diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 6600ae44f29d..cd1360095036 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -33,14 +33,14 @@ static int sas_get_pr_transport_id( - struct se_node_acl *nacl, + struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) { int ret; /* Skip over 'naa. prefix */ - ret = hex2bin(&buf[4], &nacl->initiatorname[4], 8); + ret = hex2bin(&buf[4], &pr_reg->pr_iport[4], 8); if (ret) { pr_debug("%s: invalid hex string\n", __func__); return ret; @@ -50,7 +50,7 @@ static int sas_get_pr_transport_id( } static int fc_get_pr_transport_id( - struct se_node_acl *se_nacl, + struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) { @@ -62,7 +62,7 @@ static int fc_get_pr_transport_id( * We convert the ASCII formatted N Port name into a binary * encoded TransportID. */ - ptr = &se_nacl->initiatorname[0]; + ptr = &pr_reg->pr_iport[0]; for (i = 0; i < 23; ) { if (!strncmp(&ptr[i], ":", 1)) { i++; @@ -82,13 +82,13 @@ static int fc_get_pr_transport_id( } static int sbp_get_pr_transport_id( - struct se_node_acl *nacl, + struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) { int ret; - ret = hex2bin(&buf[8], nacl->initiatorname, 8); + ret = hex2bin(&buf[8], pr_reg->pr_iport, 8); if (ret) { pr_debug("%s: invalid hex string\n", __func__); return ret; @@ -98,7 +98,7 @@ static int sbp_get_pr_transport_id( } static int srp_get_pr_transport_id( - struct se_node_acl *nacl, + struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) { @@ -106,7 +106,7 @@ static int srp_get_pr_transport_id( unsigned len, count, leading_zero_bytes; int rc; - p = nacl->initiatorname; + p = pr_reg->pr_iport; if (strncasecmp(p, "0x", 2) == 0) p += 2; len = strlen(p); @@ -126,7 +126,6 @@ static int srp_get_pr_transport_id( } static int iscsi_get_pr_transport_id( - struct se_node_acl *se_nacl, struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) @@ -135,7 +134,6 @@ static int iscsi_get_pr_transport_id( int isid_len; u16 len = 0; - spin_lock_irq(&se_nacl->nacl_sess_lock); /* * Only null terminate the last field. * @@ -151,7 +149,7 @@ static int iscsi_get_pr_transport_id( * length of the iSCSI TransportID or the contents of the ADDITIONAL * LENGTH field. */ - len = sprintf(&buf[off], "%s", se_nacl->initiatorname); + len = sprintf(&buf[off], "%s", pr_reg->pr_iport); off += len; if ((*format_code == 1) && (pr_reg->isid_present_at_reg)) { /* @@ -193,7 +191,6 @@ static int iscsi_get_pr_transport_id( } buf[off] = '\0'; len += 1; - spin_unlock_irq(&se_nacl->nacl_sess_lock); /* * The ADDITIONAL LENGTH field specifies the number of bytes that follow * in the TransportID. The additional length shall be at least 20 and @@ -214,14 +211,12 @@ static int iscsi_get_pr_transport_id( } static int iscsi_get_pr_transport_id_len( - struct se_node_acl *se_nacl, struct t10_pr_registration *pr_reg, int *format_code) { u32 len = 0, padding = 0; - spin_lock_irq(&se_nacl->nacl_sess_lock); - len = strlen(se_nacl->initiatorname); + len = strlen(pr_reg->pr_iport); /* * Add extra byte for NULL terminator */ @@ -239,7 +234,6 @@ static int iscsi_get_pr_transport_id_len( *format_code = 1; } else *format_code = 0; - spin_unlock_irq(&se_nacl->nacl_sess_lock); /* * The ADDITIONAL LENGTH field specifies the number of bytes that follow * in the TransportID. The additional length shall be at least 20 and @@ -342,19 +336,19 @@ static char *iscsi_parse_pr_out_transport_id( return &buf[4]; } -int target_get_pr_transport_id_len(struct se_node_acl *nacl, +int target_get_pr_transport_id_len( struct t10_pr_registration *pr_reg, int *format_code) { - switch (nacl->se_tpg->proto_id) { + switch (pr_reg->se_tpg->proto_id) { case SCSI_PROTOCOL_FCP: case SCSI_PROTOCOL_SBP: case SCSI_PROTOCOL_SRP: case SCSI_PROTOCOL_SAS: break; case SCSI_PROTOCOL_ISCSI: - return iscsi_get_pr_transport_id_len(nacl, pr_reg, format_code); + return iscsi_get_pr_transport_id_len(pr_reg, format_code); default: - pr_err("Unknown proto_id: 0x%02x\n", nacl->se_tpg->proto_id); + pr_err("Unknown proto_id: 0x%02x\n", pr_reg->se_tpg->proto_id); return -EINVAL; } @@ -365,24 +359,24 @@ int target_get_pr_transport_id_len(struct se_node_acl *nacl, return 24; } -int target_get_pr_transport_id(struct se_node_acl *nacl, +int target_get_pr_transport_id( struct t10_pr_registration *pr_reg, int *format_code, unsigned char *buf) { - switch (nacl->se_tpg->proto_id) { + switch (pr_reg->se_tpg->proto_id) { case SCSI_PROTOCOL_SAS: - return sas_get_pr_transport_id(nacl, format_code, buf); + return sas_get_pr_transport_id(pr_reg, format_code, buf); case SCSI_PROTOCOL_SBP: - return sbp_get_pr_transport_id(nacl, format_code, buf); + return sbp_get_pr_transport_id(pr_reg, format_code, buf); case SCSI_PROTOCOL_SRP: - return srp_get_pr_transport_id(nacl, format_code, buf); + return srp_get_pr_transport_id(pr_reg, format_code, buf); case SCSI_PROTOCOL_FCP: - return fc_get_pr_transport_id(nacl, format_code, buf); + return fc_get_pr_transport_id(pr_reg, format_code, buf); case SCSI_PROTOCOL_ISCSI: - return iscsi_get_pr_transport_id(nacl, pr_reg, format_code, + return iscsi_get_pr_transport_id(pr_reg, format_code, buf); default: - pr_err("Unknown proto_id: 0x%02x\n", nacl->se_tpg->proto_id); + pr_err("Unknown proto_id: 0x%02x\n", pr_reg->se_tpg->proto_id); return -EINVAL; } } diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index d4ace697edb0..8bd48eb1cf9b 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -97,11 +97,10 @@ void target_setup_backend_cits(struct target_backend *); int target_fabric_setup_cits(struct target_fabric_configfs *); /* target_core_fabric_lib.c */ -int target_get_pr_transport_id_len(struct se_node_acl *nacl, - struct t10_pr_registration *pr_reg, int *format_code); -int target_get_pr_transport_id(struct se_node_acl *nacl, - struct t10_pr_registration *pr_reg, int *format_code, - unsigned char *buf); +int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg, + int *format_code); +int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, + int *format_code, unsigned char *buf); const char *target_parse_pr_out_transport_id(struct se_portal_group *tpg, char *buf, u32 *out_tid_len, char **port_nexus_ptr); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 844b971827e6..aa967ea74e5b 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -64,7 +64,7 @@ enum preempt_type { PREEMPT_AND_ABORT, }; -static void __core_scsi3_complete_pro_release(struct se_device *, struct se_node_acl *, +static void __core_scsi3_complete_pro_release(struct se_device *, struct t10_pr_registration *, int, int); static int is_reservation_holder( @@ -609,7 +609,9 @@ static u32 core_scsi3_pr_generation(struct se_device *dev) static struct t10_pr_registration *__core_scsi3_do_alloc_registration( struct se_device *dev, struct se_node_acl *nacl, - struct se_lun *lun, + const char *initiatorname, + u64 unpacked_lun, + struct se_portal_group *se_tpg, struct se_dev_entry *dest_deve, u64 mapped_lun, unsigned char *isid, @@ -632,6 +634,7 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration( INIT_LIST_HEAD(&pr_reg->pr_reg_atp_mem_list); atomic_set(&pr_reg->pr_res_holders, 0); pr_reg->pr_reg_nacl = nacl; + strscpy(pr_reg->pr_iport, initiatorname, PR_APTPL_MAX_IPORT_LEN); /* * For destination registrations for ALL_TG_PT=1 and SPEC_I_PT=1, * the se_dev_entry->pr_ref will have been already obtained by @@ -656,8 +659,9 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration( rcu_read_unlock(); } pr_reg->pr_res_mapped_lun = mapped_lun; - pr_reg->pr_aptpl_target_lun = lun->unpacked_lun; - pr_reg->tg_pt_sep_rtpi = lun->lun_tpg->tpg_rtpi; + pr_reg->pr_aptpl_target_lun = unpacked_lun; + pr_reg->tg_pt_sep_rtpi = se_tpg->tpg_rtpi; + pr_reg->se_tpg = se_tpg; pr_reg->pr_res_key = sa_res_key; pr_reg->pr_reg_all_tg_pt = all_tg_pt; pr_reg->pr_reg_aptpl = aptpl; @@ -703,7 +707,10 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( * Create a registration for the I_T Nexus upon which the * PROUT REGISTER was received. */ - pr_reg = __core_scsi3_do_alloc_registration(dev, nacl, lun, deve, mapped_lun, + pr_reg = __core_scsi3_do_alloc_registration(dev, nacl, + nacl->initiatorname, lun->unpacked_lun, + lun->lun_tpg, + deve, mapped_lun, isid, sa_res_key, all_tg_pt, aptpl); if (!pr_reg) @@ -782,7 +789,11 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( kref_read(&deve_tmp->pr_kref) != 0); pr_reg_atp = __core_scsi3_do_alloc_registration(dev, - nacl_tmp, dest_lun, deve_tmp, + nacl_tmp, + nacl_tmp->initiatorname, + dest_lun->unpacked_lun, + lun->lun_tpg, + deve_tmp, deve_tmp->mapped_lun, NULL, sa_res_key, all_tg_pt, aptpl); if (!pr_reg_atp) { @@ -1009,11 +1020,10 @@ int core_scsi3_check_aptpl_registration( static void __core_scsi3_dump_registration( const struct target_core_fabric_ops *tfo, struct se_device *dev, - struct se_node_acl *nacl, struct t10_pr_registration *pr_reg, enum register_type register_type) { - struct se_portal_group *se_tpg = nacl->se_tpg; + struct se_portal_group *se_tpg = pr_reg->se_tpg; char i_buf[PR_REG_ISID_ID_LEN] = { }; core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); @@ -1021,7 +1031,7 @@ static void __core_scsi3_dump_registration( pr_debug("SPC-3 PR [%s] Service Action: REGISTER%s Initiator" " Node: %s%s\n", tfo->fabric_name, (register_type == REGISTER_AND_MOVE) ? "_AND_MOVE" : (register_type == REGISTER_AND_IGNORE_EXISTING_KEY) ? - "_AND_IGNORE_EXISTING_KEY" : "", nacl->initiatorname, + "_AND_IGNORE_EXISTING_KEY" : "", pr_reg->pr_iport, i_buf); pr_debug("SPC-3 PR [%s] registration on Target Port: %s,0x%04x\n", tfo->fabric_name, tfo->tpg_get_wwn(se_tpg), @@ -1057,7 +1067,7 @@ static void __core_scsi3_add_registration( spin_lock(&pr_tmpl->registration_lock); list_add_tail(&pr_reg->pr_reg_list, &pr_tmpl->registration_list); - __core_scsi3_dump_registration(tfo, dev, nacl, pr_reg, register_type); + __core_scsi3_dump_registration(tfo, dev, pr_reg, register_type); spin_unlock(&pr_tmpl->registration_lock); /* * Skip extra processing for ALL_TG_PT=0 or REGISTER_AND_MOVE. @@ -1070,8 +1080,6 @@ static void __core_scsi3_add_registration( */ list_for_each_entry_safe(pr_reg_tmp, pr_reg_tmp_safe, &pr_reg->pr_reg_atp_list, pr_reg_atp_mem_list) { - struct se_node_acl *nacl_tmp = pr_reg_tmp->pr_reg_nacl; - list_del(&pr_reg_tmp->pr_reg_atp_mem_list); pr_reg_tmp->pr_res_generation = core_scsi3_pr_generation(dev); @@ -1080,7 +1088,7 @@ static void __core_scsi3_add_registration( list_add_tail(&pr_reg_tmp->pr_reg_list, &pr_tmpl->registration_list); - __core_scsi3_dump_registration(tfo, dev, nacl_tmp, pr_reg_tmp, + __core_scsi3_dump_registration(tfo, dev, pr_reg_tmp, register_type); spin_unlock(&pr_tmpl->registration_lock); /* @@ -1208,7 +1216,6 @@ static int core_scsi3_check_implicit_release( struct se_device *dev, struct t10_pr_registration *pr_reg) { - struct se_node_acl *nacl = pr_reg->pr_reg_nacl; struct t10_pr_registration *pr_res_holder; int ret = 0; @@ -1232,7 +1239,7 @@ static int core_scsi3_check_implicit_release( * service action with the SERVICE ACTION RESERVATION KEY * field set to zero (see 5.7.11.3). */ - __core_scsi3_complete_pro_release(dev, nacl, pr_reg, 0, 1); + __core_scsi3_complete_pro_release(dev, pr_reg, 0, 1); ret = 1; /* * For 'All Registrants' reservation types, all existing @@ -1241,8 +1248,7 @@ static int core_scsi3_check_implicit_release( * reservation holder is implicitly released here. */ } else if (pr_reg->pr_reg_all_tg_pt && - (!strcmp(pr_res_holder->pr_reg_nacl->initiatorname, - pr_reg->pr_reg_nacl->initiatorname)) && + (!strcmp(pr_res_holder->pr_iport, pr_reg->pr_iport)) && (pr_res_holder->pr_res_key == pr_reg->pr_res_key)) { pr_err("SPC-3 PR: Unable to perform ALL_TG_PT=1" " UNREGISTER while existing reservation with matching" @@ -1264,7 +1270,7 @@ static void __core_scsi3_free_registration( __acquires(&pr_tmpl->registration_lock) { const struct target_core_fabric_ops *tfo = - pr_reg->pr_reg_nacl->se_tpg->se_tpg_tfo; + pr_reg->se_tpg->se_tpg_tfo; struct t10_reservation *pr_tmpl = &dev->t10_pr; struct se_node_acl *nacl = pr_reg->pr_reg_nacl; struct se_dev_entry *deve; @@ -1305,8 +1311,7 @@ static void __core_scsi3_free_registration( spin_lock(&pr_tmpl->registration_lock); pr_debug("SPC-3 PR [%s] Service Action: UNREGISTER Initiator" " Node: %s%s\n", tfo->fabric_name, - pr_reg->pr_reg_nacl->initiatorname, - i_buf); + pr_reg->pr_iport, i_buf); pr_debug("SPC-3 PR [%s] for %s TCM Subsystem %s Object Target" " Port(s)\n", tfo->fabric_name, (pr_reg->pr_reg_all_tg_pt) ? "ALL" : "SINGLE", @@ -1328,43 +1333,6 @@ static void __core_scsi3_free_registration( list_add_tail(&pr_reg->pr_reg_abort_list, preempt_and_abort_list); } -void core_scsi3_free_pr_reg_from_nacl( - struct se_device *dev, - struct se_node_acl *nacl) -{ - struct t10_reservation *pr_tmpl = &dev->t10_pr; - struct t10_pr_registration *pr_reg, *pr_reg_tmp, *pr_res_holder; - bool free_reg = false; - /* - * If the passed se_node_acl matches the reservation holder, - * release the reservation. - */ - spin_lock(&dev->dev_reservation_lock); - pr_res_holder = dev->dev_pr_res_holder; - if ((pr_res_holder != NULL) && - (pr_res_holder->pr_reg_nacl == nacl)) { - __core_scsi3_complete_pro_release(dev, nacl, pr_res_holder, 0, 1); - free_reg = true; - } - spin_unlock(&dev->dev_reservation_lock); - /* - * Release any registration associated with the struct se_node_acl. - */ - spin_lock(&pr_tmpl->registration_lock); - if (pr_res_holder && free_reg) - __core_scsi3_free_registration(dev, pr_res_holder, NULL, 0); - - list_for_each_entry_safe(pr_reg, pr_reg_tmp, - &pr_tmpl->registration_list, pr_reg_list) { - - if (pr_reg->pr_reg_nacl != nacl) - continue; - - __core_scsi3_free_registration(dev, pr_reg, NULL, 0); - } - spin_unlock(&pr_tmpl->registration_lock); -} - void core_scsi3_free_all_registrations( struct se_device *dev) { @@ -1373,11 +1341,8 @@ void core_scsi3_free_all_registrations( spin_lock(&dev->dev_reservation_lock); pr_res_holder = dev->dev_pr_res_holder; - if (pr_res_holder != NULL) { - struct se_node_acl *pr_res_nacl = pr_res_holder->pr_reg_nacl; - __core_scsi3_complete_pro_release(dev, pr_res_nacl, - pr_res_holder, 0, 0); - } + if (pr_res_holder != NULL) + __core_scsi3_complete_pro_release(dev, pr_res_holder, 0, 0); spin_unlock(&dev->dev_reservation_lock); spin_lock(&pr_tmpl->registration_lock); @@ -1872,7 +1837,7 @@ static int core_scsi3_update_aptpl_buf( tmp[0] = '\0'; isid_buf[0] = '\0'; - tpg = pr_reg->pr_reg_nacl->se_tpg; + tpg = pr_reg->se_tpg; /* * Write out any ISID value to APTPL metadata that was included * in the original registration. @@ -1893,7 +1858,7 @@ static int core_scsi3_update_aptpl_buf( "res_scope=%02x\nres_all_tg_pt=%d\n" "mapped_lun=%llu\n", reg_count, tpg->se_tpg_tfo->fabric_name, - pr_reg->pr_reg_nacl->initiatorname, isid_buf, + pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_res_type, pr_reg->pr_res_scope, pr_reg->pr_reg_all_tg_pt, pr_reg->pr_res_mapped_lun); @@ -1903,7 +1868,7 @@ static int core_scsi3_update_aptpl_buf( "sa_res_key=%llu\nres_holder=0\n" "res_all_tg_pt=%d\nmapped_lun=%llu\n", reg_count, tpg->se_tpg_tfo->fabric_name, - pr_reg->pr_reg_nacl->initiatorname, isid_buf, + pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_reg_all_tg_pt, pr_reg->pr_res_mapped_lun); } @@ -2149,7 +2114,7 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key, " Key for %s to: 0x%016Lx PRgeneration:" " 0x%08x\n", cmd->se_tfo->fabric_name, (register_type == REGISTER_AND_IGNORE_EXISTING_KEY) ? "_AND_IGNORE_EXISTING_KEY" : "", - pr_reg->pr_reg_nacl->initiatorname, + pr_reg->pr_iport, pr_reg->pr_res_key, pr_reg->pr_res_generation); } else { @@ -2180,8 +2145,8 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key, continue; if (pr_reg == pr_reg_p) continue; - if (strcmp(pr_reg->pr_reg_nacl->initiatorname, - pr_reg_p->pr_reg_nacl->initiatorname)) + if (strcmp(pr_reg->pr_iport, + pr_reg_p->pr_iport)) continue; __core_scsi3_free_registration(dev, @@ -2329,14 +2294,13 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) * RESERVATION CONFLICT status. */ if (!is_reservation_holder(pr_res_holder, pr_reg)) { - struct se_node_acl *pr_res_nacl = pr_res_holder->pr_reg_nacl; pr_err("SPC-3 PR: Attempted RESERVE from" " [%s]: %s while reservation already held by" " [%s]: %s, returning RESERVATION_CONFLICT\n", cmd->se_tfo->fabric_name, se_sess->se_node_acl->initiatorname, - pr_res_nacl->se_tpg->se_tpg_tfo->fabric_name, - pr_res_holder->pr_reg_nacl->initiatorname); + pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); ret = TCM_RESERVATION_CONFLICT; @@ -2351,15 +2315,14 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) */ if ((pr_res_holder->pr_res_type != type) || (pr_res_holder->pr_res_scope != scope)) { - struct se_node_acl *pr_res_nacl = pr_res_holder->pr_reg_nacl; pr_err("SPC-3 PR: Attempted RESERVE from" " [%s]: %s trying to change TYPE and/or SCOPE," " while reservation already held by [%s]: %s," " returning RESERVATION_CONFLICT\n", cmd->se_tfo->fabric_name, se_sess->se_node_acl->initiatorname, - pr_res_nacl->se_tpg->se_tpg_tfo->fabric_name, - pr_res_holder->pr_reg_nacl->initiatorname); + pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); ret = TCM_RESERVATION_CONFLICT; @@ -2429,12 +2392,11 @@ core_scsi3_emulate_pro_reserve(struct se_cmd *cmd, int type, int scope, static void __core_scsi3_complete_pro_release( struct se_device *dev, - struct se_node_acl *se_nacl, struct t10_pr_registration *pr_reg, int explicit, int unreg) { - const struct target_core_fabric_ops *tfo = se_nacl->se_tpg->se_tpg_tfo; + const struct target_core_fabric_ops *tfo = pr_reg->se_tpg->se_tpg_tfo; char i_buf[PR_REG_ISID_ID_LEN] = { }; int pr_res_type = 0, pr_res_scope = 0; @@ -2486,7 +2448,7 @@ static void __core_scsi3_complete_pro_release( (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); } pr_debug("SPC-3 PR [%s] RELEASE Node: %s%s\n", - tfo->fabric_name, se_nacl->initiatorname, + tfo->fabric_name, pr_reg->pr_iport, i_buf); /* * Clear TYPE and SCOPE for the next PROUT Service Action: RESERVE @@ -2579,15 +2541,14 @@ core_scsi3_emulate_pro_release(struct se_cmd *cmd, int type, int scope, */ if ((pr_res_holder->pr_res_type != type) || (pr_res_holder->pr_res_scope != scope)) { - struct se_node_acl *pr_res_nacl = pr_res_holder->pr_reg_nacl; pr_err("SPC-3 PR RELEASE: Attempted to release" " reservation from [%s]: %s with different TYPE " "and/or SCOPE while reservation already held by" " [%s]: %s, returning RESERVATION_CONFLICT\n", cmd->se_tfo->fabric_name, se_sess->se_node_acl->initiatorname, - pr_res_nacl->se_tpg->se_tpg_tfo->fabric_name, - pr_res_holder->pr_reg_nacl->initiatorname); + pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); ret = TCM_RESERVATION_CONFLICT; @@ -2609,8 +2570,7 @@ core_scsi3_emulate_pro_release(struct se_cmd *cmd, int type, int scope, * d) If the persistent reservation is of any other type, the device * server shall not establish a unit attention condition. */ - __core_scsi3_complete_pro_release(dev, se_sess->se_node_acl, - pr_reg, 1, 0); + __core_scsi3_complete_pro_release(dev, pr_reg, 1, 0); spin_unlock(&dev->dev_reservation_lock); @@ -2695,11 +2655,8 @@ core_scsi3_emulate_pro_clear(struct se_cmd *cmd, u64 res_key) */ spin_lock(&dev->dev_reservation_lock); pr_res_holder = dev->dev_pr_res_holder; - if (pr_res_holder) { - struct se_node_acl *pr_res_nacl = pr_res_holder->pr_reg_nacl; - __core_scsi3_complete_pro_release(dev, pr_res_nacl, - pr_res_holder, 0, 0); - } + if (pr_res_holder) + __core_scsi3_complete_pro_release(dev, pr_res_holder, 0, 0); spin_unlock(&dev->dev_reservation_lock); /* * b) Remove all registration(s) (see spc4r17 5.7.7); @@ -2743,8 +2700,7 @@ static void __core_scsi3_complete_pro_preempt( int scope, enum preempt_type preempt_type) { - struct se_node_acl *nacl = pr_reg->pr_reg_nacl; - const struct target_core_fabric_ops *tfo = nacl->se_tpg->se_tpg_tfo; + const struct target_core_fabric_ops *tfo = pr_reg->se_tpg->se_tpg_tfo; char i_buf[PR_REG_ISID_ID_LEN] = { }; lockdep_assert_held(&dev->dev_reservation_lock); @@ -2754,8 +2710,7 @@ static void __core_scsi3_complete_pro_preempt( * Do an implicit RELEASE of the existing reservation. */ if (dev->dev_pr_res_holder) - __core_scsi3_complete_pro_release(dev, nacl, - dev->dev_pr_res_holder, 0, 0); + __core_scsi3_complete_pro_release(dev, dev->dev_pr_res_holder, 0, 0); dev->dev_pr_res_holder = pr_reg; pr_reg->pr_res_holder = 1; @@ -2769,7 +2724,7 @@ static void __core_scsi3_complete_pro_preempt( (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); pr_debug("SPC-3 PR [%s] PREEMPT%s from Node: %s%s\n", tfo->fabric_name, (preempt_type == PREEMPT_AND_ABORT) ? "_AND_ABORT" : "", - nacl->initiatorname, i_buf); + pr_reg->pr_iport, i_buf); /* * For PREEMPT_AND_ABORT, add the preempting reservation's * struct t10_pr_registration to the list that will be compared @@ -3015,7 +2970,6 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, */ if (pr_reg_n != pr_res_holder) __core_scsi3_complete_pro_release(dev, - pr_res_holder->pr_reg_nacl, dev->dev_pr_res_holder, 0, 0); /* * b) Remove the registrations for all I_T nexuses identified @@ -3142,7 +3096,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, struct se_device *dev = cmd->se_dev; struct se_dev_entry *dest_se_deve = NULL; struct se_lun *se_lun = cmd->se_lun, *tmp_lun; - struct se_node_acl *pr_res_nacl, *pr_reg_nacl, *dest_node_acl = NULL; + struct se_node_acl *dest_node_acl = NULL; struct se_portal_group *se_tpg, *dest_se_tpg = NULL; const struct target_core_fabric_ops *dest_tf_ops = NULL, *tf_ops; struct t10_pr_registration *pr_reg, *pr_res_holder, *dest_pr_reg; @@ -3297,23 +3251,21 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, * ILLEGAL REQUEST, and the additional sense code set to INVALID FIELD * IN PARAMETER LIST. */ - pr_reg_nacl = pr_reg->pr_reg_nacl; - matching_iname = (!strcmp(initiator_str, - pr_reg_nacl->initiatorname)) ? 1 : 0; + matching_iname = !strcmp(initiator_str, pr_reg->pr_iport); if (!matching_iname) goto after_iport_check; if (!iport_ptr || !pr_reg->isid_present_at_reg) { pr_err("SPC-3 PR REGISTER_AND_MOVE: TransportID: %s" " matches: %s on received I_T Nexus\n", initiator_str, - pr_reg_nacl->initiatorname); + pr_reg->pr_iport); ret = TCM_INVALID_PARAMETER_LIST; goto out; } if (!strcmp(iport_ptr, pr_reg->pr_reg_isid)) { pr_err("SPC-3 PR REGISTER_AND_MOVE: TransportID: %s %s" " matches: %s %s on received I_T Nexus\n", - initiator_str, iport_ptr, pr_reg_nacl->initiatorname, + initiator_str, iport_ptr, pr_reg->pr_iport, pr_reg->pr_reg_isid); ret = TCM_INVALID_PARAMETER_LIST; goto out; @@ -3419,7 +3371,6 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, ret = TCM_RESERVATION_CONFLICT; goto out; } - pr_res_nacl = pr_res_holder->pr_reg_nacl; /* * b) Ignore the contents of the (received) SCOPE and TYPE fields; */ @@ -3471,8 +3422,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, * f) Release the persistent reservation for the persistent reservation * holder (i.e., the I_T nexus on which the */ - __core_scsi3_complete_pro_release(dev, pr_res_nacl, - dev->dev_pr_res_holder, 0, 0); + __core_scsi3_complete_pro_release(dev, dev->dev_pr_res_holder, 0, 0); /* * g) Move the persistent reservation to the specified I_T nexus using * the same scope and type as the persistent reservation released in @@ -3497,7 +3447,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, dest_pr_reg->pr_res_generation); pr_debug("SPC-3 PR Successfully moved reservation from" " %s Fabric Node: %s%s -> %s Fabric Node: %s %s\n", - tf_ops->fabric_name, pr_reg_nacl->initiatorname, + tf_ops->fabric_name, pr_reg->pr_iport, i_buf, dest_tf_ops->fabric_name, dest_node_acl->initiatorname, (iport_ptr != NULL) ? iport_ptr : ""); @@ -3885,7 +3835,6 @@ static sense_reason_t core_scsi3_pri_read_full_status(struct se_cmd *cmd) { struct se_device *dev = cmd->se_dev; - struct se_node_acl *se_nacl; struct se_portal_group *se_tpg; struct t10_pr_registration *pr_reg, *pr_reg_tmp; struct t10_reservation *pr_tmpl = &dev->t10_pr; @@ -3925,8 +3874,7 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) list_for_each_entry_safe(pr_reg, pr_reg_tmp, &pr_tmpl->registration_list, pr_reg_list) { - se_nacl = pr_reg->pr_reg_nacl; - se_tpg = pr_reg->pr_reg_nacl->se_tpg; + se_tpg = pr_reg->se_tpg; add_desc_len = 0; atomic_inc_mb(&pr_reg->pr_res_holders); @@ -3935,7 +3883,7 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) * Determine expected length of $FABRIC_MOD specific * TransportID full status descriptor.. */ - exp_desc_len = target_get_pr_transport_id_len(se_nacl, pr_reg, + exp_desc_len = target_get_pr_transport_id_len(pr_reg, &format_code); if (exp_desc_len < 0 || exp_desc_len + add_len > cmd->data_length) { @@ -4001,7 +3949,7 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) /* * Now, have the $FABRIC_MOD fill in the transport ID. */ - desc_len = target_get_pr_transport_id(se_nacl, pr_reg, + desc_len = target_get_pr_transport_id(pr_reg, &format_code, &buf[off+4]); spin_lock(&pr_tmpl->registration_lock); diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index b793c99637ab..202f9aec963a 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -68,8 +68,6 @@ extern int core_scsi3_alloc_aptpl_registration( extern int core_scsi3_check_aptpl_registration(struct se_device *, struct se_portal_group *, struct se_lun *, struct se_node_acl *, u64); -extern void core_scsi3_free_pr_reg_from_nacl(struct se_device *, - struct se_node_acl *); extern void core_scsi3_free_all_registrations(struct se_device *); extern unsigned char *core_scsi3_pr_dump_type(int); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 91c068525c02..1b673b25c6cd 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -369,6 +369,7 @@ struct t10_pr_registration { u64 pr_res_key; atomic_t pr_res_holders; struct se_node_acl *pr_reg_nacl; + struct se_portal_group *se_tpg; /* Used by ALL_TG_PT=1 registration with deve->pr_ref taken */ struct se_dev_entry *pr_reg_deve; struct list_head pr_reg_list; From patchwork Fri Dec 10 12:43:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A908C25B08 for ; Wed, 3 Aug 2022 16:29:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238424AbiHCQ3r (ORCPT ); Wed, 3 Aug 2022 12:29:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238303AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01A0547BA0; Wed, 3 Aug 2022 09:29:25 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 79B5A412FA; Wed, 3 Aug 2022 16:29:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544162; x=1661358563; bh=xG+WTqoRBkLpo8ujtBOSAJV99 F2eWXKbZllm0H+sg0M=; b=BxyoCKEEeVssxHJ9c/AqbdqLrnXHFllsQeEvfcDsP J09X3z51SoSgQCedOcurVFjuvPgClKp4hP6WOS9CNuMYOQJQHxTxlbK/twuPi7E/ HpCcZ557IWV1RpAbpaPuSY8ubWunH8hVGpMyk8990JCAJaUvrDlURpI4gXln+ffQ LU= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IvMADmlFvq1i; Wed, 3 Aug 2022 19:29:22 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 9352341291; Wed, 3 Aug 2022 19:29:16 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:16 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:15 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 17/48] target: core: make some functions public Date: Fri, 10 Dec 2021 15:43:09 +0300 Message-ID: <20220803162857.27770-18-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Make functions of alloc/free and reserve/release reservation be public to use them from cluster code. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 51 +++++++++++++++++++-------------- drivers/target/target_core_pr.h | 24 ++++++++++++++++ 2 files changed, 54 insertions(+), 21 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index aa967ea74e5b..aaa383e1ee58 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -64,9 +64,6 @@ enum preempt_type { PREEMPT_AND_ABORT, }; -static void __core_scsi3_complete_pro_release(struct se_device *, - struct t10_pr_registration *, int, int); - static int is_reservation_holder( struct t10_pr_registration *pr_res_holder, struct t10_pr_registration *pr_reg) @@ -606,7 +603,7 @@ static u32 core_scsi3_pr_generation(struct se_device *dev) return atomic_inc_return(&dev->t10_pr.pr_generation); } -static struct t10_pr_registration *__core_scsi3_do_alloc_registration( +struct t10_pr_registration *__core_scsi3_do_alloc_registration( struct se_device *dev, struct se_node_acl *nacl, const char *initiatorname, @@ -1261,7 +1258,7 @@ static int core_scsi3_check_implicit_release( return ret; } -static void __core_scsi3_free_registration( +void __core_scsi3_free_registration( struct se_device *dev, struct t10_pr_registration *pr_reg, struct list_head *preempt_and_abort_list, @@ -2219,6 +2216,32 @@ unsigned char *core_scsi3_pr_dump_type(int type) return "Unknown SPC-3 PR Type"; } +void __core_scsi3_set_reservation(struct se_device *dev, + struct t10_pr_registration *pr_reg, + int scope, int type) +{ + char i_buf[PR_REG_ISID_ID_LEN]; + + memset(i_buf, 0, PR_REG_ISID_ID_LEN); + + lockdep_assert_held(&dev->dev_reservation_lock); + + pr_reg->pr_res_scope = scope; + pr_reg->pr_res_type = type; + pr_reg->pr_res_holder = 1; + dev->dev_pr_res_holder = pr_reg; + core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); + + pr_debug("SPC-3 PR [%s] Service Action: RESERVE created new reservation holder TYPE: %s ALL_TG_PT: %d\n", + pr_reg->se_tpg->se_tpg_tfo->fabric_name, + core_scsi3_pr_dump_type(type), + (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); + pr_debug("SPC-3 PR [%s] RESERVE Node: %s%s\n", + pr_reg->se_tpg->se_tpg_tfo->fabric_name, + pr_reg->pr_iport, + i_buf); +} + static sense_reason_t core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) { @@ -2227,7 +2250,6 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) struct se_lun *se_lun = cmd->se_lun; struct t10_pr_registration *pr_reg, *pr_res_holder; struct t10_reservation *pr_tmpl = &dev->t10_pr; - char i_buf[PR_REG_ISID_ID_LEN] = { }; sense_reason_t ret; if (!se_sess || !se_lun) { @@ -2346,20 +2368,7 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) * Otherwise, our *pr_reg becomes the PR reservation holder for said * TYPE/SCOPE. Also set the received scope and type in *pr_reg. */ - pr_reg->pr_res_scope = scope; - pr_reg->pr_res_type = type; - pr_reg->pr_res_holder = 1; - dev->dev_pr_res_holder = pr_reg; - core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); - - pr_debug("SPC-3 PR [%s] Service Action: RESERVE created new" - " reservation holder TYPE: %s ALL_TG_PT: %d\n", - cmd->se_tfo->fabric_name, core_scsi3_pr_dump_type(type), - (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); - pr_debug("SPC-3 PR [%s] RESERVE Node: %s%s\n", - cmd->se_tfo->fabric_name, - se_sess->se_node_acl->initiatorname, - i_buf); + __core_scsi3_set_reservation(dev, pr_reg, scope, type); spin_unlock(&dev->dev_reservation_lock); if (pr_tmpl->pr_aptpl_active) @@ -2390,7 +2399,7 @@ core_scsi3_emulate_pro_reserve(struct se_cmd *cmd, int type, int scope, } } -static void __core_scsi3_complete_pro_release( +void __core_scsi3_complete_pro_release( struct se_device *dev, struct t10_pr_registration *pr_reg, int explicit, diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index 202f9aec963a..9c4710f34d94 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -75,4 +75,28 @@ extern sense_reason_t target_scsi3_emulate_pr_in(struct se_cmd *); extern sense_reason_t target_scsi3_emulate_pr_out(struct se_cmd *); extern sense_reason_t target_check_reservation(struct se_cmd *); +void __core_scsi3_set_reservation(struct se_device *dev, + struct t10_pr_registration *pr_reg, + int scope, int type); +void __core_scsi3_complete_pro_release(struct se_device *dev, + struct t10_pr_registration *pr_reg, + int explicit, int unreg); +struct t10_pr_registration *__core_scsi3_do_alloc_registration( + struct se_device *dev, + struct se_node_acl *nacl, + const char *initiatorname, + u64 unpacked_lun, + struct se_portal_group *se_tpg, + struct se_dev_entry *dest_deve, + u64 mapped_lun, + unsigned char *isid, + u64 sa_res_key, + int all_tg_pt, + int aptpl); +void __core_scsi3_free_registration(struct se_device *dev, + struct t10_pr_registration *pr_reg, + struct list_head *preempt_and_abort_list, + int dec_holders); + + #endif /* TARGET_CORE_PR_H */ From patchwork Mon Dec 13 18:58:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595468 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C17EC19F29 for ; Wed, 3 Aug 2022 16:29:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238430AbiHCQ3s (ORCPT ); Wed, 3 Aug 2022 12:29:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238326AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D55F145079; Wed, 3 Aug 2022 09:29:25 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 800D9412E2; Wed, 3 Aug 2022 16:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544163; x=1661358564; bh=yvW0+De8n3RtbDdXr4Va8SRnE RepVX+tuieaZ8OWrcs=; b=Xm2sBgh5bINOL40uS3M66Nt2UNSEEkqOvERsCU+VH ryLhmNvHOg297QcdHpHfZHUOIdW62RETvHx438mFbCVVTStrYvWB//U/pDGQJYHH uMNxT6NwDegKlZfv0iWz239I85i8ChytaH/0C4djcrABDYCTprF+2+cAtoW/+8bW pc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JinRm1P4A7Zl; Wed, 3 Aug 2022 19:29:23 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id DF9D44128A; Wed, 3 Aug 2022 19:29:16 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:16 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:16 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 18/48] target: core: proper clear reservation on LUN RESET Date: Mon, 13 Dec 2021 21:58:23 +0300 Message-ID: <20220803162857.27770-19-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Use an existing helper to clear SCSI-2 reservation on LUN RESET. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_tmr.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index bac111456fa1..51d95e0b8885 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -394,8 +394,7 @@ int core_tmr_lun_reset( if (!preempt_and_abort_list && (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)) { spin_lock(&dev->dev_reservation_lock); - dev->reservation_holder = NULL; - dev->dev_reservation_flags &= ~DRF_SPC2_RESERVATIONS; + target_release_reservation(dev); spin_unlock(&dev->dev_reservation_lock); pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); } From patchwork Mon Dec 13 19:15:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4960DC19F28 for ; Wed, 3 Aug 2022 16:29:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238454AbiHCQ3z (ORCPT ); Wed, 3 Aug 2022 12:29:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238333AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5C23481D9; Wed, 3 Aug 2022 09:29:26 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 7EDD1412E7; Wed, 3 Aug 2022 16:29:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544164; x=1661358565; bh=yh6LXQKlGYjI0UzGNwr6MY9lw j4LYJh3vaAjP2xKYPQ=; b=km7ljrz7zEtU8gyY3GRgDnRkBL1uFXYKqzyPlMJHa UatXHM3w4037HkMNabemiLq+dY5CYnpxO7YBRkylOliQ57E3PDd2dkx0yzAC9bCO GH7S+Vf1Gtfj63WmSEdb2oYZ7eAqx/NnXztyl1+lr3xe2ULKUQnNv4r8OiYreZxK Vc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sucqdIsIgKFG; Wed, 3 Aug 2022 19:29:24 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 4591841292; Wed, 3 Aug 2022 19:29:17 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:17 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:16 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 19/48] target: core: remove superfluous checks Date: Mon, 13 Dec 2021 22:15:47 +0300 Message-ID: <20220803162857.27770-20-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org remove double checks on path of SCSI-2 reservations Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index aaa383e1ee58..4bb16de29a1e 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -95,9 +95,6 @@ target_scsi2_reservation_check(struct se_cmd *cmd) break; } - if (!dev->reservation_holder || !sess) - return 0; - if (dev->reservation_holder->se_node_acl != sess->se_node_acl) return TCM_RESERVATION_CONFLICT; @@ -211,7 +208,7 @@ target_scsi2_reservation_release(struct se_cmd *cmd) return TCM_RESERVATION_CONFLICT; spin_lock(&dev->dev_reservation_lock); - if (!dev->reservation_holder || !sess) + if (!dev->reservation_holder) goto out_unlock; if (dev->reservation_holder->se_node_acl != sess->se_node_acl) From patchwork Mon Dec 13 19:20:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595466 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A32BBC25B06 for ; Wed, 3 Aug 2022 16:29:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238325AbiHCQ35 (ORCPT ); Wed, 3 Aug 2022 12:29:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238341AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AFD86437; Wed, 3 Aug 2022 09:29:27 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 2075241301; Wed, 3 Aug 2022 16:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544164; x=1661358565; bh=up0BrvEsLm9+0/EKtwIjmwbdV 5hi3S99QmPqqfsgtwE=; b=o2wd1HWv06zkmYjJ7E8pena7Qkyg5+k7EHz0upieh BhqQ0sR2CQmame0R+wnIFGnUa+LmrobiO0+C6quojuQYzd7mcU8YfczzDyQIntTf dqvjOWx1a8eObfcqyQyPZCxcIw/Bl6Q+1cbNa9rCswk469F3dlz5n+c/EAeGRlCd ag= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BfhN0CMC4GnL; Wed, 3 Aug 2022 19:29:24 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 7CAED4127C; Wed, 3 Aug 2022 19:29:17 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:17 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:16 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 20/48] target: core: proper check of SCSI-2 reservation Date: Mon, 13 Dec 2021 22:20:31 +0300 Message-ID: <20220803162857.27770-21-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Use a check of the flag instead of session pointer like in other places. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 17 +++++++++++------ drivers/target/target_core_pr.c | 6 +++--- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 6e654edc4399..34901f0b2350 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1756,12 +1756,17 @@ static ssize_t target_core_dev_pr_show_spc2_res(struct se_device *dev, struct se_node_acl *se_nacl; ssize_t len; - if (sess) { - se_nacl = sess->se_node_acl; - len = sprintf(page, - "SPC-2 Reservation: %s Initiator: %s\n", - se_nacl->se_tpg->se_tpg_tfo->fabric_name, - se_nacl->initiatorname); + if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { + if (sess) { + se_nacl = sess->se_node_acl; + len = sprintf(page, + "SPC-2 Reservation: %s Initiator: %s\n", + se_nacl->se_tpg->se_tpg_tfo->fabric_name, + se_nacl->initiatorname); + } else { + len = sprintf(page, + "SPC-2 Reservation: peer node\n"); + } } else { len = sprintf(page, "No SPC-2 Reservation holder\n"); } diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 4bb16de29a1e..2f4c65d5d766 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -208,7 +208,7 @@ target_scsi2_reservation_release(struct se_cmd *cmd) return TCM_RESERVATION_CONFLICT; spin_lock(&dev->dev_reservation_lock); - if (!dev->reservation_holder) + if (!(dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)) goto out_unlock; if (dev->reservation_holder->se_node_acl != sess->se_node_acl) @@ -261,8 +261,8 @@ target_scsi2_reservation_reserve(struct se_cmd *cmd) tpg = sess->se_tpg; spin_lock(&dev->dev_reservation_lock); - if (dev->reservation_holder && - dev->reservation_holder->se_node_acl != sess->se_node_acl) { + if ((dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) && + dev->reservation_holder->se_node_acl != sess->se_node_acl) { pr_err("SCSI-2 RESERVATION CONFLICT for %s fabric\n", tpg->se_tpg_tfo->fabric_name); pr_err("Original reserver LUN: %llu %s\n", From patchwork Mon Dec 13 19:28:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB7B4C19F2C for ; Wed, 3 Aug 2022 16:29:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238450AbiHCQ3w (ORCPT ); Wed, 3 Aug 2022 12:29:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238317AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1F6B474E5; Wed, 3 Aug 2022 09:29:27 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9F10241305; Wed, 3 Aug 2022 16:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544165; x=1661358566; bh=UZx7O79uzg91XujHbNS/78m3g 0VCYVZ6MyHH03e8Ggc=; b=RkJQtEA9k6/TFzROpY52XRaSpaDuozV8YB5oN4/P9 jealZYIKEFe+SeWnNNedS6UD4Jr4Hxpi6dOXOKmhkGmHe29POvl8HAu0b4/E8pCJ 5KGFJiqZem/uCRObO3tev1tobwxk6JUMVidSQcv3kTwMVshxoWcTBzZFPqCjJjr2 Q0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KoC34oFUx45M; Wed, 3 Aug 2022 19:29:25 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id CAF1B412C5; Wed, 3 Aug 2022 19:29:17 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:17 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:17 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 21/48] target: core: checks against peer node SCSI2 reservation Date: Mon, 13 Dec 2021 22:28:57 +0300 Message-ID: <20220803162857.27770-22-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org For the case of SCSI-2 reservation by peer node dev->reservation_holder session is NULL. So use this information as reservation existence. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 2f4c65d5d766..c4329f639323 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -95,7 +95,8 @@ target_scsi2_reservation_check(struct se_cmd *cmd) break; } - if (dev->reservation_holder->se_node_acl != sess->se_node_acl) + if (!dev->reservation_holder || + dev->reservation_holder->se_node_acl != sess->se_node_acl) return TCM_RESERVATION_CONFLICT; if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS_WITH_ISID) { @@ -211,6 +212,9 @@ target_scsi2_reservation_release(struct se_cmd *cmd) if (!(dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)) goto out_unlock; + if (!dev->reservation_holder) + goto out_unlock; + if (dev->reservation_holder->se_node_acl != sess->se_node_acl) goto out_unlock; @@ -262,12 +266,15 @@ target_scsi2_reservation_reserve(struct se_cmd *cmd) tpg = sess->se_tpg; spin_lock(&dev->dev_reservation_lock); if ((dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) && - dev->reservation_holder->se_node_acl != sess->se_node_acl) { + (!dev->reservation_holder || + dev->reservation_holder->se_node_acl != sess->se_node_acl)) { pr_err("SCSI-2 RESERVATION CONFLICT for %s fabric\n", tpg->se_tpg_tfo->fabric_name); pr_err("Original reserver LUN: %llu %s\n", cmd->se_lun->unpacked_lun, - dev->reservation_holder->se_node_acl->initiatorname); + (dev->reservation_holder) ? + dev->reservation_holder->se_node_acl->initiatorname : + "peer"); pr_err("Current attempt - LUN: %llu -> MAPPED LUN: %llu" " from %s \n", cmd->se_lun->unpacked_lun, cmd->orig_fe_lun, From patchwork Fri Mar 11 08:11:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B6B9C19F28 for ; Wed, 3 Aug 2022 16:29:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238456AbiHCQ35 (ORCPT ); Wed, 3 Aug 2022 12:29:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238342AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F96B48CB3; Wed, 3 Aug 2022 09:29:28 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 50B5E412FA; Wed, 3 Aug 2022 16:29:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544166; x=1661358567; bh=6t31BwzaJqtNyNhVTPhBBhTP5 uz2EKN7rApucpAjv7I=; b=lKMB7uv64JWtCI32xvggjjYQRZ4jpN9V72CKUlRDw 5djyT+CMC1O8sjBaLHdoYbz5CFSDmLkVepOp3ZPOnRkDB43BFzKdV+CPNY4n21iq uFHXRfOTtn2fbVQ5AeFJHvHOH6ADIAGICtCEIyogMsOCl2xvZIT3J3ed4Yf0O9bf 9w= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9eq6r0XM3CWw; Wed, 3 Aug 2022 19:29:26 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 36664412DB; Wed, 3 Aug 2022 19:29:18 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:18 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:17 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 22/48] target: core: UA on all luns after reset Date: Fri, 11 Mar 2022 11:11:59 +0300 Message-ID: <20220803162857.27770-23-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allocate UA "BUS DEVICE RESET OCCURRED" on all LUNs on the device upon reception of TMF LUN RESET. That patch passes a libisci test SCSI.MultipathIO.Reset. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_device.c | 19 +++++++++++++++++++ drivers/target/target_core_internal.h | 1 + drivers/target/target_core_transport.c | 3 +-- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index a4c31f3112ce..7c887792bb88 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -285,6 +285,25 @@ void target_pr_kref_release(struct kref *kref) complete(&deve->pr_comp); } +/* + * Establish UA condition on SCSI device - all LUNs + */ +void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq) +{ + struct se_dev_entry *se_deve; + struct se_lun *lun; + + spin_lock(&dev->se_port_lock); + list_for_each_entry(lun, &dev->dev_sep_list, lun_dev_link) { + + spin_lock(&lun->lun_deve_lock); + list_for_each_entry(se_deve, &lun->lun_deve_list, lun_link) + core_scsi3_ua_allocate(se_deve, asc, ascq); + spin_unlock(&lun->lun_deve_lock); + } + spin_unlock(&dev->se_port_lock); +} + static void target_luns_data_has_changed(struct se_node_acl *nacl, struct se_dev_entry *new, bool skip_new) diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 8bd48eb1cf9b..eeb405908987 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -88,6 +88,7 @@ int target_configure_device(struct se_device *dev); void target_free_device(struct se_device *); int target_for_each_device(int (*fn)(struct se_device *dev, void *data), void *data); +void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq); /* target_core_configfs.c */ extern struct configfs_item_operations target_core_dev_item_ops; diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 7838dc20f713..5926316252eb 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -3531,8 +3531,7 @@ static void target_tmr_work(struct work_struct *work) tmr->response = (!ret) ? TMR_FUNCTION_COMPLETE : TMR_FUNCTION_REJECTED; if (tmr->response == TMR_FUNCTION_COMPLETE) { - target_ua_allocate_lun(cmd->se_sess->se_node_acl, - cmd->orig_fe_lun, 0x29, + target_dev_ua_allocate(dev, 0x29, ASCQ_29H_BUS_DEVICE_RESET_FUNCTION_OCCURRED); } break; From patchwork Fri Mar 11 08:33:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9408C19F2C for ; Wed, 3 Aug 2022 16:30:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238480AbiHCQ37 (ORCPT ); Wed, 3 Aug 2022 12:29:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238361AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 672D847BAA; Wed, 3 Aug 2022 09:29:29 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 14063412E2; Wed, 3 Aug 2022 16:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544166; x=1661358567; bh=HIZ0Axyq6o3ZC/QsHWwAfBYmY tWHLdgHTxqoX9QsDTc=; b=ux3+TX2gOkqZWWYv+NqXkBTsI9uPKHTcvJDsehv2+ NVX+szKOY0aw57Y9qpCxyecv4e8qpsjWHh1GXkCFjYDz4CsZh9u2jKVdOf/ZNt5E jmPDc35FVt0EVv6A56XEKIvsmHs5mxoCkLmwlxWAw9g0iUnXw/KG4ef2HeFWJx8S Ko= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LZO2pnpH6CpT; Wed, 3 Aug 2022 19:29:26 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 7F38A412C8; Wed, 3 Aug 2022 19:29:18 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:18 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:17 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 23/48] target: core: refactor LUN_RESET code Date: Fri, 11 Mar 2022 11:33:58 +0300 Message-ID: <20220803162857.27770-24-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org No function change, just extract a code of aborting all active commands on the device. To make it possible to call TMF LUN_RESET code without cluster related things - SCSI2 reservation or ACA condition. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_internal.h | 5 +++-- drivers/target/target_core_pr.c | 4 ++-- drivers/target/target_core_tmr.c | 27 +++++++++++++++++++------- drivers/target/target_core_transport.c | 6 +----- 4 files changed, 26 insertions(+), 16 deletions(-) diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index eeb405908987..b32ac78a26b0 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -112,8 +112,9 @@ int core_delete_hba(struct se_hba *); /* target_core_tmr.c */ void core_tmr_abort_task(struct se_device *, struct se_tmr_req *, struct se_session *); -int core_tmr_lun_reset(struct se_device *, struct se_tmr_req *, - struct list_head *, struct se_cmd *); +int core_tmr_lun_reset(struct se_device *dev, struct se_tmr_req *tmr); +void core_local_lun_reset(struct se_device *swc, struct se_tmr_req *tmr, + struct list_head *preempt_and_abort_list, struct se_cmd *prout_cmd); /* target_core_tpg.c */ extern struct spinlock g_tpg_lock; diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index c4329f639323..dfecb155bf88 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -2936,7 +2936,7 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, * registered I_T nexuses. */ if (preempt_type == PREEMPT_AND_ABORT) { - core_tmr_lun_reset(dev, NULL, &preempt_and_abort_list, + core_local_lun_reset(dev, NULL, &preempt_and_abort_list, cmd); core_scsi3_release_preempt_and_abort( &preempt_and_abort_list, pr_reg_n); @@ -3067,7 +3067,7 @@ core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, * new persistent reservation holder, the calling Initiator Port. */ if (preempt_type == PREEMPT_AND_ABORT) { - core_tmr_lun_reset(dev, NULL, &preempt_and_abort_list, cmd); + core_local_lun_reset(dev, NULL, &preempt_and_abort_list, cmd); core_scsi3_release_preempt_and_abort(&preempt_and_abort_list, pr_reg_n); } diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 51d95e0b8885..2db45efedd97 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -22,6 +22,7 @@ #include "target_core_internal.h" #include "target_core_alua.h" #include "target_core_pr.h" +#include "target_core_ua.h" int core_tmr_alloc_req( struct se_cmd *se_cmd, @@ -342,7 +343,8 @@ static void core_tmr_drain_state_list( } } -int core_tmr_lun_reset( + +void core_local_lun_reset( struct se_device *dev, struct se_tmr_req *tmr, struct list_head *preempt_and_abort_list, @@ -352,7 +354,8 @@ int core_tmr_lun_reset( struct se_portal_group *tmr_tpg = NULL; struct se_session *tmr_sess = NULL; bool tas; - /* + + /* * TASK_ABORTED status bit, this is configurable via ConfigFS * struct se_device attributes. spc4r17 section 7.4.6 Control mode page * @@ -387,23 +390,33 @@ int core_tmr_lun_reset( core_tmr_drain_state_list(dev, prout_cmd, tmr_sess, tas, preempt_and_abort_list); + pr_debug("LUN_RESET: %s for [%s] Complete\n", + (preempt_and_abort_list) ? "Preempt" : "TMR", + dev->transport->name); +} + +int core_tmr_lun_reset( + struct se_device *dev, + struct se_tmr_req *tmr) +{ + core_local_lun_reset(dev, tmr, NULL, NULL); + /* * Clear any legacy SPC-2 reservation when called during * LOGICAL UNIT RESET */ - if (!preempt_and_abort_list && - (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)) { + if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { spin_lock(&dev->dev_reservation_lock); target_release_reservation(dev); spin_unlock(&dev->dev_reservation_lock); pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); } + target_dev_ua_allocate(dev, 0x29, + ASCQ_29H_BUS_DEVICE_RESET_FUNCTION_OCCURRED); + atomic_long_inc(&dev->num_resets); - pr_debug("LUN_RESET: %s for [%s] Complete\n", - (preempt_and_abort_list) ? "Preempt" : "TMR", - dev->transport->name); return 0; } diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 5926316252eb..d3cbba294510 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -3527,13 +3527,9 @@ static void target_tmr_work(struct work_struct *work) tmr->response = TMR_TASK_MGMT_FUNCTION_NOT_SUPPORTED; break; case TMR_LUN_RESET: - ret = core_tmr_lun_reset(dev, tmr, NULL, NULL); + ret = core_tmr_lun_reset(dev, tmr); tmr->response = (!ret) ? TMR_FUNCTION_COMPLETE : TMR_FUNCTION_REJECTED; - if (tmr->response == TMR_FUNCTION_COMPLETE) { - target_dev_ua_allocate(dev, 0x29, - ASCQ_29H_BUS_DEVICE_RESET_FUNCTION_OCCURRED); - } break; case TMR_TARGET_WARM_RESET: tmr->response = TMR_FUNCTION_REJECTED; From patchwork Tue Jun 21 13:05:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45481C19F2C for ; Wed, 3 Aug 2022 16:30:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238418AbiHCQaC (ORCPT ); Wed, 3 Aug 2022 12:30:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238365AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF90548E87; Wed, 3 Aug 2022 09:29:30 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 95044412C8; Wed, 3 Aug 2022 16:29:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544168; x=1661358569; bh=JHkgEFMleDElEJ2wqhkUHs2WM +dDaxJpel72TXz3koY=; b=bRRtaJp7VDpMoEXU4bXRPzl5NVBk8U9HLpGBwQepl HEg3+Z+r3cJSnk3bNuytnK1G67psdYiLjmRGY6flFgG7upYnvKyCpAVbs28mD27n vF2HY2oV+Lwpf4y1TELO9ZOD75akUKAxak3Unr44u8Umr2tz0DkihZwvBkojhCtN DU= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hRzGNMd_oW73; Wed, 3 Aug 2022 19:29:28 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id D1BEE412DE; Wed, 3 Aug 2022 19:29:18 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:18 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:18 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 24/48] target: core: pr: use RTPI in APTPL Date: Tue, 21 Jun 2022 16:05:59 +0300 Message-ID: <20220803162857.27770-25-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Since RTPI is an unique identifier of target port there is no need to check fabric name and port tag. RTPI is already saved in APTPL data, so it is backward compatible change. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 4 ++- drivers/target/target_core_pr.c | 41 +++++++-------------------- drivers/target/target_core_pr.h | 2 +- include/target/target_core_base.h | 5 ---- 4 files changed, 15 insertions(+), 37 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 34901f0b2350..f469cfde3f1e 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1993,6 +1993,7 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, u64 mapped_lun = 0, target_lun = 0; int ret = -1, res_holder = 0, all_tg_pt = 0, arg, token; u16 tpgt = 0; + u16 rtpi = 0; u8 type = 0; if (!dev->dev_attrib.emulate_pr || @@ -2127,6 +2128,7 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, ret = match_int(args, &arg); if (ret) goto out; + rtpi = (u16)arg; break; case Opt_target_lun: ret = match_u64(args, &tmp_ll); @@ -2153,7 +2155,7 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, } ret = core_scsi3_alloc_aptpl_registration(&dev->t10_pr, sa_res_key, - i_port, isid, mapped_lun, t_port, tpgt, target_lun, + i_port, isid, mapped_lun, rtpi, target_lun, res_holder, all_tg_pt, type); out: kfree(i_fabric); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index dfecb155bf88..1f19bfd0fa00 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -832,8 +832,7 @@ int core_scsi3_alloc_aptpl_registration( unsigned char *i_port, unsigned char *isid, u64 mapped_lun, - unsigned char *t_port, - u16 tpgt, + u16 rtpi, u64 target_lun, int res_holder, int all_tg_pt, @@ -841,7 +840,7 @@ int core_scsi3_alloc_aptpl_registration( { struct t10_pr_registration *pr_reg; - if (!i_port || !t_port || !sa_res_key) { + if (!i_port || !sa_res_key) { pr_err("Illegal parameters for APTPL registration\n"); return -EINVAL; } @@ -867,6 +866,7 @@ int core_scsi3_alloc_aptpl_registration( pr_reg->pr_reg_aptpl = 1; pr_reg->pr_res_scope = 0; /* Always LUN_SCOPE */ pr_reg->pr_res_type = type; + pr_reg->tg_pt_sep_rtpi = rtpi; /* * If an ISID value had been saved in APTPL metadata for this * SCSI Initiator Port, restore it now. @@ -877,11 +877,9 @@ int core_scsi3_alloc_aptpl_registration( pr_reg->isid_present_at_reg = 1; } /* - * Copy the i_port and t_port information from caller. + * Copy the i_port information from caller. */ snprintf(pr_reg->pr_iport, PR_APTPL_MAX_IPORT_LEN, "%s", i_port); - snprintf(pr_reg->pr_tport, PR_APTPL_MAX_TPORT_LEN, "%s", t_port); - pr_reg->pr_reg_tpgt = tpgt; /* * Set pr_res_holder from caller, the pr_reg who is the reservation * holder will get it's pointer set in core_scsi3_aptpl_reserve() once @@ -933,17 +931,7 @@ static int __core_scsi3_check_aptpl_registration( { struct t10_pr_registration *pr_reg, *pr_reg_tmp; struct t10_reservation *pr_tmpl = &dev->t10_pr; - unsigned char i_port[PR_APTPL_MAX_IPORT_LEN] = { }; - unsigned char t_port[PR_APTPL_MAX_TPORT_LEN] = { }; - u16 tpgt; - /* - * Copy Initiator Port information from struct se_node_acl - */ - snprintf(i_port, PR_APTPL_MAX_IPORT_LEN, "%s", nacl->initiatorname); - snprintf(t_port, PR_APTPL_MAX_TPORT_LEN, "%s", - tpg->se_tpg_tfo->tpg_get_wwn(tpg)); - tpgt = tpg->se_tpg_tfo->tpg_get_tag(tpg); /* * Look for the matching registrations+reservation from those * created from APTPL metadata. Note that multiple registrations @@ -954,11 +942,8 @@ static int __core_scsi3_check_aptpl_registration( list_for_each_entry_safe(pr_reg, pr_reg_tmp, &pr_tmpl->aptpl_reg_list, pr_reg_aptpl_list) { - if (!strcmp(pr_reg->pr_iport, i_port) && - (pr_reg->pr_res_mapped_lun == mapped_lun) && - !(strcmp(pr_reg->pr_tport, t_port)) && - (pr_reg->pr_reg_tpgt == tpgt) && - (pr_reg->pr_aptpl_target_lun == target_lun)) { + if (!strcmp(pr_reg->pr_iport, nacl->initiatorname) && + (pr_reg->tg_pt_sep_rtpi == tpg->tpg_rtpi)) { /* * Obtain the ->pr_reg_deve pointer + reference, that * is released by __core_scsi3_add_registration() below. @@ -1851,25 +1836,22 @@ static int core_scsi3_update_aptpl_buf( * reservation holder. */ if (dev->dev_pr_res_holder == pr_reg) { - snprintf(tmp, 512, "PR_REG_START: %d" - "\ninitiator_fabric=%s\n" + snprintf(tmp, 512, "PR_REG_START: %d\n" "initiator_node=%s\n%s" "sa_res_key=%llu\n" "res_holder=1\nres_type=%02x\n" "res_scope=%02x\nres_all_tg_pt=%d\n" "mapped_lun=%llu\n", reg_count, - tpg->se_tpg_tfo->fabric_name, pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_res_type, pr_reg->pr_res_scope, pr_reg->pr_reg_all_tg_pt, pr_reg->pr_res_mapped_lun); } else { snprintf(tmp, 512, "PR_REG_START: %d\n" - "initiator_fabric=%s\ninitiator_node=%s\n%s" + "initiator_node=%s\n%s" "sa_res_key=%llu\nres_holder=0\n" "res_all_tg_pt=%d\nmapped_lun=%llu\n", - reg_count, tpg->se_tpg_tfo->fabric_name, - pr_reg->pr_iport, isid_buf, + reg_count, pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_reg_all_tg_pt, pr_reg->pr_res_mapped_lun); } @@ -1885,9 +1867,8 @@ static int core_scsi3_update_aptpl_buf( /* * Include information about the associated SCSI target port. */ - snprintf(tmp, 512, "target_fabric=%s\ntarget_node=%s\n" - "tpgt=%hu\nport_rtpi=%hu\ntarget_lun=%llu\nPR_REG_END:" - " %d\n", tpg->se_tpg_tfo->fabric_name, + snprintf(tmp, 512, "target_node=%s\n" + "tpgt=%hu\nport_rtpi=%hu\ntarget_lun=%llu\nPR_REG_END: %d\n", tpg->se_tpg_tfo->tpg_get_wwn(tpg), tpg->se_tpg_tfo->tpg_get_tag(tpg), pr_reg->tg_pt_sep_rtpi, pr_reg->pr_aptpl_target_lun, diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index 9c4710f34d94..e4bdabfc4b31 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -64,7 +64,7 @@ extern sense_reason_t target_scsi2_reservation_reserve(struct se_cmd *); extern int core_scsi3_alloc_aptpl_registration( struct t10_reservation *, u64, unsigned char *, unsigned char *, u64, - unsigned char *, u16, u64, int, int, u8); + u16, u64, int, int, u8); extern int core_scsi3_check_aptpl_registration(struct se_device *, struct se_portal_group *, struct se_lun *, struct se_node_acl *, u64); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 1b673b25c6cd..095b96cb3557 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -347,11 +347,6 @@ struct t10_pr_registration { /* Used during APTPL metadata reading */ #define PR_APTPL_MAX_IPORT_LEN 256 unsigned char pr_iport[PR_APTPL_MAX_IPORT_LEN]; - /* Used during APTPL metadata reading */ -#define PR_APTPL_MAX_TPORT_LEN 256 - unsigned char pr_tport[PR_APTPL_MAX_TPORT_LEN]; - u16 pr_aptpl_rpti; - u16 pr_reg_tpgt; /* Reservation effects all target ports */ int pr_reg_all_tg_pt; /* Activate Persistence across Target Power Loss */ From patchwork Wed Jul 27 16:21:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2AD4C19F28 for ; Wed, 3 Aug 2022 16:30:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238365AbiHCQaD (ORCPT ); Wed, 3 Aug 2022 12:30:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238377AbiHCQ3e (ORCPT ); Wed, 3 Aug 2022 12:29:34 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B20FF48EAA; Wed, 3 Aug 2022 09:29:32 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 62655412DE; Wed, 3 Aug 2022 16:29:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544169; x=1661358570; bh=ZtkUYIiT6/VUOQLgeQB4CnRUx cqaT5oWQY8y1mEYyMw=; b=iqMknBiXmhWWjQIYeiBNKlLD84tE+P+625Eh35RHU NoMyYicWMpkURDqWoEaYhc4GiAwU+AHbTZ4qpau/B+eY6lTCxz+fUFTwuer0EKzi cydN66u8IGI5gia/d/tN83MguwLDYg1oz4XASveKtwmbGtSB4+ztyu/2JN5iaUZE kk= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SNVQ2N-9_AQQ; Wed, 3 Aug 2022 19:29:29 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 3D3E041239; Wed, 3 Aug 2022 19:29:19 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:19 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:18 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 25/48] target: core: pr: have Transport ID stored Date: Wed, 27 Jul 2022 19:21:02 +0300 Message-ID: <20220803162857.27770-26-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Store TransportID in pr_reg at its creation. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_fabric_lib.c | 130 +++++++++--------------- drivers/target/target_core_internal.h | 9 +- drivers/target/target_core_pr.c | 19 ++-- include/target/target_core_base.h | 2 + 4 files changed, 68 insertions(+), 92 deletions(-) diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index cd1360095036..32c2afbb3033 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -33,14 +33,14 @@ static int sas_get_pr_transport_id( - struct t10_pr_registration *pr_reg, - int *format_code, + const char *initiatorname, unsigned char *buf) { int ret; + buf[0] = SCSI_PROTOCOL_SAS; /* Skip over 'naa. prefix */ - ret = hex2bin(&buf[4], &pr_reg->pr_iport[4], 8); + ret = hex2bin(&buf[4], &initiatorname[4], 8); if (ret) { pr_debug("%s: invalid hex string\n", __func__); return ret; @@ -50,19 +50,19 @@ static int sas_get_pr_transport_id( } static int fc_get_pr_transport_id( - struct t10_pr_registration *pr_reg, - int *format_code, + const char *initiatorname, unsigned char *buf) { - unsigned char *ptr; + const char *ptr; int i, ret; u32 off = 8; + buf[0] = SCSI_PROTOCOL_FCP; /* * We convert the ASCII formatted N Port name into a binary * encoded TransportID. */ - ptr = &pr_reg->pr_iport[0]; + ptr = &initiatorname[0]; for (i = 0; i < 23; ) { if (!strncmp(&ptr[i], ":", 1)) { i++; @@ -82,13 +82,14 @@ static int fc_get_pr_transport_id( } static int sbp_get_pr_transport_id( - struct t10_pr_registration *pr_reg, - int *format_code, + const char *initiatorname, unsigned char *buf) { int ret; - ret = hex2bin(&buf[8], pr_reg->pr_iport, 8); + buf[0] = SCSI_PROTOCOL_SBP; + + ret = hex2bin(&buf[8], initiatorname, 8); if (ret) { pr_debug("%s: invalid hex string\n", __func__); return ret; @@ -98,15 +99,16 @@ static int sbp_get_pr_transport_id( } static int srp_get_pr_transport_id( - struct t10_pr_registration *pr_reg, - int *format_code, + const char *initiatorname, unsigned char *buf) { const char *p; unsigned len, count, leading_zero_bytes; int rc; - p = pr_reg->pr_iport; + buf[0] = SCSI_PROTOCOL_SRP; + + p = initiatorname; if (strncasecmp(p, "0x", 2) == 0) p += 2; len = strlen(p); @@ -126,14 +128,16 @@ static int srp_get_pr_transport_id( } static int iscsi_get_pr_transport_id( - struct t10_pr_registration *pr_reg, - int *format_code, + const char *initiatorname, + unsigned char *isid, unsigned char *buf) { u32 off = 4, padding = 0; int isid_len; u16 len = 0; + buf[0] = SCSI_PROTOCOL_ISCSI; + /* * Only null terminate the last field. * @@ -149,9 +153,9 @@ static int iscsi_get_pr_transport_id( * length of the iSCSI TransportID or the contents of the ADDITIONAL * LENGTH field. */ - len = sprintf(&buf[off], "%s", pr_reg->pr_iport); + len = sprintf(&buf[off], "%s", initiatorname); off += len; - if ((*format_code == 1) && (pr_reg->isid_present_at_reg)) { + if (isid) { /* * Set FORMAT CODE 01b for iSCSI Initiator port TransportID * format. @@ -185,7 +189,7 @@ static int iscsi_get_pr_transport_id( buf[off++] = 0x78; /* ASCII Character: "x" */ len += 5; - isid_len = sprintf(buf + off, "%s", pr_reg->pr_reg_isid); + isid_len = sprintf(buf + off, "%s", isid); off += isid_len; len += isid_len; } @@ -210,47 +214,6 @@ static int iscsi_get_pr_transport_id( return len; } -static int iscsi_get_pr_transport_id_len( - struct t10_pr_registration *pr_reg, - int *format_code) -{ - u32 len = 0, padding = 0; - - len = strlen(pr_reg->pr_iport); - /* - * Add extra byte for NULL terminator - */ - len++; - /* - * If there is ISID present with the registration, use format code: - * 01b: iSCSI Initiator port TransportID format - * - * If there is not an active iSCSI session, use format code: - * 00b: iSCSI Initiator device TransportID format - */ - if (pr_reg->isid_present_at_reg) { - len += 5; /* For ",i,0x" ASCII separator */ - len += strlen(pr_reg->pr_reg_isid); - *format_code = 1; - } else - *format_code = 0; - /* - * The ADDITIONAL LENGTH field specifies the number of bytes that follow - * in the TransportID. The additional length shall be at least 20 and - * shall be a multiple of four. - */ - padding = ((-len) & 3); - if (padding != 0) - len += padding; - /* - * Increment value for total payload + header length for - * full status descriptor - */ - len += 4; - - return len; -} - static char *iscsi_parse_pr_out_transport_id( struct se_portal_group *se_tpg, char *buf, @@ -336,47 +299,54 @@ static char *iscsi_parse_pr_out_transport_id( return &buf[4]; } -int target_get_pr_transport_id_len( - struct t10_pr_registration *pr_reg, int *format_code) +int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg) { - switch (pr_reg->se_tpg->proto_id) { + switch (pr_reg->pr_tid[0] & 0xF) { case SCSI_PROTOCOL_FCP: case SCSI_PROTOCOL_SBP: case SCSI_PROTOCOL_SRP: case SCSI_PROTOCOL_SAS: - break; + return 24; case SCSI_PROTOCOL_ISCSI: - return iscsi_get_pr_transport_id_len(pr_reg, format_code); + return get_unaligned_be16(&pr_reg->pr_tid[2]) + 4; default: - pr_err("Unknown proto_id: 0x%02x\n", pr_reg->se_tpg->proto_id); + WARN(1, "Unknown proto_id: %#x\n", pr_reg->pr_tid[0] & 0xF); return -EINVAL; } - - /* - * Most transports use a fixed length 24 byte identifier. - */ - *format_code = 0; - return 24; } int target_get_pr_transport_id( - struct t10_pr_registration *pr_reg, int *format_code, + struct t10_pr_registration *pr_reg, unsigned char *buf) { - switch (pr_reg->se_tpg->proto_id) { + int len = target_get_pr_transport_id_len(pr_reg); + + if (len > 0) + memcpy(buf, pr_reg->pr_tid, len); + + return len; +} + +int target_gen_pr_transport_id( + struct t10_pr_registration *pr_reg, + int proto_id, + const char *initiatorname, + unsigned char *isid) +{ + + switch (proto_id) { case SCSI_PROTOCOL_SAS: - return sas_get_pr_transport_id(pr_reg, format_code, buf); + return sas_get_pr_transport_id(initiatorname, pr_reg->pr_tid); case SCSI_PROTOCOL_SBP: - return sbp_get_pr_transport_id(pr_reg, format_code, buf); + return sbp_get_pr_transport_id(initiatorname, pr_reg->pr_tid); case SCSI_PROTOCOL_SRP: - return srp_get_pr_transport_id(pr_reg, format_code, buf); + return srp_get_pr_transport_id(initiatorname, pr_reg->pr_tid); case SCSI_PROTOCOL_FCP: - return fc_get_pr_transport_id(pr_reg, format_code, buf); + return fc_get_pr_transport_id(initiatorname, pr_reg->pr_tid); case SCSI_PROTOCOL_ISCSI: - return iscsi_get_pr_transport_id(pr_reg, format_code, - buf); + return iscsi_get_pr_transport_id(initiatorname, isid, pr_reg->pr_tid); default: - pr_err("Unknown proto_id: 0x%02x\n", pr_reg->se_tpg->proto_id); + pr_err("Unknown proto_id: 0x%02x\n", proto_id); return -EINVAL; } } diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index b32ac78a26b0..95d5add73578 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -98,10 +98,13 @@ void target_setup_backend_cits(struct target_backend *); int target_fabric_setup_cits(struct target_fabric_configfs *); /* target_core_fabric_lib.c */ -int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg, - int *format_code); +int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg); int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, - int *format_code, unsigned char *buf); + unsigned char *buf); +int target_gen_pr_transport_id(struct t10_pr_registration *pr_reg, + int proto_id, + const char *initiatorname, + unsigned char *isid); const char *target_parse_pr_out_transport_id(struct se_portal_group *tpg, char *buf, u32 *out_tid_len, char **port_nexus_ptr); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 1f19bfd0fa00..d9e7d177b65a 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -716,6 +716,10 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( aptpl); if (!pr_reg) return NULL; + + target_gen_pr_transport_id(pr_reg, nacl->se_tpg->proto_id, + nacl->initiatorname, isid); + /* * Return pointer to pr_reg for ALL_TG_PT=0 */ @@ -802,6 +806,9 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( core_scsi3_lunacl_undepend_item(deve_tmp); goto out; } + target_gen_pr_transport_id(pr_reg_atp, + nacl_tmp->se_tpg->proto_id, + nacl_tmp->initiatorname, isid); list_add_tail(&pr_reg_atp->pr_reg_atp_mem_list, &pr_reg->pr_reg_atp_list); @@ -3829,13 +3836,12 @@ static sense_reason_t core_scsi3_pri_read_full_status(struct se_cmd *cmd) { struct se_device *dev = cmd->se_dev; - struct se_portal_group *se_tpg; struct t10_pr_registration *pr_reg, *pr_reg_tmp; struct t10_reservation *pr_tmpl = &dev->t10_pr; unsigned char *buf; u32 add_desc_len = 0, add_len = 0; u32 off = 8; /* off into first Full Status descriptor */ - int format_code = 0, pr_res_type = 0, pr_res_scope = 0; + int pr_res_type = 0, pr_res_scope = 0; int exp_desc_len, desc_len; bool all_reg = false; @@ -3868,7 +3874,6 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) list_for_each_entry_safe(pr_reg, pr_reg_tmp, &pr_tmpl->registration_list, pr_reg_list) { - se_tpg = pr_reg->se_tpg; add_desc_len = 0; atomic_inc_mb(&pr_reg->pr_res_holders); @@ -3877,8 +3882,7 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) * Determine expected length of $FABRIC_MOD specific * TransportID full status descriptor.. */ - exp_desc_len = target_get_pr_transport_id_len(pr_reg, - &format_code); + exp_desc_len = target_get_pr_transport_id_len(pr_reg); if (exp_desc_len < 0 || exp_desc_len + add_len > cmd->data_length) { pr_warn("SPC-3 PRIN READ_FULL_STATUS ran" @@ -3938,13 +3942,10 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd) } else off += 2; /* Skip over RELATIVE TARGET PORT IDENTIFIER */ - buf[off+4] = se_tpg->proto_id; - /* * Now, have the $FABRIC_MOD fill in the transport ID. */ - desc_len = target_get_pr_transport_id(pr_reg, - &format_code, &buf[off+4]); + desc_len = target_get_pr_transport_id(pr_reg, &buf[off+4]); spin_lock(&pr_tmpl->registration_lock); atomic_dec_mb(&pr_reg->pr_res_holders); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 095b96cb3557..64cb943b5d3a 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -347,6 +347,8 @@ struct t10_pr_registration { /* Used during APTPL metadata reading */ #define PR_APTPL_MAX_IPORT_LEN 256 unsigned char pr_iport[PR_APTPL_MAX_IPORT_LEN]; +#define PR_REG_TID_LEN 228 + char pr_tid[PR_REG_TID_LEN]; /* Reservation effects all target ports */ int pr_reg_all_tg_pt; /* Activate Persistence across Target Power Loss */ From patchwork Wed Jun 22 10:25:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5D4FC19F29 for ; Wed, 3 Aug 2022 16:30:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236989AbiHCQaF (ORCPT ); Wed, 3 Aug 2022 12:30:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238388AbiHCQ3h (ORCPT ); Wed, 3 Aug 2022 12:29:37 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BCE4DEDB; Wed, 3 Aug 2022 09:29:35 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id D426041239; Wed, 3 Aug 2022 16:29:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544171; x=1661358572; bh=I3rp/BgsyFIeH4WQjR1Cm+2Hz +cW1bhj18c1hoF+Dr0=; b=NzMzlWtrJxthWjPtN4tvsX7O2lmfcERwDsg0l0Oks eUJ386ziM5my1bhinFZVdGCG8JQMOrIPY2YvFbML//7KsFbPy7Twnekm3q3Ur/fx Vmhq8wx+WF0ja2pmxQTuXPvfiRemqvXSJAxIDnzSsimcw6bW81n+1xjvWAwcqUIf EQ= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JLQjFTxNaGVO; Wed, 3 Aug 2022 19:29:31 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 81905412C9; Wed, 3 Aug 2022 19:29:19 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:19 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:18 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 26/48] target: core: pr: remove se_tpg from pr_reg Date: Wed, 22 Jun 2022 13:25:24 +0300 Message-ID: <20220803162857.27770-27-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Undepend pr_reg from se_tpg. Use RTPI at PR creation. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 47 ++++--- drivers/target/target_core_fabric_configfs.c | 2 +- drivers/target/target_core_internal.h | 1 + drivers/target/target_core_pr.c | 126 ++++++++----------- drivers/target/target_core_pr.h | 2 +- drivers/target/target_core_tpg.c | 17 ++- include/target/target_core_base.h | 1 - 7 files changed, 104 insertions(+), 92 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index f469cfde3f1e..0978279bd5e3 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1736,6 +1736,8 @@ static ssize_t target_core_dev_pr_show_spc3_res(struct se_device *dev, char *page) { struct t10_pr_registration *pr_reg; + struct se_portal_group *se_tpg; + const char *fabric_name = "Unknown"; char i_buf[PR_REG_ISID_ID_LEN] = { }; pr_reg = dev->dev_pr_res_holder; @@ -1744,9 +1746,12 @@ static ssize_t target_core_dev_pr_show_spc3_res(struct se_device *dev, core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); + se_tpg = core_get_tpg_by_rtpi(pr_reg->tg_pt_sep_rtpi); + if (se_tpg) + fabric_name = se_tpg->se_tpg_tfo->fabric_name; + return sprintf(page, "SPC-3 Reservation: %s Initiator: %s%s\n", - pr_reg->se_tpg->se_tpg_tfo->fabric_name, - pr_reg->pr_iport, i_buf); + fabric_name, pr_reg->pr_iport, i_buf); } static ssize_t target_core_dev_pr_show_spc2_res(struct se_device *dev, @@ -1838,18 +1843,24 @@ static ssize_t target_pr_res_pr_holder_tg_port_show(struct config_item *item, goto out_unlock; } - se_tpg = pr_reg->se_tpg; - tfo = se_tpg->se_tpg_tfo; - - len += sprintf(page+len, "SPC-3 Reservation: %s" - " Target Node Endpoint: %s\n", tfo->fabric_name, - tfo->tpg_get_wwn(se_tpg)); - len += sprintf(page+len, "SPC-3 Reservation: Relative Port" - " Identifier Tag: %hu %s Portal Group Tag: %hu" - " %s Logical Unit: %llu\n", pr_reg->tg_pt_sep_rtpi, - tfo->fabric_name, tfo->tpg_get_tag(se_tpg), - tfo->fabric_name, pr_reg->pr_aptpl_target_lun); + se_tpg = core_get_tpg_by_rtpi(pr_reg->tg_pt_sep_rtpi); + if (se_tpg) { + tfo = se_tpg->se_tpg_tfo; + len += sprintf(page+len, "SPC-3 Reservation: %s Target Node Endpoint: %s\n", + tfo->fabric_name, tfo->tpg_get_wwn(se_tpg)); + len += sprintf(page+len, "SPC-3 Reservation: Relative Port Identifier Tag: %hu %s Portal Group Tag: %hu %s Logical Unit: %llu\n", + pr_reg->tg_pt_sep_rtpi, + tfo->fabric_name, tfo->tpg_get_tag(se_tpg), + tfo->fabric_name, pr_reg->pr_aptpl_target_lun); + } else { + len += sprintf(page+len, "SPC-3 Reservation: %s Target Node Endpoint: %s\n", + "Unknown", "Unknown"); + len += sprintf(page+len, "SPC-3 Reservation: Relative Port Identifier Tag: %hu %s Portal Group Tag: %hu %s Logical Unit: %llu\n", + pr_reg->tg_pt_sep_rtpi, + "Unknown", 0, + "Unknown", pr_reg->pr_aptpl_target_lun); + } out_unlock: spin_unlock(&dev->dev_reservation_lock); return len; @@ -1860,8 +1871,9 @@ static ssize_t target_pr_res_pr_registered_i_pts_show(struct config_item *item, char *page) { struct se_device *dev = pr_to_dev(item); - const struct target_core_fabric_ops *tfo; struct t10_pr_registration *pr_reg; + const char *fabric_name = "Unknown"; + struct se_portal_group *se_tpg; unsigned char buf[384]; char i_buf[PR_REG_ISID_ID_LEN]; ssize_t len = 0; @@ -1875,11 +1887,14 @@ static ssize_t target_pr_res_pr_registered_i_pts_show(struct config_item *item, memset(buf, 0, 384); memset(i_buf, 0, PR_REG_ISID_ID_LEN); - tfo = pr_reg->se_tpg->se_tpg_tfo; + se_tpg = core_get_tpg_by_rtpi(pr_reg->tg_pt_sep_rtpi); + if (se_tpg) + fabric_name = se_tpg->se_tpg_tfo->fabric_name; + core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); sprintf(buf, "%s Node: %s%s Key: 0x%016Lx PRgen: 0x%08x\n", - tfo->fabric_name, + fabric_name, pr_reg->pr_iport, i_buf, pr_reg->pr_res_key, pr_reg->pr_res_generation); diff --git a/drivers/target/target_core_fabric_configfs.c b/drivers/target/target_core_fabric_configfs.c index fc1b8f54fb54..fd8f3b4c4bf8 100644 --- a/drivers/target/target_core_fabric_configfs.c +++ b/drivers/target/target_core_fabric_configfs.c @@ -871,7 +871,7 @@ static ssize_t target_fabric_tpg_base_enable_store(struct config_item *item, spin_lock(&g_tpg_lock); if (op) { - tpg = core_get_tpg_by_rtpi(se_tpg->tpg_rtpi); + tpg = _core_get_tpg_by_rtpi(se_tpg->tpg_rtpi); if (tpg) { spin_unlock(&g_tpg_lock); diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 95d5add73578..51c0151c05c2 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -136,6 +136,7 @@ void core_tpg_remove_lun(struct se_portal_group *, struct se_lun *); struct se_node_acl *core_tpg_add_initiator_node_acl(struct se_portal_group *tpg, const char *initiatorname); void core_tpg_del_initiator_node_acl(struct se_node_acl *acl); +struct se_portal_group *_core_get_tpg_by_rtpi(u16 rtpi); struct se_portal_group *core_get_tpg_by_rtpi(u16 rtpi); /* target_core_transport.c */ diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index d9e7d177b65a..c4ee6999cf96 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -612,7 +612,7 @@ struct t10_pr_registration *__core_scsi3_do_alloc_registration( struct se_node_acl *nacl, const char *initiatorname, u64 unpacked_lun, - struct se_portal_group *se_tpg, + u16 rtpi, struct se_dev_entry *dest_deve, u64 mapped_lun, unsigned char *isid, @@ -661,8 +661,7 @@ struct t10_pr_registration *__core_scsi3_do_alloc_registration( } pr_reg->pr_res_mapped_lun = mapped_lun; pr_reg->pr_aptpl_target_lun = unpacked_lun; - pr_reg->tg_pt_sep_rtpi = se_tpg->tpg_rtpi; - pr_reg->se_tpg = se_tpg; + pr_reg->tg_pt_sep_rtpi = rtpi; pr_reg->pr_res_key = sa_res_key; pr_reg->pr_reg_all_tg_pt = all_tg_pt; pr_reg->pr_reg_aptpl = aptpl; @@ -710,7 +709,7 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( */ pr_reg = __core_scsi3_do_alloc_registration(dev, nacl, nacl->initiatorname, lun->unpacked_lun, - lun->lun_tpg, + lun->lun_tpg->tpg_rtpi, deve, mapped_lun, isid, sa_res_key, all_tg_pt, aptpl); @@ -797,7 +796,7 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( nacl_tmp, nacl_tmp->initiatorname, dest_lun->unpacked_lun, - lun->lun_tpg, + lun->lun_tpg->tpg_rtpi, deve_tmp, deve_tmp->mapped_lun, NULL, sa_res_key, all_tg_pt, aptpl); @@ -1011,30 +1010,25 @@ int core_scsi3_check_aptpl_registration( } static void __core_scsi3_dump_registration( - const struct target_core_fabric_ops *tfo, struct se_device *dev, struct t10_pr_registration *pr_reg, enum register_type register_type) { - struct se_portal_group *se_tpg = pr_reg->se_tpg; char i_buf[PR_REG_ISID_ID_LEN] = { }; core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); - pr_debug("SPC-3 PR [%s] Service Action: REGISTER%s Initiator" - " Node: %s%s\n", tfo->fabric_name, (register_type == REGISTER_AND_MOVE) ? + pr_debug("SPC-3 PR Service Action: REGISTER%s Initiator Node: %s%s\n", + (register_type == REGISTER_AND_MOVE) ? "_AND_MOVE" : (register_type == REGISTER_AND_IGNORE_EXISTING_KEY) ? "_AND_IGNORE_EXISTING_KEY" : "", pr_reg->pr_iport, i_buf); - pr_debug("SPC-3 PR [%s] registration on Target Port: %s,0x%04x\n", - tfo->fabric_name, tfo->tpg_get_wwn(se_tpg), - tfo->tpg_get_tag(se_tpg)); - pr_debug("SPC-3 PR [%s] for %s TCM Subsystem %s Object Target" - " Port(s)\n", tfo->fabric_name, + pr_debug("SPC-3 PR registration on Target Port: %d\n", + pr_reg->tg_pt_sep_rtpi); + pr_debug("SPC-3 PR for %s TCM Subsystem %s Object Target Port(s)\n", (pr_reg->pr_reg_all_tg_pt) ? "ALL" : "SINGLE", dev->transport->name); - pr_debug("SPC-3 PR [%s] SA Res Key: 0x%016Lx PRgeneration:" - " 0x%08x APTPL: %d\n", tfo->fabric_name, + pr_debug("SPC-3 PR SA Res Key: 0x%016llx PRgeneration: 0x%08x APTPL: %d\n", pr_reg->pr_res_key, pr_reg->pr_res_generation, pr_reg->pr_reg_aptpl); } @@ -1046,7 +1040,6 @@ static void __core_scsi3_add_registration( enum register_type register_type, int register_move) { - const struct target_core_fabric_ops *tfo = nacl->se_tpg->se_tpg_tfo; struct t10_pr_registration *pr_reg_tmp, *pr_reg_tmp_safe; struct t10_reservation *pr_tmpl = &dev->t10_pr; struct se_dev_entry *deve; @@ -1060,7 +1053,7 @@ static void __core_scsi3_add_registration( spin_lock(&pr_tmpl->registration_lock); list_add_tail(&pr_reg->pr_reg_list, &pr_tmpl->registration_list); - __core_scsi3_dump_registration(tfo, dev, pr_reg, register_type); + __core_scsi3_dump_registration(dev, pr_reg, register_type); spin_unlock(&pr_tmpl->registration_lock); /* * Skip extra processing for ALL_TG_PT=0 or REGISTER_AND_MOVE. @@ -1081,8 +1074,7 @@ static void __core_scsi3_add_registration( list_add_tail(&pr_reg_tmp->pr_reg_list, &pr_tmpl->registration_list); - __core_scsi3_dump_registration(tfo, dev, pr_reg_tmp, - register_type); + __core_scsi3_dump_registration(dev, pr_reg_tmp, register_type); spin_unlock(&pr_tmpl->registration_lock); /* * Drop configfs group dependency reference and deve->pr_kref @@ -1262,8 +1254,6 @@ void __core_scsi3_free_registration( __releases(&pr_tmpl->registration_lock) __acquires(&pr_tmpl->registration_lock) { - const struct target_core_fabric_ops *tfo = - pr_reg->se_tpg->se_tpg_tfo; struct t10_reservation *pr_tmpl = &dev->t10_pr; struct se_node_acl *nacl = pr_reg->pr_reg_nacl; struct se_dev_entry *deve; @@ -1290,8 +1280,7 @@ void __core_scsi3_free_registration( * count back to zero, and we release *pr_reg. */ while (atomic_read(&pr_reg->pr_res_holders) != 0) { - pr_debug("SPC-3 PR [%s] waiting for pr_res_holders\n", - tfo->fabric_name); + pr_debug("SPC-3 PR waiting for pr_res_holders\n"); cpu_relax(); } @@ -1302,15 +1291,13 @@ void __core_scsi3_free_registration( rcu_read_unlock(); spin_lock(&pr_tmpl->registration_lock); - pr_debug("SPC-3 PR [%s] Service Action: UNREGISTER Initiator" - " Node: %s%s\n", tfo->fabric_name, - pr_reg->pr_iport, i_buf); - pr_debug("SPC-3 PR [%s] for %s TCM Subsystem %s Object Target" - " Port(s)\n", tfo->fabric_name, + pr_debug("SPC-3 PR Service Action: UNREGISTER Initiator Node: %s%s\n", + pr_reg->pr_iport, i_buf); + pr_debug("SPC-3 PR for %s TCM Subsystem %s Object Target Port(s)\n", (pr_reg->pr_reg_all_tg_pt) ? "ALL" : "SINGLE", dev->transport->name); - pr_debug("SPC-3 PR [%s] SA Res Key: 0x%016Lx PRgeneration:" - " 0x%08x\n", tfo->fabric_name, pr_reg->pr_res_key, + pr_debug("SPC-3 PR SA Res Key: 0x%016llx PRgeneration: 0x%08x\n", + pr_reg->pr_res_key, pr_reg->pr_res_generation); if (!preempt_and_abort_list) { @@ -1830,7 +1817,9 @@ static int core_scsi3_update_aptpl_buf( tmp[0] = '\0'; isid_buf[0] = '\0'; - tpg = pr_reg->se_tpg; + + tpg = core_get_tpg_by_rtpi(pr_reg->tg_pt_sep_rtpi); + /* * Write out any ISID value to APTPL metadata that was included * in the original registration. @@ -1874,13 +1863,21 @@ static int core_scsi3_update_aptpl_buf( /* * Include information about the associated SCSI target port. */ - snprintf(tmp, 512, "target_node=%s\n" - "tpgt=%hu\nport_rtpi=%hu\ntarget_lun=%llu\nPR_REG_END: %d\n", - tpg->se_tpg_tfo->tpg_get_wwn(tpg), - tpg->se_tpg_tfo->tpg_get_tag(tpg), - pr_reg->tg_pt_sep_rtpi, pr_reg->pr_aptpl_target_lun, - reg_count); - + if (tpg) { + snprintf(tmp, 512, "target_node=%s\n" + "tpgt=%hu\nport_rtpi=%hu\ntarget_lun=%llu\nPR_REG_END: %d\n", + tpg->se_tpg_tfo->tpg_get_wwn(tpg), + tpg->se_tpg_tfo->tpg_get_tag(tpg), + pr_reg->tg_pt_sep_rtpi, pr_reg->pr_aptpl_target_lun, + reg_count); + } else { + snprintf(tmp, 512, "target_node=%s\n" + "tpgt=%hu\nport_rtpi=%hu\ntarget_lun=%llu\nPR_REG_END: %d\n", + "", + 0, + pr_reg->tg_pt_sep_rtpi, pr_reg->pr_aptpl_target_lun, + reg_count); + } if ((len + strlen(tmp) >= pr_aptpl_buf_len)) { pr_err("Unable to update renaming APTPL metadata," " reallocating larger buffer\n"); @@ -2224,12 +2221,10 @@ void __core_scsi3_set_reservation(struct se_device *dev, dev->dev_pr_res_holder = pr_reg; core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); - pr_debug("SPC-3 PR [%s] Service Action: RESERVE created new reservation holder TYPE: %s ALL_TG_PT: %d\n", - pr_reg->se_tpg->se_tpg_tfo->fabric_name, + pr_debug("SPC-3 PR Service Action: RESERVE created new reservation holder TYPE: %s ALL_TG_PT: %d\n", core_scsi3_pr_dump_type(type), (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); - pr_debug("SPC-3 PR [%s] RESERVE Node: %s%s\n", - pr_reg->se_tpg->se_tpg_tfo->fabric_name, + pr_debug("SPC-3 PR RESERVE Node: %s%s\n", pr_reg->pr_iport, i_buf); } @@ -2308,12 +2303,10 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) * RESERVATION CONFLICT status. */ if (!is_reservation_holder(pr_res_holder, pr_reg)) { - pr_err("SPC-3 PR: Attempted RESERVE from" - " [%s]: %s while reservation already held by" - " [%s]: %s, returning RESERVATION_CONFLICT\n", - cmd->se_tfo->fabric_name, + pr_err("SPC-3 PR: Attempted RESERVE from [%d]: %s while reservation already held by [%d]: %s, returning RESERVATION_CONFLICT\n", + cmd->se_lun->lun_tpg->tpg_rtpi, se_sess->se_node_acl->initiatorname, - pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->tg_pt_sep_rtpi, pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); @@ -2329,13 +2322,10 @@ core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) */ if ((pr_res_holder->pr_res_type != type) || (pr_res_holder->pr_res_scope != scope)) { - pr_err("SPC-3 PR: Attempted RESERVE from" - " [%s]: %s trying to change TYPE and/or SCOPE," - " while reservation already held by [%s]: %s," - " returning RESERVATION_CONFLICT\n", - cmd->se_tfo->fabric_name, + pr_err("SPC-3 PR: Attempted RESERVE from [%d]: %s trying to change TYPE and/or SCOPE, while reservation already held by [%d]: %s, returning RESERVATION_CONFLICT\n", + cmd->se_lun->lun_tpg->tpg_rtpi, se_sess->se_node_acl->initiatorname, - pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->tg_pt_sep_rtpi, pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); @@ -2397,7 +2387,6 @@ void __core_scsi3_complete_pro_release( int explicit, int unreg) { - const struct target_core_fabric_ops *tfo = pr_reg->se_tpg->se_tpg_tfo; char i_buf[PR_REG_ISID_ID_LEN] = { }; int pr_res_type = 0, pr_res_scope = 0; @@ -2442,15 +2431,14 @@ void __core_scsi3_complete_pro_release( spin_unlock(&dev->t10_pr.registration_lock); out: if (!dev->dev_pr_res_holder) { - pr_debug("SPC-3 PR [%s] Service Action: %s RELEASE cleared" + pr_debug("SPC-3 PR Service Action: %s RELEASE cleared" " reservation holder TYPE: %s ALL_TG_PT: %d\n", - tfo->fabric_name, (explicit) ? "explicit" : + (explicit) ? "explicit" : "implicit", core_scsi3_pr_dump_type(pr_res_type), (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); } - pr_debug("SPC-3 PR [%s] RELEASE Node: %s%s\n", - tfo->fabric_name, pr_reg->pr_iport, - i_buf); + pr_debug("SPC-3 PR RELEASE Node: %s%s\n", + pr_reg->pr_iport, i_buf); /* * Clear TYPE and SCOPE for the next PROUT Service Action: RESERVE */ @@ -2542,13 +2530,10 @@ core_scsi3_emulate_pro_release(struct se_cmd *cmd, int type, int scope, */ if ((pr_res_holder->pr_res_type != type) || (pr_res_holder->pr_res_scope != scope)) { - pr_err("SPC-3 PR RELEASE: Attempted to release" - " reservation from [%s]: %s with different TYPE " - "and/or SCOPE while reservation already held by" - " [%s]: %s, returning RESERVATION_CONFLICT\n", - cmd->se_tfo->fabric_name, + pr_err("SPC-3 PR RELEASE: Attempted to release reservation from [%d]: %s with different TYPE and/or SCOPE while reservation already held by [%d]: %s, returning RESERVATION_CONFLICT\n", + cmd->se_lun->lun_tpg->tpg_rtpi, se_sess->se_node_acl->initiatorname, - pr_res_holder->se_tpg->se_tpg_tfo->fabric_name, + pr_res_holder->tg_pt_sep_rtpi, pr_res_holder->pr_iport); spin_unlock(&dev->dev_reservation_lock); @@ -2701,7 +2686,6 @@ static void __core_scsi3_complete_pro_preempt( int scope, enum preempt_type preempt_type) { - const struct target_core_fabric_ops *tfo = pr_reg->se_tpg->se_tpg_tfo; char i_buf[PR_REG_ISID_ID_LEN] = { }; lockdep_assert_held(&dev->dev_reservation_lock); @@ -2718,13 +2702,13 @@ static void __core_scsi3_complete_pro_preempt( pr_reg->pr_res_type = type; pr_reg->pr_res_scope = scope; - pr_debug("SPC-3 PR [%s] Service Action: PREEMPT%s created new" + pr_debug("SPC-3 PR Service Action: PREEMPT%s created new" " reservation holder TYPE: %s ALL_TG_PT: %d\n", - tfo->fabric_name, (preempt_type == PREEMPT_AND_ABORT) ? "_AND_ABORT" : "", + (preempt_type == PREEMPT_AND_ABORT) ? "_AND_ABORT" : "", core_scsi3_pr_dump_type(type), (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); - pr_debug("SPC-3 PR [%s] PREEMPT%s from Node: %s%s\n", - tfo->fabric_name, (preempt_type == PREEMPT_AND_ABORT) ? "_AND_ABORT" : "", + pr_debug("SPC-3 PR PREEMPT%s from Node: %s%s\n", + (preempt_type == PREEMPT_AND_ABORT) ? "_AND_ABORT" : "", pr_reg->pr_iport, i_buf); /* * For PREEMPT_AND_ABORT, add the preempting reservation's diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index e4bdabfc4b31..954b0dff32bf 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -86,7 +86,7 @@ struct t10_pr_registration *__core_scsi3_do_alloc_registration( struct se_node_acl *nacl, const char *initiatorname, u64 unpacked_lun, - struct se_portal_group *se_tpg, + u16 rtpi, struct se_dev_entry *dest_deve, u64 mapped_lun, unsigned char *isid, diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c index 5a81d592530f..bdcd65ce0f62 100644 --- a/drivers/target/target_core_tpg.c +++ b/drivers/target/target_core_tpg.c @@ -738,8 +738,9 @@ static ssize_t core_tpg_rtpi_show(struct config_item *item, char *page) return sprintf(page, "%#x\n", se_tpg->tpg_rtpi); } + struct se_portal_group * -core_get_tpg_by_rtpi(u16 rtpi) +_core_get_tpg_by_rtpi(u16 rtpi) { struct se_portal_group *tpg; @@ -753,6 +754,18 @@ core_get_tpg_by_rtpi(u16 rtpi) return NULL; } +struct se_portal_group * +core_get_tpg_by_rtpi(u16 rtpi) +{ + struct se_portal_group *tpg; + + spin_lock(&g_tpg_lock); + tpg = _core_get_tpg_by_rtpi(rtpi); + spin_unlock(&g_tpg_lock); + + return tpg; +} + static ssize_t core_tpg_rtpi_store(struct config_item *item, const char *page, size_t count) { @@ -773,7 +786,7 @@ static ssize_t core_tpg_rtpi_store(struct config_item *item, spin_lock(&g_tpg_lock); if (se_tpg->enabled) { - tpg = core_get_tpg_by_rtpi(val); + tpg = _core_get_tpg_by_rtpi(val); if (tpg) { spin_unlock(&g_tpg_lock); pr_err("TARGET_CORE[%s]->TPG[%u] - RTPI %#x conflicts with TARGET_CORE[%s]->TPG[%u]\n", diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 64cb943b5d3a..5378e2d3825f 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -366,7 +366,6 @@ struct t10_pr_registration { u64 pr_res_key; atomic_t pr_res_holders; struct se_node_acl *pr_reg_nacl; - struct se_portal_group *se_tpg; /* Used by ALL_TG_PT=1 registration with deve->pr_ref taken */ struct se_dev_entry *pr_reg_deve; struct list_head pr_reg_list; From patchwork Wed Jun 29 07:50:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC83EC19F2C for ; Wed, 3 Aug 2022 16:30:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236354AbiHCQaG (ORCPT ); Wed, 3 Aug 2022 12:30:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238391AbiHCQ3i (ORCPT ); Wed, 3 Aug 2022 12:29:38 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5466459B6; Wed, 3 Aug 2022 09:29:36 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 7A7AD412C9; Wed, 3 Aug 2022 16:29:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544173; x=1661358574; bh=TCXblozmIVmynphH0B4NyTbP9 sorYOmcrwF9Vmk/j2s=; b=h248h8LPmIFcudGiZ/r1U1LJIWz4Ydh0vKMjp/Nwb KUqYumI8dKpw0n+z5Jdil4mCHP2yfOrRvqhG5bvMCroNj5UTd5U/J/FX5AApCU7o YrUcNcwFV2LBUb0l07idcrwfnacK/JBb6NYRfrKtrfHUXe7Pc+A9rmz2nPQFh8mI iE= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9W0VOue28IP1; Wed, 3 Aug 2022 19:29:33 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id C9161412E1; Wed, 3 Aug 2022 19:29:19 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:19 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:19 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 27/48] target: core: fix parsing PR OUT TID Date: Wed, 29 Jun 2022 10:50:28 +0300 Message-ID: <20220803162857.27770-28-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Now only iSCSI TransportID is parsed. From other protocols the binary representation is passed to tcm core. That is wrong as that TID is compared against nacl->initiatorname which is in human readable format. This patch adds a parsing of TID for other protocols. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_fabric_lib.c | 119 ++++++++++++++++-------- drivers/target/target_core_internal.h | 4 +- drivers/target/target_core_pr.c | 14 ++- 3 files changed, 89 insertions(+), 48 deletions(-) diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 32c2afbb3033..34fdab558e24 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -214,15 +214,67 @@ static int iscsi_get_pr_transport_id( return len; } -static char *iscsi_parse_pr_out_transport_id( - struct se_portal_group *se_tpg, - char *buf, - u32 *out_tid_len, +static u32 sas_parse_pr_out_transport_id( + const char *buf, + char *initiatorname, + char **port_nexus_ptr) +{ + char port_name[17]; + + bin2hex(port_name, buf + 4, 8); + snprintf(initiatorname, 36, "naa.%s", port_name); + + return 24; +} + + +static u32 sbp_parse_pr_out_transport_id( + const char *buf, + char *initiatorname, + char **port_nexus_ptr) +{ + bin2hex(initiatorname, buf + 8, 8); + + return 24; +} + +static u32 srp_parse_pr_out_transport_id( + const char *buf, + char *initiatorname, + char **port_nexus_ptr) +{ + char port_name[33]; + + bin2hex(port_name, buf + 8, 16); + snprintf(initiatorname, 36, "0x%s", port_name); + + return 24; +} + +static u32 fc_parse_pr_out_transport_id( + const char *buf, + char *initiatorname, + char **port_nexus_ptr) +{ + /* + * Format the FCP Initiator port_name into colon separated values to + * match the format by tcm_qla2xxx explicit ConfigFS NodeACLs. + */ + snprintf(initiatorname, 36, "%8phC", buf + 8); + + return 24; +} + +u32 iscsi_parse_pr_out_transport_id( + const char *buf, + char *initiatorname, char **port_nexus_ptr) { char *p; int i; u8 format_code = (buf[0] & 0xc0); + u32 out_tid_len; + /* * Check for FORMAT CODE 00b or 01b from spc4r17, section 7.5.4.6: * @@ -239,18 +291,10 @@ static char *iscsi_parse_pr_out_transport_id( if ((format_code != 0x00) && (format_code != 0x40)) { pr_err("Illegal format code: 0x%02x for iSCSI" " Initiator Transport ID\n", format_code); - return NULL; - } - /* - * If the caller wants the TransportID Length, we set that value for the - * entire iSCSI Tarnsport ID now. - */ - if (out_tid_len) { - /* The shift works thanks to integer promotion rules */ - *out_tid_len = get_unaligned_be16(&buf[2]); - /* Add four bytes for iSCSI Transport ID header */ - *out_tid_len += 4; + return 0; } + /* The shift works thanks to integer promotion rules */ + out_tid_len = get_unaligned_be16(&buf[2]); /* * Check for ',i,0x' separator between iSCSI Name and iSCSI Initiator @@ -263,9 +307,10 @@ static char *iscsi_parse_pr_out_transport_id( pr_err("Unable to locate \",i,0x\" separator" " for Initiator port identifier: %s\n", &buf[4]); - return NULL; + return 0; } - *p = '\0'; /* Terminate iSCSI Name */ + memcpy(initiatorname, &buf[4], p - buf - 4); + initiatorname[p - buf - 4] = '\0'; /* Terminate iSCSI Name */ p += 5; /* Skip over ",i,0x" separator */ *port_nexus_ptr = p; @@ -293,10 +338,16 @@ static char *iscsi_parse_pr_out_transport_id( *p = tolower(*p); p++; } - } else + } else { + strscpy(initiatorname, &buf[4], out_tid_len); + initiatorname[p - buf] = '\0'; /* Terminate iSCSI Name */ *port_nexus_ptr = NULL; + } + + /* Add four bytes for iSCSI Transport ID header */ + out_tid_len += 4; - return &buf[4]; + return out_tid_len; } int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg) @@ -351,33 +402,25 @@ int target_gen_pr_transport_id( } } -const char *target_parse_pr_out_transport_id(struct se_portal_group *tpg, - char *buf, u32 *out_tid_len, char **port_nexus_ptr) +u32 target_parse_pr_out_transport_id( + const char *buf, char *initiatorname, char **port_nexus_ptr) { - u32 offset; - - switch (tpg->proto_id) { + switch (buf[0] & 0xF) { case SCSI_PROTOCOL_SAS: - /* - * Assume the FORMAT CODE 00b from spc4r17, 7.5.4.7 TransportID - * for initiator ports using SCSI over SAS Serial SCSI Protocol. - */ - offset = 4; - break; + return sas_parse_pr_out_transport_id(buf, initiatorname, port_nexus_ptr); case SCSI_PROTOCOL_SBP: + return sbp_parse_pr_out_transport_id(buf, initiatorname, port_nexus_ptr); case SCSI_PROTOCOL_SRP: + return srp_parse_pr_out_transport_id(buf, initiatorname, port_nexus_ptr); case SCSI_PROTOCOL_FCP: - offset = 8; - break; + return fc_parse_pr_out_transport_id(buf, initiatorname, port_nexus_ptr); case SCSI_PROTOCOL_ISCSI: - return iscsi_parse_pr_out_transport_id(tpg, buf, out_tid_len, - port_nexus_ptr); + return iscsi_parse_pr_out_transport_id(buf, initiatorname, port_nexus_ptr); default: - pr_err("Unknown proto_id: 0x%02x\n", tpg->proto_id); - return NULL; + pr_err("Unknown proto_id: 0x%02x\n", buf[0] & 0xF); + return 0; } *port_nexus_ptr = NULL; - *out_tid_len = 24; - return buf + offset; + return 24; } diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 51c0151c05c2..bbdddcd83fa4 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -105,8 +105,8 @@ int target_gen_pr_transport_id(struct t10_pr_registration *pr_reg, int proto_id, const char *initiatorname, unsigned char *isid); -const char *target_parse_pr_out_transport_id(struct se_portal_group *tpg, - char *buf, u32 *out_tid_len, char **port_nexus_ptr); +u32 target_parse_pr_out_transport_id( + const char *buf, char *initiatorname, char **port_nexus_ptr); /* target_core_hba.c */ struct se_hba *core_alloc_hba(const char *, u32, u32); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index c4ee6999cf96..f5796d60ed76 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -1419,7 +1419,7 @@ core_scsi3_decode_spec_i_port( LIST_HEAD(tid_dest_list); struct pr_transport_id_holder *tidh_new, *tidh, *tidh_tmp; unsigned char *buf, *ptr, proto_ident; - const unsigned char *i_str = NULL; + unsigned char i_str[TRANSPORT_IQN_LEN] = {0}; char *iport_ptr = NULL, i_buf[PR_REG_ISID_ID_LEN]; sense_reason_t ret; u32 tpdl, tid_len = 0; @@ -1512,9 +1512,8 @@ core_scsi3_decode_spec_i_port( dest_rtpi = tmp_lun->lun_tpg->tpg_rtpi; iport_ptr = NULL; - i_str = target_parse_pr_out_transport_id(tmp_tpg, - ptr, &tid_len, &iport_ptr); - if (!i_str) + tid_len = target_parse_pr_out_transport_id(ptr, i_str, &iport_ptr); + if (!tid_len) continue; /* * Determine if this SCSI device server requires that @@ -3087,7 +3086,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, struct t10_pr_registration *pr_reg, *pr_res_holder, *dest_pr_reg; struct t10_reservation *pr_tmpl = &dev->t10_pr; unsigned char *buf; - const unsigned char *initiator_str; + unsigned char initiator_str[TRANSPORT_IQN_LEN] = { }; char *iport_ptr = NULL, i_buf[PR_REG_ISID_ID_LEN] = { }; u32 tid_len, tmp_tid_len; int new_reg = 0, type, scope, matching_iname; @@ -3212,9 +3211,8 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key, ret = TCM_INVALID_PARAMETER_LIST; goto out; } - initiator_str = target_parse_pr_out_transport_id(dest_se_tpg, - &buf[24], &tmp_tid_len, &iport_ptr); - if (!initiator_str) { + tmp_tid_len = target_parse_pr_out_transport_id(&buf[24], initiator_str, &iport_ptr); + if (!tmp_tid_len) { pr_err("SPC-3 PR REGISTER_AND_MOVE: Unable to locate" " initiator_str from Transport ID\n"); ret = TCM_INVALID_PARAMETER_LIST; From patchwork Wed Jun 29 10:06:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACC6FC19F28 for ; Wed, 3 Aug 2022 16:30:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237313AbiHCQaJ (ORCPT ); Wed, 3 Aug 2022 12:30:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238237AbiHCQ3j (ORCPT ); Wed, 3 Aug 2022 12:29:39 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBFBA474F4; Wed, 3 Aug 2022 09:29:37 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id A18BD412E1; Wed, 3 Aug 2022 16:29:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544175; x=1661358576; bh=jubGnQCrnCDFL33MMWXxIHBDx I+VOZF7E1/ZWoaPbsw=; b=SwZnMUBM8ToAfRvUWEPoAu3htANkVUvh//TB32N1J QlHTwJgLUL1lfKnuAluKe8F++y9yFjh1XuKl+ZM9nWDqj2JGFXH6Peu3Me2gATmu 1sJzRvEf8/k6mOWUWSx04cBUAbSBGzdn8P/V3LvGJiwcWrHuF/9CQVghJ2gqsNre Gw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MiMwK-Ux0DeK; Wed, 3 Aug 2022 19:29:35 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 026C841283; Wed, 3 Aug 2022 19:29:20 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:20 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:19 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 28/48] target: core: add function to compare TransportID Date: Wed, 29 Jun 2022 13:06:41 +0300 Message-ID: <20220803162857.27770-29-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_fabric_lib.c | 27 +++++++++++++++++++++---- drivers/target/target_core_internal.h | 2 ++ 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 34fdab558e24..0eb03d7b9e78 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -350,22 +350,27 @@ u32 iscsi_parse_pr_out_transport_id( return out_tid_len; } -int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg) +static int __target_get_pr_transport_id_len(const char *tid) { - switch (pr_reg->pr_tid[0] & 0xF) { + switch (tid[0] & 0xF) { case SCSI_PROTOCOL_FCP: case SCSI_PROTOCOL_SBP: case SCSI_PROTOCOL_SRP: case SCSI_PROTOCOL_SAS: return 24; case SCSI_PROTOCOL_ISCSI: - return get_unaligned_be16(&pr_reg->pr_tid[2]) + 4; + return get_unaligned_be16(&tid[2]) + 4; default: - WARN(1, "Unknown proto_id: %#x\n", pr_reg->pr_tid[0] & 0xF); + WARN(1, "Unknown proto_id: %#x\n", tid[0] & 0xF); return -EINVAL; } } +int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg) +{ + return __target_get_pr_transport_id_len(pr_reg->pr_tid); +} + int target_get_pr_transport_id( struct t10_pr_registration *pr_reg, unsigned char *buf) @@ -378,6 +383,20 @@ int target_get_pr_transport_id( return len; } +int target_cmp_pr_transport_id( + struct t10_pr_registration *pr_reg, + unsigned char *tid) +{ + int len1 = __target_get_pr_transport_id_len(pr_reg->pr_tid); + int len2 = __target_get_pr_transport_id_len(tid); + + if (len1 != len2) + return 1; + + return memcmp(pr_reg->pr_tid, tid, len1); +} + + int target_gen_pr_transport_id( struct t10_pr_registration *pr_reg, int proto_id, diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index bbdddcd83fa4..b0d29c30ab1e 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -101,6 +101,8 @@ int target_fabric_setup_cits(struct target_fabric_configfs *); int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg); int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, unsigned char *buf); +int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, + unsigned char *tid); int target_gen_pr_transport_id(struct t10_pr_registration *pr_reg, int proto_id, const char *initiatorname, From patchwork Thu Jun 30 11:11:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595244 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA96C19F29 for ; Wed, 3 Aug 2022 16:30:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238491AbiHCQaM (ORCPT ); Wed, 3 Aug 2022 12:30:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238362AbiHCQ3n (ORCPT ); Wed, 3 Aug 2022 12:29:43 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18CB646D9A; Wed, 3 Aug 2022 09:29:42 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id B4BDC412D1; Wed, 3 Aug 2022 16:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544179; x=1661358580; bh=CfEYe0eUF+xFUwUpdFGO4qS65 rJQg7BGNpQzrHb6DGU=; b=wIf3GVhPyomicjYN4mYHfHfljmJmSQfi/xNNZEN5N GSw4s783D6yPP1p9/sIBSJb1I13OYa1ifFoDhWVl1dO5Mfzp8H8GewElSwEwpRYU lcu9RVp+aUHeHjxBYAJ0wHPS9kuPqGlexPTFIGgp4G5JPTW28nbC1t9LbRUYvUyF DA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yQ9jNpZhM58j; Wed, 3 Aug 2022 19:29:39 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 7625D412D8; Wed, 3 Aug 2022 19:29:20 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:20 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:19 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 29/48] target: core: store proto_id in APTPL Date: Thu, 30 Jun 2022 14:11:01 +0300 Message-ID: <20220803162857.27770-30-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Store and read proto_id in APTPL to create TransportID. That will allow to report registrations for not yet created TPG/ACL/LUNs Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 13 +++++++++++-- drivers/target/target_core_pr.c | 15 ++++++++++----- drivers/target/target_core_pr.h | 2 +- 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index 0978279bd5e3..f63c37344104 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -1974,7 +1974,8 @@ enum { Opt_initiator_fabric, Opt_initiator_node, Opt_initiator_sid, Opt_sa_res_key, Opt_res_holder, Opt_res_type, Opt_res_scope, Opt_res_all_tg_pt, Opt_mapped_lun, Opt_target_fabric, - Opt_target_node, Opt_tpgt, Opt_port_rtpi, Opt_target_lun, Opt_err + Opt_target_node, Opt_tpgt, Opt_port_rtpi, Opt_target_lun, + Opt_proto_id, Opt_err }; static match_table_t tokens = { @@ -1992,6 +1993,7 @@ static match_table_t tokens = { {Opt_tpgt, "tpgt=%d"}, {Opt_port_rtpi, "port_rtpi=%d"}, {Opt_target_lun, "target_lun=%u"}, + {Opt_proto_id, "proto_id=%u"}, {Opt_err, NULL} }; @@ -2010,6 +2012,7 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, u16 tpgt = 0; u16 rtpi = 0; u8 type = 0; + u8 proto_id = 0; if (!dev->dev_attrib.emulate_pr || (dev->transport_flags & TRANSPORT_FLAG_PASSTHROUGH_PGR)) @@ -2151,6 +2154,12 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, goto out; target_lun = (u64)tmp_ll; break; + case Opt_proto_id: + ret = match_int(args, &arg); + if (ret) + goto out; + proto_id = (u16)arg; + break; default: break; } @@ -2171,7 +2180,7 @@ static ssize_t target_pr_res_aptpl_metadata_store(struct config_item *item, ret = core_scsi3_alloc_aptpl_registration(&dev->t10_pr, sa_res_key, i_port, isid, mapped_lun, rtpi, target_lun, - res_holder, all_tg_pt, type); + res_holder, all_tg_pt, type, proto_id); out: kfree(i_fabric); kfree(i_port); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index f5796d60ed76..d177db3024c9 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -842,7 +842,8 @@ int core_scsi3_alloc_aptpl_registration( u64 target_lun, int res_holder, int all_tg_pt, - u8 type) + u8 type, + u8 proto_id) { struct t10_pr_registration *pr_reg; @@ -886,6 +887,7 @@ int core_scsi3_alloc_aptpl_registration( * Copy the i_port information from caller. */ snprintf(pr_reg->pr_iport, PR_APTPL_MAX_IPORT_LEN, "%s", i_port); + target_gen_pr_transport_id(pr_reg, proto_id, i_port, isid); /* * Set pr_res_holder from caller, the pr_reg who is the reservation * holder will get it's pointer set in core_scsi3_aptpl_reserve() once @@ -1836,19 +1838,22 @@ static int core_scsi3_update_aptpl_buf( "sa_res_key=%llu\n" "res_holder=1\nres_type=%02x\n" "res_scope=%02x\nres_all_tg_pt=%d\n" - "mapped_lun=%llu\n", reg_count, + "mapped_lun=%llu\nproto_id=%u\n", reg_count, pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_res_type, pr_reg->pr_res_scope, pr_reg->pr_reg_all_tg_pt, - pr_reg->pr_res_mapped_lun); + pr_reg->pr_res_mapped_lun, + pr_reg->pr_tid[0] & 0xF); } else { snprintf(tmp, 512, "PR_REG_START: %d\n" "initiator_node=%s\n%s" "sa_res_key=%llu\nres_holder=0\n" - "res_all_tg_pt=%d\nmapped_lun=%llu\n", + "res_all_tg_pt=%d\nmapped_lun=%llu\n" + "proto_id=%u\n", reg_count, pr_reg->pr_iport, isid_buf, pr_reg->pr_res_key, pr_reg->pr_reg_all_tg_pt, - pr_reg->pr_res_mapped_lun); + pr_reg->pr_res_mapped_lun, + pr_reg->pr_tid[0] & 0xF); } if ((len + strlen(tmp) >= pr_aptpl_buf_len)) { diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index 954b0dff32bf..589fdac5470f 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -64,7 +64,7 @@ extern sense_reason_t target_scsi2_reservation_reserve(struct se_cmd *); extern int core_scsi3_alloc_aptpl_registration( struct t10_reservation *, u64, unsigned char *, unsigned char *, u64, - u16, u64, int, int, u8); + u16, u64, int, int, u8, u8); extern int core_scsi3_check_aptpl_registration(struct se_device *, struct se_portal_group *, struct se_lun *, struct se_node_acl *, u64); From patchwork Fri Jul 1 07:26:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E2A5C19F2C for ; Wed, 3 Aug 2022 16:30:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238115AbiHCQaK (ORCPT ); Wed, 3 Aug 2022 12:30:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238222AbiHCQ3p (ORCPT ); Wed, 3 Aug 2022 12:29:45 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A312946DBF; Wed, 3 Aug 2022 09:29:43 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 4202441319; Wed, 3 Aug 2022 16:29:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544180; x=1661358581; bh=wgioVz2EmvpJGetHzuYpI2nLZ hLhNwegLTtp00ImE8k=; b=JFGJdBIEf2zphq/PbZf4/hX0pypFdufuv4aa3uzgF eYAtrkVOGkFGSVGizDxbGmDTw9CMc3TTvFFuBwmTviFocs/qOtuG88juIejwG0Vm vKKDvksT3exGye6x/GFhV0gBtjCGCbZTo8BcjfyNPpirzNzPft5waezRllQ6tPN6 Ok= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PD38rZ6KE3fa; Wed, 3 Aug 2022 19:29:40 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id D92EF41284; Wed, 3 Aug 2022 19:29:20 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:20 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:20 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 30/48] target: core: rethink APTPL registrations Date: Fri, 1 Jul 2022 10:26:25 +0300 Message-ID: <20220803162857.27770-31-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Make APTPL registrations be real registrations from the very beginning. Just mark at deve creation that it has a PR registration if it exists. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_pr.c | 104 +++++++++++++------------------- 1 file changed, 43 insertions(+), 61 deletions(-) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index d177db3024c9..a984987f0a82 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -832,6 +832,28 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration( return NULL; } +static void __core_scsi3_add_registration(struct se_device *, struct se_node_acl *, + struct t10_pr_registration *, enum register_type, int); + +static void core_scsi3_aptpl_reserve( + struct se_device *dev, + const char *initiatorname, + struct t10_pr_registration *pr_reg) +{ + char i_buf[PR_REG_ISID_ID_LEN] = { }; + + core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); + + spin_lock(&dev->dev_reservation_lock); + dev->dev_pr_res_holder = pr_reg; + spin_unlock(&dev->dev_reservation_lock); + + pr_debug("SPC-3 PR Service Action: APTPL RESERVE created new reservation holder TYPE: %s ALL_TG_PT: %d\n", + core_scsi3_pr_dump_type(pr_reg->pr_res_type), + (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); + pr_debug("SPC-3 PR RESERVE Node: %s%s\n", initiatorname, i_buf); +} + int core_scsi3_alloc_aptpl_registration( struct t10_reservation *pr_tmpl, u64 sa_res_key, @@ -845,6 +867,7 @@ int core_scsi3_alloc_aptpl_registration( u8 type, u8 proto_id) { + struct se_device *dev = container_of(pr_tmpl, struct se_device, t10_pr); struct t10_pr_registration *pr_reg; if (!i_port || !sa_res_key) { @@ -888,47 +911,24 @@ int core_scsi3_alloc_aptpl_registration( */ snprintf(pr_reg->pr_iport, PR_APTPL_MAX_IPORT_LEN, "%s", i_port); target_gen_pr_transport_id(pr_reg, proto_id, i_port, isid); + + pr_reg->pr_res_holder = res_holder; + __core_scsi3_add_registration(dev, NULL, pr_reg, REGISTER, 0); + + if (pr_reg->pr_res_holder) + core_scsi3_aptpl_reserve(dev, i_port, pr_reg); + /* - * Set pr_res_holder from caller, the pr_reg who is the reservation - * holder will get it's pointer set in core_scsi3_aptpl_reserve() once - * the Initiator Node LUN ACL from the fabric module is created for - * this registration. + * Reenable pr_aptpl_active to accept new metadata + * updates once the SCSI device is active again.. */ - pr_reg->pr_res_holder = res_holder; + pr_tmpl->pr_aptpl_active = 1; - list_add_tail(&pr_reg->pr_reg_aptpl_list, &pr_tmpl->aptpl_reg_list); pr_debug("SPC-3 PR APTPL Successfully added registration%s from" " metadata\n", (res_holder) ? "+reservation" : ""); return 0; } -static void core_scsi3_aptpl_reserve( - struct se_device *dev, - struct se_portal_group *tpg, - struct se_node_acl *node_acl, - struct t10_pr_registration *pr_reg) -{ - char i_buf[PR_REG_ISID_ID_LEN] = { }; - - core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN); - - spin_lock(&dev->dev_reservation_lock); - dev->dev_pr_res_holder = pr_reg; - spin_unlock(&dev->dev_reservation_lock); - - pr_debug("SPC-3 PR [%s] Service Action: APTPL RESERVE created" - " new reservation holder TYPE: %s ALL_TG_PT: %d\n", - tpg->se_tpg_tfo->fabric_name, - core_scsi3_pr_dump_type(pr_reg->pr_res_type), - (pr_reg->pr_reg_all_tg_pt) ? 1 : 0); - pr_debug("SPC-3 PR [%s] RESERVE Node: %s%s\n", - tpg->se_tpg_tfo->fabric_name, node_acl->initiatorname, - i_buf); -} - -static void __core_scsi3_add_registration(struct se_device *, struct se_node_acl *, - struct t10_pr_registration *, enum register_type, int); - static int __core_scsi3_check_aptpl_registration( struct se_device *dev, struct se_portal_group *tpg, @@ -946,12 +946,13 @@ static int __core_scsi3_check_aptpl_registration( * may exist for fabrics that use ISIDs in their SCSI Initiator Port * TransportIDs. */ - spin_lock(&pr_tmpl->aptpl_reg_lock); - list_for_each_entry_safe(pr_reg, pr_reg_tmp, &pr_tmpl->aptpl_reg_list, - pr_reg_aptpl_list) { + spin_lock(&pr_tmpl->registration_lock); + list_for_each_entry_safe(pr_reg, pr_reg_tmp, + &pr_tmpl->registration_list, pr_reg_list) { - if (!strcmp(pr_reg->pr_iport, nacl->initiatorname) && - (pr_reg->tg_pt_sep_rtpi == tpg->tpg_rtpi)) { + if (pr_reg->pr_reg_nacl == NULL && + !strcmp(pr_reg->pr_iport, nacl->initiatorname) && + (pr_reg->tg_pt_sep_rtpi == tpg->tpg_rtpi)) { /* * Obtain the ->pr_reg_deve pointer + reference, that * is released by __core_scsi3_add_registration() below. @@ -964,34 +965,15 @@ static int __core_scsi3_check_aptpl_registration( rcu_read_unlock(); continue; } - kref_get(&pr_reg->pr_reg_deve->pr_kref); + set_bit(DEF_PR_REG_ACTIVE, &pr_reg->pr_reg_deve->deve_flags); + pr_reg->pr_reg_deve = NULL; rcu_read_unlock(); pr_reg->pr_reg_nacl = nacl; - pr_reg->tg_pt_sep_rtpi = lun->lun_tpg->tpg_rtpi; - list_del(&pr_reg->pr_reg_aptpl_list); - spin_unlock(&pr_tmpl->aptpl_reg_lock); - /* - * At this point all of the pointers in *pr_reg will - * be setup, so go ahead and add the registration. - */ - __core_scsi3_add_registration(dev, nacl, pr_reg, 0, 0); - /* - * If this registration is the reservation holder, - * make that happen now.. - */ - if (pr_reg->pr_res_holder) - core_scsi3_aptpl_reserve(dev, tpg, - nacl, pr_reg); - /* - * Reenable pr_aptpl_active to accept new metadata - * updates once the SCSI device is active again.. - */ - spin_lock(&pr_tmpl->aptpl_reg_lock); - pr_tmpl->pr_aptpl_active = 1; + pr_reg->pr_res_mapped_lun = mapped_lun; } } - spin_unlock(&pr_tmpl->aptpl_reg_lock); + spin_unlock(&pr_tmpl->registration_lock); return 0; } From patchwork Thu Nov 18 07:52:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D8C4C19F2D for ; Wed, 3 Aug 2022 16:30:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238442AbiHCQaP (ORCPT ); Wed, 3 Aug 2022 12:30:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236361AbiHCQ3r (ORCPT ); Wed, 3 Aug 2022 12:29:47 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4E68481EE; Wed, 3 Aug 2022 09:29:45 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 376294131D; Wed, 3 Aug 2022 16:29:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544182; x=1661358583; bh=3V8PNmQx6JMWrGXlflRLkYHv1 l+opJDmmQGxfMLHRRQ=; b=p8rn44AoQJmAFK+qgsTAiF/72iFrfz3JobEofbje7 zH0+Irt+PIcRHHFtGHOpeiS/jNpb9V7+8V81J02Zgc3osHGfrHVanRnTptepvNXT VMBHs19vCrXdC4hOlkXzdDNw61Bwxbt7FkSIQtUlFfXbM9hSErYTR2oFuyd0duMO 8U= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 03OqYSBL-xlQ; Wed, 3 Aug 2022 19:29:42 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1F5AA412D6; Wed, 3 Aug 2022 19:29:21 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:21 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:20 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 31/48] dlm_ckv: introduce DLM cluster key-value storage Date: Thu, 18 Nov 2021 10:52:33 +0300 Message-ID: <20220803162857.27770-32-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce the first version of DLM CKV module that could be used by different kernel subsystems to share some information in a cluster. This commit has just cluster level locks. Signed-off-by: Dmitry Bogdanov --- drivers/target/Kconfig | 6 + drivers/target/Makefile | 2 + drivers/target/dlm_ckv.c | 323 +++++++++++++++++++++++++++++++++++++++ drivers/target/dlm_ckv.h | 19 +++ 4 files changed, 350 insertions(+) create mode 100644 drivers/target/dlm_ckv.c create mode 100644 drivers/target/dlm_ckv.h diff --git a/drivers/target/Kconfig b/drivers/target/Kconfig index 72171ea3dd53..75d5e1d23a1c 100644 --- a/drivers/target/Kconfig +++ b/drivers/target/Kconfig @@ -35,6 +35,12 @@ config TCM_PSCSI Say Y here to enable the TCM/pSCSI subsystem plugin for non-buffered passthrough access to Linux/SCSI device +config DLM_CKV + tristate "Cluster key value storage over DLM" + depends on DLM + help + Say Y here to enable the cluster key value storage over DLM + config TCM_USER2 tristate "TCM/USER Subsystem Plugin for Linux" depends on UIO && NET diff --git a/drivers/target/Makefile b/drivers/target/Makefile index 45634747377e..8bc9ac2bd629 100644 --- a/drivers/target/Makefile +++ b/drivers/target/Makefile @@ -30,3 +30,5 @@ obj-$(CONFIG_LOOPBACK_TARGET) += loopback/ obj-$(CONFIG_TCM_FC) += tcm_fc/ obj-$(CONFIG_ISCSI_TARGET) += iscsi/ obj-$(CONFIG_SBP_TARGET) += sbp/ + +obj-$(CONFIG_DLM_CKV) += dlm_ckv.o diff --git a/drivers/target/dlm_ckv.c b/drivers/target/dlm_ckv.c new file mode 100644 index 000000000000..a2e1a191c433 --- /dev/null +++ b/drivers/target/dlm_ckv.c @@ -0,0 +1,323 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dlm_ckv.h" + +struct dlm_ckv_lksb { + struct dlm_lksb lksb; + struct completion compl; +}; + +struct dlm_ckv_lock { + struct dlm_ckv_bucket *bucket; + struct dlm_ckv_lksb lksb; + char name[DLM_RESNAME_MAXLEN]; +}; + +struct dlm_ckv_bucket { + dlm_lockspace_t *ls; + struct kref refcount; + u32 local_nodeid; + u32 local_slotid; + size_t num_nodes; + int nodeid[64]; + void *userarg; + struct completion sync_compl; +}; + + +#define DLM_CKV_LVB_SIZE 256 + +static void bucket_release(struct kref *ref); + +/* dlm calls before it does lock recovery */ + +static void dlm_ckv_recover_prep(void *arg) +{ + +} + +/* dlm calls after recover_prep has been completed on all lockspace members; + * identifies slot/jid of failed member + */ + +static void dlm_ckv_recover_slot(void *arg, struct dlm_slot *slot) +{ + pr_info("nodeid %d left the cluster\n", slot->nodeid); +} + +/* dlm calls after recover_slot and after it completes lock recovery */ + +static void dlm_ckv_recover_done(void *arg, struct dlm_slot *slots, int num_slots, + int our_slot, uint32_t generation) +{ + struct dlm_ckv_bucket *bucket = arg; + int i; + + for (i = 0; i < num_slots; i++) { + bucket->nodeid[i] = slots[i].nodeid; + if (slots[i].slot == our_slot) + bucket->local_nodeid = slots[i].nodeid; + } + bucket->local_slotid = our_slot; + bucket->num_nodes = num_slots; + complete(&bucket->sync_compl); +} + +static const struct dlm_lockspace_ops dlm_ckv_lockspace_ops = { + .recover_prep = dlm_ckv_recover_prep, + .recover_slot = dlm_ckv_recover_slot, + .recover_done = dlm_ckv_recover_done, +}; + +static void dlm_ast(void *astarg) +{ + struct dlm_ckv_lksb *dlm_ckv_lksb = astarg; + + complete(&dlm_ckv_lksb->compl); +} + +/* + * dlm_ckv_cancel - Synchronously cancel a pending dlm_lock() operation + */ +static int dlm_ckv_cancel(dlm_lockspace_t *ls, struct dlm_ckv_lksb *lksb, + int flags, const char *name) +{ + int res; + + res = dlm_unlock(ls, lksb->lksb.sb_lkid, + DLM_LKF_CANCEL | (flags & DLM_LKF_VALBLK), + &lksb->lksb, lksb); + if (res < 0) + goto out; + res = wait_for_completion_timeout(&lksb->compl, 10 * HZ); + +out: + return res; +} + +/** + * dlm_ckv_lock_wait - Wait until a DLM lock has been granted + * @ls: DLM lock space. + * @mode: DLM lock mode. + * @lksb: DLM lock status block. + * @flags: DLM flags. + * @name: DLM lock name. Only required for non-conversion requests. + * @bast: AST to be invoked in case this lock blocks another one. + */ +static int dlm_ckv_lock_wait(dlm_lockspace_t *ls, int mode, + struct dlm_ckv_lksb *lksb, int flags, + const char *name, void (*bast)(void *, int)) +{ + int res; + + res = dlm_lock(ls, mode, &lksb->lksb, flags, + (void *)name, name ? strlen(name) : 0, 0, + dlm_ast, lksb, bast); + if (res < 0) + goto out; + res = wait_for_completion_timeout(&lksb->compl, 60 * HZ); + if (res > 0) + res = lksb->lksb.sb_status; + else if (res == 0) + res = -ETIMEDOUT; + if (res < 0) { + int res2 = dlm_ckv_cancel(ls, lksb, flags, name); + + if (res2 < 0) + pr_warn("canceling lock %s / %08x failed: %d\n", + name ? : "?", lksb->lksb.sb_lkid, res2); + } + +out: + return res; +} + +/* + * dlm_ckv_unlock_wait - Release a DLM lock + */ +static int dlm_ckv_unlock_wait(dlm_lockspace_t *ls, struct dlm_ckv_lksb *lksb) +{ + int res; + + res = dlm_unlock(ls, lksb->lksb.sb_lkid, 0, &lksb->lksb, lksb); + if (res < 0) + goto out; + res = wait_for_completion_timeout(&lksb->compl, 60 * HZ); + if (res > 0) { + res = lksb->lksb.sb_status; + if (res == -DLM_EUNLOCK || res == -DLM_ECANCEL) + res = 0; + } else if (res == 0) { + res = -ETIMEDOUT; + } + +out: + return res; +} + +static void +dlm_ckv_lock_init(struct dlm_ckv_lock *ckv_lock, + struct dlm_ckv_bucket *bucket, + const char *name) +{ + init_completion(&ckv_lock->lksb.compl); + strscpy(ckv_lock->name, name, DLM_RESNAME_MAXLEN); + ckv_lock->bucket = bucket; +} + +struct dlm_ckv_lock * +dlm_ckv_create_lock(struct dlm_ckv_bucket *bucket, const char *name) +{ + struct dlm_ckv_lock *ckv_lock; + + ckv_lock = kzalloc(sizeof(struct dlm_ckv_lock), GFP_KERNEL); + if (!ckv_lock) + return NULL; + + kref_get(&bucket->refcount); + dlm_ckv_lock_init(ckv_lock, bucket, name); + + return ckv_lock; +} +EXPORT_SYMBOL(dlm_ckv_create_lock); + +void +dlm_ckv_free_lock(struct dlm_ckv_lock *ckv_lock) +{ + struct dlm_ckv_bucket *bucket = ckv_lock->bucket; + + kfree(ckv_lock); + + kref_put(&bucket->refcount, bucket_release); +} +EXPORT_SYMBOL(dlm_ckv_free_lock); + +int +dlm_ckv_lock_get(struct dlm_ckv_lock *ckv_lock) +{ + int res; + + BUG_ON(!ckv_lock); + + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_EX, + &ckv_lock->lksb, 0, ckv_lock->name, NULL); + + return res; +} +EXPORT_SYMBOL(dlm_ckv_lock_get); + +int +dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock) +{ + int res; + + BUG_ON(!ckv_lock); + + res = dlm_ckv_unlock_wait(ckv_lock->bucket->ls, &ckv_lock->lksb); + + return res; +} +EXPORT_SYMBOL(dlm_ckv_lock_release); + + +static void bucket_release(struct kref *ref) +{ + struct dlm_ckv_bucket *bucket = container_of(ref, struct dlm_ckv_bucket, + refcount); + int res; + + res = dlm_release_lockspace(bucket->ls, 2); + if (res) + pr_err("forcibly releasing lockspace failed: %d\n", + res); + + kfree(bucket); +} + +struct dlm_ckv_bucket * +dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) +{ + struct dlm_ckv_bucket *bucket; + int name_len = strlen(name); + int ops_result; + int err; + + if (!name) + return ERR_PTR(-EINVAL); + + if (name_len > DLM_LOCKSPACE_LEN) + return ERR_PTR(-EINVAL); + + bucket = kzalloc(sizeof(struct dlm_ckv_bucket), GFP_KERNEL); + kref_init(&bucket->refcount); + + bucket->userarg = userarg; + init_completion(&bucket->sync_compl); + + err = dlm_new_lockspace(name, cluster_name, + DLM_LSFL_FS | DLM_LSFL_NEWEXCL, DLM_CKV_LVB_SIZE, + &dlm_ckv_lockspace_ops, bucket, &ops_result, + &bucket->ls); + if (err) { + pr_err("dlm_new_lockspace error %d\n", err); + goto fail_free; + } + + if (ops_result < 0) { + pr_err("dlm does not support ops callbacks\n"); + err = -EOPNOTSUPP; + goto fail_free; + } + + wait_for_completion_timeout(&bucket->sync_compl, 10 * HZ); + if (bucket->num_nodes == 0) { + pr_err("Cluster joining timed out\n"); + goto fail_init; + } + + return bucket; + +fail_init: + dlm_release_lockspace(bucket->ls, 2); +fail_free: + kfree(bucket); + + return NULL; +} +EXPORT_SYMBOL(dlm_ckv_open_bucket); + +int dlm_ckv_close_bucket(struct dlm_ckv_bucket *bucket) +{ + kref_put(&bucket->refcount, bucket_release); + + return 0; +} +EXPORT_SYMBOL(dlm_ckv_close_bucket); + +static int __init dlm_ckv_module_init(void) +{ + return 0; +} + +static void __exit dlm_ckv_module_exit(void) +{ + +} + +MODULE_DESCRIPTION("Cluster KV storage over DLM"); +MODULE_AUTHOR("Dmitry Bogdanov "); +MODULE_LICENSE("GPL"); + +module_init(dlm_ckv_module_init); +module_exit(dlm_ckv_module_exit); diff --git a/drivers/target/dlm_ckv.h b/drivers/target/dlm_ckv.h new file mode 100644 index 000000000000..1a3f79e42bf6 --- /dev/null +++ b/drivers/target/dlm_ckv.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef DLM_CKV_H +#define DLM_CKV_H + +struct dlm_ckv_bucket; +struct dlm_ckv_lock; + +struct dlm_ckv_bucket *dlm_ckv_open_bucket(const char *name, + const char *cluster_name, + void *userarg); +int dlm_ckv_close_bucket(struct dlm_ckv_bucket *bucket); + +struct dlm_ckv_lock * +dlm_ckv_create_lock(struct dlm_ckv_bucket *bucket, const char *name); +void dlm_ckv_free_lock(struct dlm_ckv_lock *ckv_lock); +int dlm_ckv_lock_get(struct dlm_ckv_lock *ckv_lock); +int dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock); + +#endif From patchwork Mon Nov 22 17:07:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DACFC19F28 for ; Wed, 3 Aug 2022 16:30:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238523AbiHCQaS (ORCPT ); Wed, 3 Aug 2022 12:30:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238413AbiHCQ3s (ORCPT ); Wed, 3 Aug 2022 12:29:48 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C974167D4; Wed, 3 Aug 2022 09:29:46 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id BB8F54131B; Wed, 3 Aug 2022 16:29:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544183; x=1661358584; bh=JCrp/0ZWIAF1PCh2kq7jIhBKx KyR4CV/wOZ0aRubvw4=; b=bQKmR/MKZiCe616HheFN7X80plgLh/7StAEJcpfUJ yN8QMLwyf4oPm/isood6CRf6JLY8eWtiIgFRDCZozwVO/vDvr/R7VFJECtBgAx1I w4GgRnM+rv54rT6H/libLSeiScunCfOLZGXThKvReovv4PEfya3fTXU3RCQkm6Rv 7o= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zXCii_bqgW6q; Wed, 3 Aug 2022 19:29:43 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 4331D412C6; Wed, 3 Aug 2022 19:29:21 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:21 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:20 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 32/48] dlm_ckv: add notification service Date: Mon, 22 Nov 2021 20:07:17 +0300 Message-ID: <20220803162857.27770-33-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Notification broadcasts over DLM cluster. They do not have any payload. Used to be used to notify other nodes to read an updated data. Signed-off-by: Dmitry Bogdanov --- drivers/target/dlm_ckv.c | 186 +++++++++++++++++++++++++++++++++++++++ drivers/target/dlm_ckv.h | 8 ++ 2 files changed, 194 insertions(+) diff --git a/drivers/target/dlm_ckv.c b/drivers/target/dlm_ckv.c index a2e1a191c433..cffe4f2dcb82 100644 --- a/drivers/target/dlm_ckv.c +++ b/drivers/target/dlm_ckv.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include "dlm_ckv.h" @@ -33,8 +34,17 @@ struct dlm_ckv_bucket { int nodeid[64]; void *userarg; struct completion sync_compl; + struct workqueue_struct *notify_wq; }; +struct dlm_ckv_notify { + dlm_ckv_notify_cb notify_cb; + char name[DLM_RESNAME_MAXLEN]; + struct dlm_ckv_lock pre_n; + struct dlm_ckv_lock post_n; + struct work_struct pre_n_work; + struct work_struct post_n_work; +}; #define DLM_CKV_LVB_SIZE 256 @@ -230,6 +240,179 @@ dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock) } EXPORT_SYMBOL(dlm_ckv_lock_release); +static void dlm_cvk_pre_n_bast(void *astarg, int mode) +{ + struct dlm_ckv_lksb *lksb = astarg; + struct dlm_ckv_lock *ckv_lock; + struct dlm_ckv_bucket *bucket; + struct dlm_ckv_notify *notify; + + ckv_lock = container_of(lksb, struct dlm_ckv_lock, lksb); + notify = container_of(ckv_lock, struct dlm_ckv_notify, pre_n); + bucket = ckv_lock->bucket; + + queue_work(bucket->notify_wq, ¬ify->pre_n_work); +} + +static void dlm_cvk_post_n_bast(void *astarg, int mode) +{ + struct dlm_ckv_lksb *lksb = astarg; + struct dlm_ckv_lock *ckv_lock; + struct dlm_ckv_bucket *bucket; + struct dlm_ckv_notify *notify; + + ckv_lock = container_of(lksb, struct dlm_ckv_lock, lksb); + notify = container_of(ckv_lock, struct dlm_ckv_notify, post_n); + bucket = ckv_lock->bucket; + + queue_work(bucket->notify_wq, ¬ify->post_n_work); +} + +static void +dlm_cvk_pre_n_work(struct work_struct *work) +{ + struct dlm_ckv_notify *notify = container_of(work, + struct dlm_ckv_notify, pre_n_work); + struct dlm_ckv_bucket *bucket = notify->pre_n.bucket; + + dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_EX, + ¬ify->post_n.lksb, + DLM_LKF_CONVERT, + notify->post_n.name, dlm_cvk_post_n_bast); + dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_NL, + ¬ify->pre_n.lksb, + DLM_LKF_CONVERT, + notify->pre_n.name, dlm_cvk_pre_n_bast); +} + +static void +dlm_cvk_post_n_work(struct work_struct *work) +{ + struct dlm_ckv_notify *notify = container_of(work, + struct dlm_ckv_notify, post_n_work); + struct dlm_ckv_bucket *bucket = notify->post_n.bucket; + + dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_EX, + ¬ify->pre_n.lksb, + DLM_LKF_CONVERT, + notify->pre_n.name, dlm_cvk_pre_n_bast); + + notify->notify_cb(bucket->userarg); + + dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_NL, + ¬ify->post_n.lksb, + DLM_LKF_CONVERT, + notify->post_n.name, dlm_cvk_post_n_bast); +} + +struct dlm_ckv_notify * +dlm_ckv_create_notification(struct dlm_ckv_bucket *bucket, const char *name, + dlm_ckv_notify_cb cb) +{ + char lockname[DLM_RESNAME_MAXLEN]; + struct dlm_ckv_notify *notify; + int res; + + notify = kzalloc(sizeof(struct dlm_ckv_bucket), GFP_KERNEL); + if (!notify) + return NULL; + + INIT_WORK(¬ify->post_n_work, dlm_cvk_post_n_work); + INIT_WORK(¬ify->pre_n_work, dlm_cvk_pre_n_work); + + strscpy(notify->name, name, DLM_RESNAME_MAXLEN); + notify->notify_cb = cb; + + kref_get(&bucket->refcount); + + sprintf(lockname, "pre.%s.%d", name, bucket->local_nodeid); + dlm_ckv_lock_init(¬ify->pre_n, bucket, lockname); + + sprintf(lockname, "post.%s.%d", name, bucket->local_nodeid); + dlm_ckv_lock_init(¬ify->post_n, bucket, lockname); + + res = dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_EX, + ¬ify->pre_n.lksb, 0, + notify->pre_n.name, dlm_cvk_pre_n_bast); + if (res) + goto fail_locks; + + res = dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_NL, + ¬ify->post_n.lksb, 0, + notify->post_n.name, dlm_cvk_post_n_bast); + if (res) + goto fail_locks; + + return notify; + +fail_locks: + kref_put(&bucket->refcount, bucket_release); + kfree(notify); + + return NULL; +} +EXPORT_SYMBOL(dlm_ckv_create_notification); + +void +dlm_ckv_free_notification(struct dlm_ckv_notify *notify) +{ + struct dlm_ckv_bucket *bucket = notify->pre_n.bucket; + + cancel_work_sync(¬ify->pre_n_work); + cancel_work_sync(¬ify->post_n_work); + + kfree(notify); + kref_put(&bucket->refcount, bucket_release); +} +EXPORT_SYMBOL(dlm_ckv_free_notification); + +static void +dlm_ckv_toggle_lock(struct dlm_ckv_bucket *bucket, const char *name_prefix) +{ + char lockname[DLM_RESNAME_MAXLEN]; + struct dlm_ckv_lksb lksb; + int res; + int i; + + init_completion(&lksb.compl); + + for (i = 0; i < bucket->num_nodes; ++i) { + if (bucket->nodeid[i] == bucket->local_nodeid) + continue; + snprintf(lockname, sizeof(lockname), "%s.%d", name_prefix, + bucket->nodeid[i]); + + lksb.lksb.sb_lkid = 0; + res = dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_PR, + &lksb, 0, lockname, NULL); + if (res < 0) + pr_warn("Locking %s failed (%d)", lockname, res); + if (!lksb.lksb.sb_lkid) + continue; + + dlm_ckv_lock_wait(bucket->ls, DLM_LOCK_NL, &lksb, + DLM_LKF_CONVERT, lockname, NULL); + dlm_ckv_unlock_wait(bucket->ls, &lksb); + } +} + +int dlm_ckv_notify(struct dlm_ckv_notify *notify) +{ + struct dlm_ckv_bucket *bucket = notify->pre_n.bucket; + char lockname[DLM_RESNAME_MAXLEN]; + + snprintf(lockname, sizeof(lockname), "post.%s", notify->name); + dlm_ckv_toggle_lock(bucket, lockname); + + snprintf(lockname, sizeof(lockname), "pre.%s", notify->name); + dlm_ckv_toggle_lock(bucket, lockname); + + snprintf(lockname, sizeof(lockname), "post.%s", notify->name); + dlm_ckv_toggle_lock(bucket, lockname); + + return 0; +} +EXPORT_SYMBOL(dlm_ckv_notify); static void bucket_release(struct kref *ref) { @@ -237,6 +420,8 @@ static void bucket_release(struct kref *ref) refcount); int res; + destroy_workqueue(bucket->notify_wq); + res = dlm_release_lockspace(bucket->ls, 2); if (res) pr_err("forcibly releasing lockspace failed: %d\n", @@ -264,6 +449,7 @@ dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) bucket->userarg = userarg; init_completion(&bucket->sync_compl); + bucket->notify_wq = alloc_ordered_workqueue("notify_wq-%s", 0, name); err = dlm_new_lockspace(name, cluster_name, DLM_LSFL_FS | DLM_LSFL_NEWEXCL, DLM_CKV_LVB_SIZE, diff --git a/drivers/target/dlm_ckv.h b/drivers/target/dlm_ckv.h index 1a3f79e42bf6..080d9498f5f9 100644 --- a/drivers/target/dlm_ckv.h +++ b/drivers/target/dlm_ckv.h @@ -5,6 +5,8 @@ struct dlm_ckv_bucket; struct dlm_ckv_lock; +typedef void (*dlm_ckv_notify_cb)(void *userarg); + struct dlm_ckv_bucket *dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg); @@ -16,4 +18,10 @@ void dlm_ckv_free_lock(struct dlm_ckv_lock *ckv_lock); int dlm_ckv_lock_get(struct dlm_ckv_lock *ckv_lock); int dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock); +struct dlm_ckv_notify * +dlm_ckv_create_notification(struct dlm_ckv_bucket *bucket, const char *name, + dlm_ckv_notify_cb cb); +void dlm_ckv_free_notification(struct dlm_ckv_notify *notify); +int dlm_ckv_notify(struct dlm_ckv_notify *notify); + #endif From patchwork Mon Nov 22 17:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D108EC3F6B0 for ; Wed, 3 Aug 2022 16:30:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238448AbiHCQaR (ORCPT ); Wed, 3 Aug 2022 12:30:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238437AbiHCQ3t (ORCPT ); Wed, 3 Aug 2022 12:29:49 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90A64491C8; Wed, 3 Aug 2022 09:29:47 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 26D0B4132C; Wed, 3 Aug 2022 16:29:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544184; x=1661358585; bh=mSDTwFr55J6/EqbLfDIs9OFkW BeHO79zTZOG0+WN3Oo=; b=PiBl76ZR7OH9T5HSctQ0q7kghZmZid52y8A2gXWBG /tiOlWdWf1twIAqH+jlBoYhngCEjLJ3hDzZICRgXUjBrwaMFcapbVi1lvp+CgQba 5zHGjeFg7cme6umyMseUgTBOAR+iAC8wjk0WED9ty5OG7zbNwJOMEQqe7LoYC1vr lI= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VAJANPcnqkoC; Wed, 3 Aug 2022 19:29:44 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id C284E412D0; Wed, 3 Aug 2022 19:29:21 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:21 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:20 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 33/48] dlm_ckv: add key-value storage service Date: Mon, 22 Nov 2021 20:12:57 +0300 Message-ID: <20220803162857.27770-34-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Simple key-value cluster storage. Signed-off-by: Dmitry Bogdanov --- drivers/target/dlm_ckv.c | 128 +++++++++++++++++++++++++++++++++++++++ drivers/target/dlm_ckv.h | 9 +++ 2 files changed, 137 insertions(+) diff --git a/drivers/target/dlm_ckv.c b/drivers/target/dlm_ckv.c index cffe4f2dcb82..22c5f0827595 100644 --- a/drivers/target/dlm_ckv.c +++ b/drivers/target/dlm_ckv.c @@ -25,6 +25,10 @@ struct dlm_ckv_lock { char name[DLM_RESNAME_MAXLEN]; }; +struct dlm_ckv_kv { + struct dlm_ckv_lock lock; +}; + struct dlm_ckv_bucket { dlm_lockspace_t *ls; struct kref refcount; @@ -240,6 +244,130 @@ dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock) } EXPORT_SYMBOL(dlm_ckv_lock_release); +struct dlm_ckv_kv * +dlm_ckv_create_kv(struct dlm_ckv_bucket *bucket, const char *key) +{ + struct dlm_ckv_lock *ckv_lock; + void *ptr; + int res; + + ptr = kzalloc(DLM_CKV_LVB_SIZE, GFP_KERNEL); + if (!ptr) + return NULL; + + ckv_lock = dlm_ckv_create_lock(bucket, key); + if (!ckv_lock) + goto create_fail; + + ckv_lock->lksb.lksb.sb_lvbptr = ptr; + + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_NL, + &ckv_lock->lksb, 0, ckv_lock->name, NULL); + if (res) + goto lock_failed; + + return (struct dlm_ckv_kv *)ckv_lock; + +lock_failed: + dlm_ckv_free_lock(ckv_lock); +create_fail: + kfree(ptr); + return NULL; +} +EXPORT_SYMBOL(dlm_ckv_create_kv); + +void +dlm_ckv_free_kv(struct dlm_ckv_kv *kv) +{ + struct dlm_ckv_bucket *bucket = kv->lock.bucket; + struct dlm_ckv_lock *ckv_lock = &kv->lock; + + dlm_ckv_unlock_wait(bucket->ls, &ckv_lock->lksb); + + kfree(ckv_lock->lksb.lksb.sb_lvbptr); + kfree(ckv_lock); + + kref_put(&bucket->refcount, bucket_release); + +} +EXPORT_SYMBOL(dlm_ckv_free_kv); + +int +dlm_ckv_get(struct dlm_ckv_kv *kv, char *value, size_t len) +{ + struct dlm_ckv_lock *ckv_lock = &kv->lock; + struct dlm_ckv_bucket *bucket; + int res; + + BUG_ON(!ckv_lock); + bucket = ckv_lock->bucket; + + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_CR, + &ckv_lock->lksb, + DLM_LKF_VALBLK | DLM_LKF_CONVERT, + ckv_lock->name, NULL); + if (res) { + pr_info("Can not get lock %s, rc=%d\n", + ckv_lock->name, res); + goto fail; + } + + if (ckv_lock->lksb.lksb.sb_flags & DLM_SBF_VALNOTVALID) { + pr_info(" %s LVB was invalid\n", ckv_lock->name); + memset(value, 0, len); + } else + memcpy(value, ckv_lock->lksb.lksb.sb_lvbptr, len); + + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_NL, + &ckv_lock->lksb, + DLM_LKF_CONVERT, + ckv_lock->name, NULL); + if (res) { + pr_info("Can not release lock %s\n", ckv_lock->name); + goto fail; + } + + return res; +fail: + return res; +} +EXPORT_SYMBOL(dlm_ckv_get); + +int +dlm_ckv_set(struct dlm_ckv_kv *kv, const char *value, size_t len) +{ + struct dlm_ckv_lock *ckv_lock = &kv->lock; + struct dlm_ckv_bucket *bucket; + int res; + + BUG_ON(!ckv_lock); + bucket = ckv_lock->bucket; + + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_EX, + &ckv_lock->lksb, + DLM_LKF_CONVERT, + ckv_lock->name, NULL); + if (res) { + pr_info("Can not get lock %s\n", ckv_lock->name); + goto fail; + } + + memcpy(ckv_lock->lksb.lksb.sb_lvbptr, value, len); + res = dlm_ckv_lock_wait(ckv_lock->bucket->ls, DLM_LOCK_NL, + &ckv_lock->lksb, + DLM_LKF_VALBLK | DLM_LKF_CONVERT, + ckv_lock->name, NULL); + if (res) { + pr_info("Can not release lock %s\n", ckv_lock->name); + goto fail; + } + + return res; +fail: + return res; +} +EXPORT_SYMBOL(dlm_ckv_set); + static void dlm_cvk_pre_n_bast(void *astarg, int mode) { struct dlm_ckv_lksb *lksb = astarg; diff --git a/drivers/target/dlm_ckv.h b/drivers/target/dlm_ckv.h index 080d9498f5f9..c01904313f1e 100644 --- a/drivers/target/dlm_ckv.h +++ b/drivers/target/dlm_ckv.h @@ -4,6 +4,9 @@ struct dlm_ckv_bucket; struct dlm_ckv_lock; +struct dlm_ckv_kv; + +#define DLM_CKV_VALUE_MAX_SIZE 255 typedef void (*dlm_ckv_notify_cb)(void *userarg); @@ -18,6 +21,12 @@ void dlm_ckv_free_lock(struct dlm_ckv_lock *ckv_lock); int dlm_ckv_lock_get(struct dlm_ckv_lock *ckv_lock); int dlm_ckv_lock_release(struct dlm_ckv_lock *ckv_lock); +struct dlm_ckv_kv * +dlm_ckv_create_kv(struct dlm_ckv_bucket *bucket, const char *key); +void dlm_ckv_free_kv(struct dlm_ckv_kv *kv); +int dlm_ckv_get(struct dlm_ckv_kv *kv, char *value, size_t len); +int dlm_ckv_set(struct dlm_ckv_kv *kv, const char *value, size_t len); + struct dlm_ckv_notify * dlm_ckv_create_notification(struct dlm_ckv_bucket *bucket, const char *name, dlm_ckv_notify_cb cb); From patchwork Wed Dec 22 12:38:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 358F7C19F2C for ; Wed, 3 Aug 2022 16:30:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238537AbiHCQaY (ORCPT ); Wed, 3 Aug 2022 12:30:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238390AbiHCQ3v (ORCPT ); Wed, 3 Aug 2022 12:29:51 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BF904A815; Wed, 3 Aug 2022 09:29:49 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id A35544133A; Wed, 3 Aug 2022 16:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544186; x=1661358587; bh=ZbDUJV5W2MMKsVdc3cnlzmakj 9+yeRVqsuaxQTcQkZU=; b=LFbHOwpDRIJNXonPCwzXKAkw9SHB9f+dNYqCyTrpU aW//dzaCpfE7EKlwJTvzq3yh1cXtY0onHzeA8ymK55rLF5sglWSArz6cM2inr6m6 QBvCloEOHpiaDDEr4xLI5cK2kvtRPdNN6royphgKLBxOtSHBwcSI6kShssYaKQah BA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8WDluY1MB3D9; Wed, 3 Aug 2022 19:29:46 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 05ADE412E5; Wed, 3 Aug 2022 19:29:22 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:22 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:21 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 34/48] dlm_ckv: add KV get/set async API Date: Wed, 22 Dec 2021 15:38:14 +0300 Message-ID: <20220803162857.27770-35-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The sync versions of KV set/get functions may take too long time if user want to store/read a number of keys. Add async versions of KV get/set functions to allow the user to group the requests and to wait for completion of all requests together. Signed-off-by: Dmitry Bogdanov --- drivers/target/dlm_ckv.c | 110 ++++++++++++++++++++++++++++++++++++++- drivers/target/dlm_ckv.h | 6 +++ 2 files changed, 115 insertions(+), 1 deletion(-) diff --git a/drivers/target/dlm_ckv.c b/drivers/target/dlm_ckv.c index 22c5f0827595..417159f18fc6 100644 --- a/drivers/target/dlm_ckv.c +++ b/drivers/target/dlm_ckv.c @@ -14,15 +14,27 @@ #include #include "dlm_ckv.h" +enum dlm_lksb_state { + ONE_STAGE_SYNC, + READ_FIRST_STAGE, + READ_SECOND_STAGE, + WRITE_FIRST_STAGE, + WRITE_SECOND_STAGE, +}; struct dlm_ckv_lksb { struct dlm_lksb lksb; struct completion compl; + enum dlm_lksb_state state; }; struct dlm_ckv_lock { struct dlm_ckv_bucket *bucket; struct dlm_ckv_lksb lksb; char name[DLM_RESNAME_MAXLEN]; + dlm_ckv_async_cb async_cb; + void *cb_arg; + void *value; + size_t len; }; struct dlm_ckv_kv { @@ -97,8 +109,48 @@ static const struct dlm_lockspace_ops dlm_ckv_lockspace_ops = { static void dlm_ast(void *astarg) { struct dlm_ckv_lksb *dlm_ckv_lksb = astarg; + struct dlm_ckv_lock *ckv_lock; + struct dlm_ckv_bucket *bucket; + int res; + + ckv_lock = container_of(dlm_ckv_lksb, struct dlm_ckv_lock, lksb); + bucket = ckv_lock->bucket; - complete(&dlm_ckv_lksb->compl); + switch (dlm_ckv_lksb->state) { + case ONE_STAGE_SYNC: + complete(&dlm_ckv_lksb->compl); + break; + case READ_FIRST_STAGE: + if (dlm_ckv_lksb->lksb.sb_flags & DLM_SBF_VALNOTVALID) { + pr_debug("%s LVB was invalid\n", ckv_lock->name); + memset(ckv_lock->value, 0, ckv_lock->len); + } else + memcpy(ckv_lock->value, dlm_ckv_lksb->lksb.sb_lvbptr, + ckv_lock->len); + + dlm_ckv_lksb->state = READ_SECOND_STAGE; + res = dlm_lock(ckv_lock->bucket->ls, DLM_LOCK_NL, + &dlm_ckv_lksb->lksb, DLM_LKF_CONVERT, 0, 0, 0, + dlm_ast, dlm_ckv_lksb, NULL); + if (res) + ckv_lock->async_cb(ckv_lock->cb_arg, res); + break; + case WRITE_FIRST_STAGE: + + dlm_ckv_lksb->state = WRITE_SECOND_STAGE; + res = dlm_lock(ckv_lock->bucket->ls, DLM_LOCK_NL, + &dlm_ckv_lksb->lksb, + DLM_LKF_VALBLK | DLM_LKF_CONVERT, 0, 0, 0, + dlm_ast, dlm_ckv_lksb, NULL); + if (res) + ckv_lock->async_cb(ckv_lock->cb_arg, res); + break; + case READ_SECOND_STAGE: + fallthrough; + case WRITE_SECOND_STAGE: + ckv_lock->async_cb(ckv_lock->cb_arg, dlm_ckv_lksb->lksb.sb_status); + break; + } } /* @@ -135,6 +187,7 @@ static int dlm_ckv_lock_wait(dlm_lockspace_t *ls, int mode, { int res; + lksb->state = ONE_STAGE_SYNC; res = dlm_lock(ls, mode, &lksb->lksb, flags, (void *)name, name ? strlen(name) : 0, 0, dlm_ast, lksb, bast); @@ -164,6 +217,7 @@ static int dlm_ckv_unlock_wait(dlm_lockspace_t *ls, struct dlm_ckv_lksb *lksb) { int res; + lksb->state = ONE_STAGE_SYNC; res = dlm_unlock(ls, lksb->lksb.sb_lkid, 0, &lksb->lksb, lksb); if (res < 0) goto out; @@ -333,6 +387,32 @@ dlm_ckv_get(struct dlm_ckv_kv *kv, char *value, size_t len) } EXPORT_SYMBOL(dlm_ckv_get); +int +dlm_ckv_get_async(struct dlm_ckv_kv *kv, char *value, size_t len, + dlm_ckv_async_cb cb, void *cb_arg) +{ + struct dlm_ckv_lock *ckv_lock = &kv->lock; + struct dlm_ckv_bucket *bucket; + int res; + + BUG_ON(!ckv_lock); + bucket = ckv_lock->bucket; + + ckv_lock->lksb.state = READ_FIRST_STAGE; + ckv_lock->len = len; + ckv_lock->value = value; + ckv_lock->async_cb = cb; + ckv_lock->cb_arg = cb_arg; + res = dlm_lock(ckv_lock->bucket->ls, DLM_LOCK_CR, &ckv_lock->lksb.lksb, + DLM_LKF_VALBLK | DLM_LKF_CONVERT, NULL, 0, 0, + dlm_ast, &ckv_lock->lksb, NULL); + if (res) + pr_info("Can not get lock %s, rc=%d\n", ckv_lock->name, res); + + return res; +} +EXPORT_SYMBOL(dlm_ckv_get_async); + int dlm_ckv_set(struct dlm_ckv_kv *kv, const char *value, size_t len) { @@ -368,6 +448,34 @@ dlm_ckv_set(struct dlm_ckv_kv *kv, const char *value, size_t len) } EXPORT_SYMBOL(dlm_ckv_set); +int +dlm_ckv_set_async(struct dlm_ckv_kv *kv, char *value, size_t len, + dlm_ckv_async_cb cb, void *cb_arg) +{ + struct dlm_ckv_lock *ckv_lock = &kv->lock; + struct dlm_ckv_bucket *bucket; + int res; + + BUG_ON(!ckv_lock); + bucket = ckv_lock->bucket; + + ckv_lock->lksb.state = WRITE_FIRST_STAGE; + ckv_lock->len = len; + ckv_lock->value = value; + ckv_lock->async_cb = cb; + ckv_lock->cb_arg = cb_arg; + memcpy(ckv_lock->lksb.lksb.sb_lvbptr, ckv_lock->value, ckv_lock->len); + + res = dlm_lock(ckv_lock->bucket->ls, DLM_LOCK_EX, &ckv_lock->lksb.lksb, + DLM_LKF_CONVERT, NULL, 0, 0, + dlm_ast, &ckv_lock->lksb, NULL); + if (res) + pr_info("Can not get lock %s\n", ckv_lock->name); + + return res; +} +EXPORT_SYMBOL(dlm_ckv_set_async); + static void dlm_cvk_pre_n_bast(void *astarg, int mode) { struct dlm_ckv_lksb *lksb = astarg; diff --git a/drivers/target/dlm_ckv.h b/drivers/target/dlm_ckv.h index c01904313f1e..e8045917067e 100644 --- a/drivers/target/dlm_ckv.h +++ b/drivers/target/dlm_ckv.h @@ -9,6 +9,8 @@ struct dlm_ckv_kv; #define DLM_CKV_VALUE_MAX_SIZE 255 typedef void (*dlm_ckv_notify_cb)(void *userarg); +typedef void (*dlm_ckv_async_cb)(void *userarg, int res); +typedef void (*dlm_ckv_nodeleft_cb)(void *arg, int nodeid); struct dlm_ckv_bucket *dlm_ckv_open_bucket(const char *name, const char *cluster_name, @@ -26,6 +28,10 @@ dlm_ckv_create_kv(struct dlm_ckv_bucket *bucket, const char *key); void dlm_ckv_free_kv(struct dlm_ckv_kv *kv); int dlm_ckv_get(struct dlm_ckv_kv *kv, char *value, size_t len); int dlm_ckv_set(struct dlm_ckv_kv *kv, const char *value, size_t len); +int dlm_ckv_get_async(struct dlm_ckv_kv *kv, char *value, size_t len, + dlm_ckv_async_cb cb, void *cb_arg); +int dlm_ckv_set_async(struct dlm_ckv_kv *kv, char *value, size_t len, + dlm_ckv_async_cb cb, void *cb_arg); struct dlm_ckv_notify * dlm_ckv_create_notification(struct dlm_ckv_bucket *bucket, const char *name, From patchwork Tue Dec 7 09:47:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0488FC3F6B0 for ; Wed, 3 Aug 2022 16:31:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238618AbiHCQbJ (ORCPT ); Wed, 3 Aug 2022 12:31:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238472AbiHCQ37 (ORCPT ); Wed, 3 Aug 2022 12:29:59 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32CF0459B6; Wed, 3 Aug 2022 09:29:51 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id DB971412CD; Wed, 3 Aug 2022 16:29:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544187; x=1661358588; bh=zlKycAt5dSClcXurbAZMX2Ti9 5ZJfsryd4YKniFhyfw=; b=qa6+B+fdHJX12/APRj98KXWiB8aDP33qJgAevbOGG sHofBkdqydLzkuvxs/GSfQbTHfxpYBAG/3Or08joiXtOdHGrsAoljOhy5nct0Wyq gSwHHtVht71yRuIJFYVHsex8mOXTGf6DRYxjN+mAB3PkezQjuLdhCKdsWsS/3J+S 58= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VP6PibY3Kz1V; Wed, 3 Aug 2022 19:29:47 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 656FA41287; Wed, 3 Aug 2022 19:29:22 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:22 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:21 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov , Konstantin Shelekhin Subject: [RFC PATCH 35/48] target: add virtual remote target Date: Tue, 7 Dec 2021 12:47:12 +0300 Message-ID: <20220803162857.27770-36-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Create virtual remote target module. It can be used to see a whole acl/lun/tpg configuration from all nodes in storage cluster. Suggested-by: Konstantin Shelekhin Signed-off-by: Dmitry Bogdanov --- drivers/target/Kconfig | 1 + drivers/target/Makefile | 1 + drivers/target/tcm_remote/Kconfig | 8 + drivers/target/tcm_remote/Makefile | 2 + drivers/target/tcm_remote/tcm_remote.c | 405 +++++++++++++++++++++++++ drivers/target/tcm_remote/tcm_remote.h | 29 ++ 6 files changed, 446 insertions(+) create mode 100644 drivers/target/tcm_remote/Kconfig create mode 100644 drivers/target/tcm_remote/Makefile create mode 100644 drivers/target/tcm_remote/tcm_remote.c create mode 100644 drivers/target/tcm_remote/tcm_remote.h diff --git a/drivers/target/Kconfig b/drivers/target/Kconfig index 75d5e1d23a1c..5440c0e93e1e 100644 --- a/drivers/target/Kconfig +++ b/drivers/target/Kconfig @@ -53,5 +53,6 @@ source "drivers/target/loopback/Kconfig" source "drivers/target/tcm_fc/Kconfig" source "drivers/target/iscsi/Kconfig" source "drivers/target/sbp/Kconfig" +source "drivers/target/tcm_remote/Kconfig" endif diff --git a/drivers/target/Makefile b/drivers/target/Makefile index 8bc9ac2bd629..be4d1bfcf79a 100644 --- a/drivers/target/Makefile +++ b/drivers/target/Makefile @@ -30,5 +30,6 @@ obj-$(CONFIG_LOOPBACK_TARGET) += loopback/ obj-$(CONFIG_TCM_FC) += tcm_fc/ obj-$(CONFIG_ISCSI_TARGET) += iscsi/ obj-$(CONFIG_SBP_TARGET) += sbp/ +obj-$(CONFIG_REMOTE_TARGET) += tcm_remote/ obj-$(CONFIG_DLM_CKV) += dlm_ckv.o diff --git a/drivers/target/tcm_remote/Kconfig b/drivers/target/tcm_remote/Kconfig new file mode 100644 index 000000000000..e6bebb5fe6f1 --- /dev/null +++ b/drivers/target/tcm_remote/Kconfig @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +config REMOTE_TARGET + tristate "TCM Virtual Remote target" + depends on SCSI + help + Say Y here to enable the TCM Virtual Remote fabric + That fabric is a dummy fabric to tell TCM about configuration + of TPG/ACL/LUN on peer nodes in a cluster. diff --git a/drivers/target/tcm_remote/Makefile b/drivers/target/tcm_remote/Makefile new file mode 100644 index 000000000000..5818ffd0b0fa --- /dev/null +++ b/drivers/target/tcm_remote/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_REMOTE_TARGET) += tcm_remote.o diff --git a/drivers/target/tcm_remote/tcm_remote.c b/drivers/target/tcm_remote/tcm_remote.c new file mode 100644 index 000000000000..a2aaeb3fdbe3 --- /dev/null +++ b/drivers/target/tcm_remote/tcm_remote.c @@ -0,0 +1,405 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "tcm_remote.h" + +#define to_tcm_remote_hba(hba) container_of(hba, struct tcm_remote_hba, dev) + +static int tcm_remote_hba_no_cnt; + +static inline struct tcm_remote_tpg *remote_tpg(struct se_portal_group *se_tpg) +{ + return container_of(se_tpg, struct tcm_remote_tpg, remote_se_tpg); +} + +static char *tcm_remote_get_endpoint_wwn(struct se_portal_group *se_tpg) +{ + /* + * Return the passed NAA identifier for the Target Port + */ + return &remote_tpg(se_tpg)->remote_hba->remote_wwn_address[0]; +} + +static u16 tcm_remote_get_tag(struct se_portal_group *se_tpg) +{ + /* + * This Tag is used when forming SCSI Name identifier in EVPD=1 0x83 + * to represent the SCSI Target Port. + */ + return remote_tpg(se_tpg)->remote_tpgt; +} + +/* + * Returning (1) here allows for target_core_mod struct se_node_acl to be generated + * based upon the incoming fabric dependent SCSI Initiator Port + */ +static int tcm_remote_check_demo_mode(struct se_portal_group *se_tpg) +{ + return 1; +} + +static int tcm_remote_check_demo_mode_cache(struct se_portal_group *se_tpg) +{ + return 0; +} + +/* + * Allow I_T Nexus full READ-WRITE access without explicit Initiator Node ACLs for + * local virtual Linux/SCSI LLD passthrough into VM hypervisor guest + */ +static int tcm_remote_check_demo_mode_write_protect(struct se_portal_group *se_tpg) +{ + return 0; +} + +/* + * It has been added here as a nop for target_fabric_tf_ops_check() + */ +static int tcm_remote_check_prod_mode_write_protect(struct se_portal_group *se_tpg) +{ + return 0; +} + +static int tcm_remote_check_prot_fabric_only(struct se_portal_group *se_tpg) +{ + struct tcm_remote_tpg *remote_tpg = container_of(se_tpg, struct tcm_remote_tpg, + remote_se_tpg); + return remote_tpg->remote_fabric_prot_type; +} + +static u32 tcm_remote_get_inst_index(struct se_portal_group *se_tpg) +{ + return 1; +} + +static u32 tcm_remote_sess_get_index(struct se_session *se_sess) +{ + return 1; +} + +static void tcm_remote_set_default_node_attributes(struct se_node_acl *se_acl) +{ + +} + +static int tcm_remote_dummy_cmd_fn(struct se_cmd *se_cmd) +{ + return 0; +} + +static void tcm_remote_dummy_cmd_void_fn(struct se_cmd *se_cmd) +{ + +} + + +static char *tcm_remote_dump_proto_id(struct tcm_remote_hba *remote_hba) +{ + switch (remote_hba->remote_proto_id) { + case SCSI_PROTOCOL_SAS: + return "SAS"; + case SCSI_PROTOCOL_SRP: + return "SRP"; + case SCSI_PROTOCOL_FCP: + return "FCP"; + case SCSI_PROTOCOL_ISCSI: + return "iSCSI"; + default: + break; + } + + return "Unknown"; +} + +/* Start items for tcm_remote_port_cit */ + +static int tcm_remote_port_link( + struct se_portal_group *se_tpg, + struct se_lun *lun) +{ + pr_debug("TCM_Remote_ConfigFS: Port Link LUN %lld Successful\n", + lun->unpacked_lun); + return 0; +} + +static void tcm_remote_port_unlink( + struct se_portal_group *se_tpg, + struct se_lun *lun) +{ + pr_debug("TCM_Remote_ConfigFS: Port Unlink LUN %lld Successful\n", + lun->unpacked_lun); +} + +/* End items for tcm_remote_port_cit */ + +static ssize_t tcm_remote_tpg_attrib_fabric_prot_type_show( + struct config_item *item, char *page) +{ + struct se_portal_group *se_tpg = attrib_to_tpg(item); + struct tcm_remote_tpg *remote_tpg = container_of(se_tpg, struct tcm_remote_tpg, + remote_se_tpg); + + return sprintf(page, "%d\n", remote_tpg->remote_fabric_prot_type); +} + +static ssize_t tcm_remote_tpg_attrib_fabric_prot_type_store( + struct config_item *item, const char *page, size_t count) +{ + struct se_portal_group *se_tpg = attrib_to_tpg(item); + struct tcm_remote_tpg *remote_tpg = container_of(se_tpg, struct tcm_remote_tpg, + remote_se_tpg); + unsigned long val; + int ret = kstrtoul(page, 0, &val); + + if (ret) { + pr_err("kstrtoul() returned %d for fabric_prot_type\n", ret); + return ret; + } + if (val != 0 && val != 1 && val != 3) { + pr_err("Invalid value of fabric_prot_type: %lu\n", val); + return -EINVAL; + } + remote_tpg->remote_fabric_prot_type = val; + + return count; +} + +CONFIGFS_ATTR(tcm_remote_tpg_attrib_, fabric_prot_type); + +static struct configfs_attribute *tcm_remote_tpg_attrib_attrs[] = { + &tcm_remote_tpg_attrib_attr_fabric_prot_type, + NULL, +}; + +static struct configfs_attribute *tcm_remote_tpg_attrs[] = { + NULL, +}; + +/* Start items for tcm_remote_naa_cit */ + +static struct se_portal_group *tcm_remote_make_tpg(struct se_wwn *wwn, + const char *name) +{ + struct tcm_remote_hba *remote_hba = container_of(wwn, + struct tcm_remote_hba, remote_hba_wwn); + struct tcm_remote_tpg *remote_tpg; + int ret; + unsigned long tpgt; + + if (strstr(name, "tpgt_") != name) { + pr_err("Unable to locate \"tpgt_#\" directory group\n"); + return ERR_PTR(-EINVAL); + } + if (kstrtoul(name+5, 10, &tpgt)) + return ERR_PTR(-EINVAL); + + if (tpgt >= TL_TPGS_PER_HBA) { + pr_err("Passed tpgt: %lu exceeds TL_TPGS_PER_HBA: %u\n", + tpgt, TL_TPGS_PER_HBA); + return ERR_PTR(-EINVAL); + } + remote_tpg = &remote_hba->remote_hba_tpgs[tpgt]; + remote_tpg->remote_hba = remote_hba; + remote_tpg->remote_tpgt = tpgt; + /* + * Register the remote_tpg as a emulated TCM Target Endpoint + */ + ret = core_tpg_register(wwn, &remote_tpg->remote_se_tpg, remote_hba->remote_proto_id); + if (ret < 0) + return ERR_PTR(-ENOMEM); + + pr_debug("TCM_Remote_ConfigFS: Allocated Emulated %s Target Port %s,t,0x%04lx\n", + tcm_remote_dump_proto_id(remote_hba), + config_item_name(&wwn->wwn_group.cg_item), tpgt); + return &remote_tpg->remote_se_tpg; +} + +static void tcm_remote_drop_tpg( + struct se_portal_group *se_tpg) +{ + struct se_wwn *wwn = se_tpg->se_tpg_wwn; + struct tcm_remote_tpg *remote_tpg = container_of(se_tpg, + struct tcm_remote_tpg, remote_se_tpg); + struct tcm_remote_hba *remote_hba; + unsigned short tpgt; + + remote_hba = remote_tpg->remote_hba; + tpgt = remote_tpg->remote_tpgt; + + /* + * Deregister the remote_tpg as a emulated TCM Target Endpoint + */ + core_tpg_deregister(se_tpg); + + remote_tpg->remote_hba = NULL; + remote_tpg->remote_tpgt = 0; + + pr_debug("TCM_Remote_ConfigFS: Deallocated Emulated %s Target Port %s,t,0x%04x\n", + tcm_remote_dump_proto_id(remote_hba), + config_item_name(&wwn->wwn_group.cg_item), tpgt); +} + +/* End items for tcm_remote_naa_cit */ + +/* Start items for tcm_remote_cit */ + +static struct se_wwn *tcm_remote_make_wwn( + struct target_fabric_configfs *tf, + struct config_group *group, + const char *name) +{ + struct tcm_remote_hba *remote_hba; + char *ptr; + int ret, off = 0; + + remote_hba = kzalloc(sizeof(*remote_hba), GFP_KERNEL); + if (!remote_hba) + return ERR_PTR(-ENOMEM); + + /* + * Determine the emulated Protocol Identifier and Target Port Name + * based on the incoming configfs directory name. + */ + ptr = strstr(name, "naa."); + if (ptr) { + remote_hba->remote_proto_id = SCSI_PROTOCOL_SAS; + goto check_len; + } + ptr = strstr(name, "fc."); + if (ptr) { + remote_hba->remote_proto_id = SCSI_PROTOCOL_FCP; + off = 3; /* Skip over "fc." */ + goto check_len; + } + ptr = strstr(name, "0x"); + if (ptr) { + remote_hba->remote_proto_id = SCSI_PROTOCOL_SRP; + off = 2; /* Skip over "0x" */ + goto check_len; + } + ptr = strstr(name, "iqn."); + if (!ptr) { + pr_err("Unable to locate prefix for emulated Target Port: %s\n", + name); + ret = -EINVAL; + goto out; + } + remote_hba->remote_proto_id = SCSI_PROTOCOL_ISCSI; + +check_len: + if (strlen(name) >= TL_WWN_ADDR_LEN) { + pr_err("Emulated NAA %s Address: %s, exceeds max: %d\n", + name, tcm_remote_dump_proto_id(remote_hba), TL_WWN_ADDR_LEN); + ret = -EINVAL; + goto out; + } + snprintf(&remote_hba->remote_wwn_address[0], TL_WWN_ADDR_LEN, "%s", &name[off]); + + tcm_remote_hba_no_cnt++; + pr_debug("TCM_Remote_ConfigFS: Allocated emulated Target %s Address: %s\n", + tcm_remote_dump_proto_id(remote_hba), name); + return &remote_hba->remote_hba_wwn; +out: + kfree(remote_hba); + return ERR_PTR(ret); +} + +static void tcm_remote_drop_wwn( + struct se_wwn *wwn) +{ + struct tcm_remote_hba *remote_hba = container_of(wwn, + struct tcm_remote_hba, remote_hba_wwn); + + pr_debug("TCM_Remote_ConfigFS: Deallocating emulated Target %s Address: %s\n", + tcm_remote_dump_proto_id(remote_hba), + remote_hba->remote_wwn_address); + +} + +/* Start items for tcm_remote_cit */ +static ssize_t tcm_remote_wwn_version_show(struct config_item *item, char *page) +{ + return sprintf(page, "TCM Remote Fabric module %s\n", TCM_REMOTE_VERSION); +} + +CONFIGFS_ATTR_RO(tcm_remote_wwn_, version); + +static struct configfs_attribute *tcm_remote_wwn_attrs[] = { + &tcm_remote_wwn_attr_version, + NULL, +}; + +/* End items for tcm_remote_cit */ + +static const struct target_core_fabric_ops remote_ops = { + .module = THIS_MODULE, + .fabric_name = "remote", + .tpg_get_wwn = tcm_remote_get_endpoint_wwn, + .tpg_get_tag = tcm_remote_get_tag, + .tpg_check_demo_mode = tcm_remote_check_demo_mode, + .tpg_check_demo_mode_cache = tcm_remote_check_demo_mode_cache, + .tpg_check_demo_mode_write_protect = + tcm_remote_check_demo_mode_write_protect, + .tpg_check_prod_mode_write_protect = + tcm_remote_check_prod_mode_write_protect, + .tpg_check_prot_fabric_only = tcm_remote_check_prot_fabric_only, + .tpg_get_inst_index = tcm_remote_get_inst_index, + .check_stop_free = tcm_remote_dummy_cmd_fn, + .release_cmd = tcm_remote_dummy_cmd_void_fn, + .sess_get_index = tcm_remote_sess_get_index, + .write_pending = tcm_remote_dummy_cmd_fn, + .set_default_node_attributes = tcm_remote_set_default_node_attributes, + .get_cmd_state = tcm_remote_dummy_cmd_fn, + .queue_data_in = tcm_remote_dummy_cmd_fn, + .queue_status = tcm_remote_dummy_cmd_fn, + .queue_tm_rsp = tcm_remote_dummy_cmd_void_fn, + .aborted_task = tcm_remote_dummy_cmd_void_fn, + .fabric_make_wwn = tcm_remote_make_wwn, + .fabric_drop_wwn = tcm_remote_drop_wwn, + .fabric_make_tpg = tcm_remote_make_tpg, + .fabric_drop_tpg = tcm_remote_drop_tpg, + .fabric_post_link = tcm_remote_port_link, + .fabric_pre_unlink = tcm_remote_port_unlink, + .tfc_wwn_attrs = tcm_remote_wwn_attrs, + .tfc_tpg_base_attrs = tcm_remote_tpg_attrs, + .tfc_tpg_attrib_attrs = tcm_remote_tpg_attrib_attrs, +}; + +static int __init tcm_remote_fabric_init(void) +{ + int ret = -ENOMEM; + + ret = target_register_template(&remote_ops); + if (ret) + goto out; + + return 0; + +out: + return ret; +} + +static void __exit tcm_remote_fabric_exit(void) +{ + target_unregister_template(&remote_ops); +} + +MODULE_DESCRIPTION("TCM virtual remote target"); +MODULE_AUTHOR("Dmitry Bogdanov "); +MODULE_LICENSE("GPL"); +module_init(tcm_remote_fabric_init); +module_exit(tcm_remote_fabric_exit); diff --git a/drivers/target/tcm_remote/tcm_remote.h b/drivers/target/tcm_remote/tcm_remote.h new file mode 100644 index 000000000000..8d674c87989e --- /dev/null +++ b/drivers/target/tcm_remote/tcm_remote.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include +#include + +#define TCM_REMOTE_VERSION "v0.1" +#define TL_WWN_ADDR_LEN 256 +#define TL_TPGS_PER_HBA 32 + + +#define TCM_TRANSPORT_ONLINE 0 +#define TCM_TRANSPORT_OFFLINE 1 + +struct tcm_remote_tpg { + unsigned short remote_tpgt; + enum target_prot_type remote_fabric_prot_type; + atomic_t remote_tpg_port_count; + struct se_portal_group remote_se_tpg; + struct tcm_remote_hba *remote_hba; +}; + +struct tcm_remote_hba { + u8 remote_proto_id; + unsigned char remote_wwn_address[TL_WWN_ADDR_LEN]; + struct se_hba_s *se_hba; + struct se_lun *remote_hba_lun; + struct se_port *remote_hba_lun_sep; + struct tcm_remote_tpg remote_hba_tpgs[TL_TPGS_PER_HBA]; + struct se_wwn remote_hba_wwn; +}; From patchwork Wed Nov 17 11:12:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C016DC19F28 for ; Wed, 3 Aug 2022 16:31:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238600AbiHCQa6 (ORCPT ); Wed, 3 Aug 2022 12:30:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238475AbiHCQ37 (ORCPT ); Wed, 3 Aug 2022 12:29:59 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9737A48E8A; Wed, 3 Aug 2022 09:29:53 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 10BE64131D; Wed, 3 Aug 2022 16:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544189; x=1661358590; bh=1/lLGStKju420GDKs/ya8qKkM bEQG4z1qGKFPfh/mR4=; b=TKX4M9Beo0i6jkWkGtjBbsMSP5c42IvwVW+D//+ES NM71mAWzoX3NIUkhl49ArInMZ97z/CfchUNqaVi/SK5xQfkKEMeShwcEoHpK0dbO pPfuF+5T9i47bf//8c74y4zfAtA6RGzTFAyw8uHZmiI/bu9CwrXt+qVJXNWhIJV3 7I= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j2pcoCGPdKTC; Wed, 3 Aug 2022 19:29:49 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id AB8A5412ED; Wed, 3 Aug 2022 19:29:22 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:22 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:21 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 36/48] target: cluster: introduce cluster ops Date: Wed, 17 Nov 2021 14:12:29 +0300 Message-ID: <20220803162857.27770-37-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce cluster ops common for all backend devices. Those ops will implement cluster level locks, store clusterwide data (like persistent reservations) and provide clusterwide notifications. That ops could be registered dynamically. Ops have own configfs folder in .../target/cluster/ Create default non-cluster ops implementation. Use cluster ops for CAW lock. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_configfs.c | 63 +++++++++++- drivers/target/target_core_device.c | 139 +++++++++++++++++++++++++- drivers/target/target_core_internal.h | 8 ++ drivers/target/target_core_sbc.c | 12 +-- include/target/target_core_base.h | 18 ++++ 5 files changed, 232 insertions(+), 8 deletions(-) diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c index f63c37344104..6c36b1f4bd92 100644 --- a/drivers/target/target_core_configfs.c +++ b/drivers/target/target_core_configfs.c @@ -69,6 +69,7 @@ static LIST_HEAD(g_tf_list); static DEFINE_MUTEX(g_tf_lock); static struct config_group target_core_hbagroup; +static struct config_group *target_core_cluster_group; static struct config_group alua_group; static struct config_group alua_lu_gps_group; @@ -1193,6 +1194,32 @@ static ssize_t pgr_support_store(struct config_item *item, return count; } +static ssize_t cluster_impl_show(struct config_item *item, char *page) +{ + return snprintf(page, PAGE_SIZE, "%s\n", + to_attrib(item)->da_dev->cl_ops->name); +} + +static ssize_t cluster_impl_store(struct config_item *item, + const char *page, size_t count) +{ + struct se_dev_attrib *da = to_attrib(item); + struct se_device *dev = da->da_dev; + char cluster_impl[16]; + int res; + + strscpy(cluster_impl, page, 16); + if (cluster_impl[strlen(cluster_impl) - 1] == '\n') + cluster_impl[strlen(cluster_impl) - 1] = 0; + + res = core_dev_set_cluster_mode(dev, cluster_impl); + if (res) + return res; + + return count; +} +CONFIGFS_ATTR(, cluster_impl); + CONFIGFS_ATTR(, emulate_model_alias); CONFIGFS_ATTR(, emulate_dpo); CONFIGFS_ATTR(, emulate_fua_write); @@ -1268,6 +1295,7 @@ struct configfs_attribute *sbc_attrib_attrs[] = { &attr_max_write_same_len, &attr_alua_support, &attr_pgr_support, + &attr_cluster_impl, NULL, }; EXPORT_SYMBOL(sbc_attrib_attrs); @@ -3615,6 +3643,38 @@ static const struct config_item_type target_core_cit = { /* Stop functions for struct config_item_type target_core_hba_cit */ +static const struct config_item_type target_core_cluster_cit = { + .ct_owner = THIS_MODULE, +}; + +int target_core_register_cluster_configfs(struct target_cluster_impl *tci) +{ + int ret = 0; + + if (!target_core_cluster_group) { + target_core_cluster_group = configfs_register_default_group( + &target_core_fabrics.su_group, + "cluster", + &target_core_cluster_cit); + if (IS_ERR(target_core_cluster_group)) { + ret = PTR_ERR(target_core_cluster_group); + pr_err("Error %d while registering cluster group\n", ret); + goto err; + } + } + + ret = configfs_register_group(target_core_cluster_group, + tci->ops->cg_group); + +err: + return ret; +} + +void target_core_unregister_cluster_configfs(struct target_cluster_impl *tci) +{ + configfs_unregister_group(tci->ops->cg_group); +} + void target_setup_backend_cits(struct target_backend *tb) { target_core_setup_dev_cit(tb); @@ -3749,7 +3809,8 @@ static void __exit target_core_exit_configfs(void) configfs_remove_default_groups(&alua_lu_gps_group); configfs_remove_default_groups(&alua_group); configfs_remove_default_groups(&target_core_hbagroup); - + if (target_core_cluster_group) + configfs_remove_default_groups(target_core_cluster_group); /* * We expect subsys->su_group.default_groups to be released * by configfs subsystem provider logic.. diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 7c887792bb88..9c7f4dc1b85d 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -44,6 +44,77 @@ static struct se_hba *lun0_hba; /* not static, needed by tpg.c */ struct se_device *g_lun0_dev; +static LIST_HEAD(cluster_impl_list); +static DEFINE_MUTEX(cluster_impl_mutex); + +int target_cluster_impl_register(const struct target_cluster_ops *ops) +{ + struct target_cluster_impl *tci, *old; + + tci = kzalloc(sizeof(*tci), GFP_KERNEL); + if (!tci) + return -ENOMEM; + tci->ops = ops; + + mutex_lock(&cluster_impl_mutex); + list_for_each_entry(old, &cluster_impl_list, list) { + if (!strcmp(old->ops->name, ops->name)) { + pr_err("Target cluster implementation %s already registered.\n", + ops->name); + mutex_unlock(&cluster_impl_mutex); + kfree(tci); + return -EEXIST; + } + } + + target_core_register_cluster_configfs(tci); + + list_add_tail(&tci->list, &cluster_impl_list); + mutex_unlock(&cluster_impl_mutex); + + pr_debug("TCM: Registered cluster plugin: %s struct module: %p\n", + ops->name, ops->owner); + return 0; +} +EXPORT_SYMBOL(target_cluster_impl_register); + +void target_cluster_impl_unregister(const struct target_cluster_ops *ops) +{ + struct target_cluster_impl *tci; + + mutex_lock(&cluster_impl_mutex); + list_for_each_entry(tci, &cluster_impl_list, list) { + if (tci->ops == ops) { + list_del(&tci->list); + target_core_unregister_cluster_configfs(tci); + mutex_unlock(&cluster_impl_mutex); + kfree(tci); + return; + } + } + mutex_unlock(&cluster_impl_mutex); +} +EXPORT_SYMBOL(target_cluster_impl_unregister); + +static struct target_cluster_impl *core_get_cluster_impl(const char *name) +{ + struct target_cluster_impl *tci; + + mutex_lock(&cluster_impl_mutex); + list_for_each_entry(tci, &cluster_impl_list, list) { + if (!strcmp(tci->ops->name, name)) + goto found; + } + mutex_unlock(&cluster_impl_mutex); + return NULL; +found: + if (tci->ops->owner && !try_module_get(tci->ops->owner)) + tci = NULL; + mutex_unlock(&cluster_impl_mutex); + return tci; +} + + sense_reason_t transport_lookup_cmd_lun(struct se_cmd *se_cmd) { @@ -686,6 +757,40 @@ static void scsi_dump_inquiry(struct se_device *dev) pr_debug(" Type: %s ", scsi_device_type(device_type)); } +static int target_init_nodlm(struct se_device *dev) +{ + sema_init(&dev->caw_sem, 1); + + return 0; +} + +static int target_cleanup_nodlm(struct se_device *dev) +{ + return 0; +} + +static void *target_caw_lock_nodlm(struct se_device *dev, u64 lba) +{ + if (down_interruptible(&dev->caw_sem)) + return NULL; + return &dev->caw_sem; +} + +static int target_caw_unlock_nodlm(struct se_device *dev, void *lock) +{ + up(&dev->caw_sem); + return 0; +} + +const struct target_cluster_ops nodlm_cluster_ops = { + .name = "single", + .owner = NULL, + .init = target_init_nodlm, + .cleanup = target_cleanup_nodlm, + .caw_lock = target_caw_lock_nodlm, + .caw_unlock = target_caw_unlock_nodlm, +}; + struct se_device *target_alloc_device(struct se_hba *hba, const char *name) { struct se_device *dev; @@ -729,7 +834,6 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name) spin_lock_init(&dev->se_port_lock); spin_lock_init(&dev->se_tmr_lock); spin_lock_init(&dev->qf_cmd_lock); - sema_init(&dev->caw_sem, 1); INIT_LIST_HEAD(&dev->t10_wwn.t10_vpd_list); spin_lock_init(&dev->t10_wwn.t10_vpd_lock); INIT_LIST_HEAD(&dev->t10_pr.registration_list); @@ -779,6 +883,9 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name) DA_UNMAP_ZEROES_DATA_DEFAULT; dev->dev_attrib.max_write_same_len = DA_MAX_WRITE_SAME_LEN; + dev->cl_ops = &nodlm_cluster_ops; + dev->cl_ops->init(dev); + xcopy_lun = &dev->xcopy_lun; rcu_assign_pointer(xcopy_lun->lun_se_dev, dev); init_completion(&xcopy_lun->lun_shutdown_comp); @@ -987,6 +1094,9 @@ void target_free_device(struct se_device *dev) WARN_ON(!list_empty(&dev->dev_sep_list)); + dev->cl_ops->cleanup(dev); + module_put(dev->cl_ops->owner); + if (target_dev_configured(dev)) { dev->transport->destroy_device(dev); @@ -1058,6 +1168,33 @@ void core_dev_release_virtual_lun0(void) core_delete_hba(hba); } +int core_dev_set_cluster_mode(struct se_device *dev, const char *cluster_impl) +{ + int ret = 0; + + struct target_cluster_impl *tci = core_get_cluster_impl(cluster_impl); + + if (!tci) + return -EINVAL; + + if (dev->cl_ops != tci->ops) { + ret = dev->cl_ops->cleanup(dev); + if (ret) + pr_err("cluster cleanup failed with %d\n", ret); + + ret = tci->ops->init(dev); + if (ret) + pr_err("cluster init failed with %d\n", ret); + + module_put(dev->cl_ops->owner); + dev->cl_ops = tci->ops; + } else { + module_put(tci->ops->owner); + } + + return ret; +} + /* * Common CDB parsing for kernel and user passthrough. */ diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index b0d29c30ab1e..58261c716776 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -55,6 +55,11 @@ struct target_fabric_configfs { struct config_item_type tf_tpg_mappedlun_stat_cit; }; +struct target_cluster_impl { + const struct target_cluster_ops *ops; + struct list_head list; +}; + /* target_core_alua.c */ extern struct t10_alua_lu_gp *default_lu_gp; @@ -83,6 +88,7 @@ void core_dev_free_initiator_node_lun_acl(struct se_portal_group *, struct se_lun_acl *lacl); int core_dev_setup_virtual_lun0(void); void core_dev_release_virtual_lun0(void); +int core_dev_set_cluster_mode(struct se_device *dev, const char *cluster_impl); struct se_device *target_alloc_device(struct se_hba *hba, const char *name); int target_configure_device(struct se_device *dev); void target_free_device(struct se_device *); @@ -93,6 +99,8 @@ void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq); /* target_core_configfs.c */ extern struct configfs_item_operations target_core_dev_item_ops; void target_setup_backend_cits(struct target_backend *); +int target_core_register_cluster_configfs(struct target_cluster_impl *tci); +void target_core_unregister_cluster_configfs(struct target_cluster_impl *tci); /* target_core_fabric_configfs.c */ int target_fabric_setup_cits(struct target_fabric_configfs *); diff --git a/drivers/target/target_core_sbc.c b/drivers/target/target_core_sbc.c index 786ca58008b0..ab2038a510c0 100644 --- a/drivers/target/target_core_sbc.c +++ b/drivers/target/target_core_sbc.c @@ -367,7 +367,7 @@ static sense_reason_t compare_and_write_post(struct se_cmd *cmd, bool success, * Unlock ->caw_sem originally obtained during sbc_compare_and_write() * before the original READ I/O submission. */ - up(&dev->caw_sem); + dev->cl_ops->caw_unlock(dev, cmd->cawlock); return ret; } @@ -550,7 +550,7 @@ static sense_reason_t compare_and_write_callback(struct se_cmd *cmd, bool succes * In the MISCOMPARE or failure case, unlock ->caw_sem obtained in * sbc_compare_and_write() before the original READ I/O submission. */ - up(&dev->caw_sem); + dev->cl_ops->caw_unlock(dev, cmd->cawlock); sg_free_table(&write_tbl); return ret; } @@ -561,13 +561,13 @@ sbc_compare_and_write(struct se_cmd *cmd) struct sbc_ops *ops = cmd->protocol_data; struct se_device *dev = cmd->se_dev; sense_reason_t ret; - int rc; + /* * Submit the READ first for COMPARE_AND_WRITE to perform the * comparision using SGLs at cmd->t_bidi_data_sg.. */ - rc = down_interruptible(&dev->caw_sem); - if (rc != 0) { + cmd->cawlock = dev->cl_ops->caw_lock(dev, cmd->t_task_lba); + if (!cmd->cawlock) { cmd->transport_complete_callback = NULL; return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; } @@ -582,7 +582,7 @@ sbc_compare_and_write(struct se_cmd *cmd) DMA_FROM_DEVICE); if (ret) { cmd->transport_complete_callback = NULL; - up(&dev->caw_sem); + dev->cl_ops->caw_unlock(dev, cmd->cawlock); return ret; } /* diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 5378e2d3825f..fbeb4777654b 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -546,6 +546,7 @@ struct se_cmd { * initialized on. Drivers can override. */ int cpuid; + void *cawlock; }; struct se_ua { @@ -777,6 +778,17 @@ struct se_device_queue { struct se_cmd_queue sq; }; +struct target_cluster_ops { + char name[16]; + struct module *owner; + struct config_group *cg_group; + + int (*init)(struct se_device *dev); + int (*cleanup)(struct se_device *dev); + void *(*caw_lock)(struct se_device *dev, u64 lba); + int (*caw_unlock)(struct se_device *dev, void *lock); +}; + struct se_device { /* Used for SAM Task Attribute ordering */ u32 dev_cur_ordered_id; @@ -847,6 +859,9 @@ struct se_device { unsigned char udev_path[SE_UDEV_PATH_LEN]; /* Pointer to template of function pointers for transport */ const struct target_backend_ops *transport; + /* Pointer to template of function pointers for cluster mode */ + const struct target_cluster_ops *cl_ops; + void *cluster_data; struct se_lun xcopy_lun; /* Protection Information */ int prot_length; @@ -981,4 +996,7 @@ static inline void target_free_tag(struct se_session *sess, struct se_cmd *cmd) sbitmap_queue_clear(&sess->sess_tag_pool, cmd->map_tag, cmd->map_cpu); } +int target_cluster_impl_register(const struct target_cluster_ops *ops); +void target_cluster_impl_unregister(const struct target_cluster_ops *ops); + #endif /* TARGET_CORE_BASE_H */ From patchwork Fri Jul 22 14:59:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F6D4C19F29 for ; Wed, 3 Aug 2022 16:31:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238558AbiHCQbg (ORCPT ); Wed, 3 Aug 2022 12:31:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238503AbiHCQaN (ORCPT ); Wed, 3 Aug 2022 12:30:13 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E13546DBF; Wed, 3 Aug 2022 09:30:00 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 01748412DF; Wed, 3 Aug 2022 16:29:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544197; x=1661358598; bh=sHfzJDUjyjPG265rFr5KNrfhE yAfHiDS/o9/OVtTYSQ=; b=ucmJx7pvsk1R2IZCwJVCyAaXwskIEse+pGPiyOUgd HeJ8QFaNw7yQi7C3IMSCZQJv/SFgzDYZJBz1GxmMvdQ6M4v2KSF6pGNeWWCgoqTE SQ57rsgC7c8uBi+gX7Hc2x5i/IIkwsf2TioRlOM2X/61UPffM34mc1aWSEqtrmf5 Bg= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oFtY2cZC2oNM; Wed, 3 Aug 2022 19:29:57 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id E873C412F7; Wed, 3 Aug 2022 19:29:22 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:23 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:22 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 37/48] target: cluster: introduce dlm cluster Date: Fri, 22 Jul 2022 17:59:09 +0300 Message-ID: <20220803162857.27770-38-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce DLM Cluster implementation of cluster ops of backstore devices. Use one DLM lockspace per device. The device is idenitfied by concatenation of T10 model and T10 unit_serial. Use DLM CKV lock for Compare And Write in cluster mode. One lock for one LBA. That allows a parallell execution of CAW commands. User should set configfs/target/cluster/dlm/cluster_name identical to configfs/dlm/cluster/cluster_name. Signed-off-by: Dmitry Bogdanov --- drivers/target/Makefile | 1 + drivers/target/target_cluster_dlm.c | 175 ++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+) create mode 100644 drivers/target/target_cluster_dlm.c diff --git a/drivers/target/Makefile b/drivers/target/Makefile index be4d1bfcf79a..16b625108eec 100644 --- a/drivers/target/Makefile +++ b/drivers/target/Makefile @@ -33,3 +33,4 @@ obj-$(CONFIG_SBP_TARGET) += sbp/ obj-$(CONFIG_REMOTE_TARGET) += tcm_remote/ obj-$(CONFIG_DLM_CKV) += dlm_ckv.o +obj-$(CONFIG_DLM_CKV) += target_cluster_dlm.o diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c new file mode 100644 index 000000000000..2edae188811d --- /dev/null +++ b/drivers/target/target_cluster_dlm.c @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#include +#include +#include +#include +#include +#include + +#include "dlm_ckv.h" + +struct target_cluster_data { + struct dlm_ckv_bucket *bucket; + u32 local_nodeid; +}; + +struct target_lksb { + struct dlm_lksb lksb; + struct completion compl; +}; + +struct target_cluster_config { + struct config_group cg_group; + char cluster_name[DLM_LOCKSPACE_LEN]; +}; + +static struct target_cluster_config *to_cfg(struct config_item *i) +{ + return i ? container_of(to_config_group(i), struct target_cluster_config, cg_group) : NULL; +} + +static ssize_t target_cluster_cluster_name_show(struct config_item *item, char *buf) +{ + return sprintf(buf, "%s\n", to_cfg(item)->cluster_name); +} + +static ssize_t target_cluster_cluster_name_store(struct config_item *item, + const char *page, + size_t len) +{ + struct target_cluster_config *cfg = to_cfg(item); + + strscpy(cfg->cluster_name, page, DLM_LOCKSPACE_LEN); + if (cfg->cluster_name[strlen(cfg->cluster_name) - 1] == '\n') + cfg->cluster_name[strlen(cfg->cluster_name) - 1] = 0; + + return len; +} + +CONFIGFS_ATTR(target_cluster_, cluster_name); + +static struct configfs_attribute *cluster_attrs[] = { + &target_cluster_attr_cluster_name, + NULL, +}; + +static const struct config_item_type ci_cluster_config = { + .ct_item_ops = NULL, + .ct_attrs = cluster_attrs, + .ct_owner = THIS_MODULE, +}; + +static struct target_cluster_config cluster_cfg; + +static int target_init_dlm(struct se_device *dev) +{ + struct target_cluster_data *cluster_data; + char ls_name[INQUIRY_MODEL_LEN + INQUIRY_VPD_SERIAL_LEN + 1]; + int err = 0; + + BUG_ON(dev->cluster_data); + + snprintf(ls_name, sizeof(ls_name), "%s%s", + dev->t10_wwn.model, dev->t10_wwn.unit_serial); + + cluster_data = kzalloc(sizeof(*cluster_data), GFP_KERNEL); + if (!cluster_data) + return -ENOMEM; + + cluster_data->bucket = dlm_ckv_open_bucket(ls_name, + cluster_cfg.cluster_name, + dev); + if (!cluster_data->bucket) { + err = -EIO; + kfree(cluster_data); + dev->cluster_data = NULL; + return err; + } + dev->cluster_data = cluster_data; + + return err; +} + +static int target_cleanup_dlm(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + int res; + + res = dlm_ckv_close_bucket(cluster_data->bucket); + if (res) + pr_err("TARGET_CORE[%d]: closing bucket failed: %d\n", + dev->dev_index, res); + + kfree(dev->cluster_data); + dev->cluster_data = NULL; + + return 0; +} + +static void *target_caw_lock_dlm(struct se_device *dev, u64 lba) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + struct dlm_ckv_lock *caw_lock; + char name[DLM_RESNAME_MAXLEN]; + int res = -ENOMEM; + + snprintf(name, sizeof(name) - 1, "caw_lba_%llx", lba); + + caw_lock = dlm_ckv_create_lock(cluster_data->bucket, name); + if (!caw_lock) + goto done; + + res = dlm_ckv_lock_get(caw_lock); + if (res) { + dlm_ckv_free_lock(caw_lock); + caw_lock = NULL; + pr_warn("target_dlm_lock_wait failed with %d\n", res); + } + +done: + return caw_lock; +} + +static int target_caw_unlock_dlm(struct se_device *dev, void *lock) +{ + struct dlm_ckv_lock *caw_lock = lock; + int res; + + res = dlm_ckv_lock_release(caw_lock); + + dlm_ckv_free_lock(caw_lock); + + return res; +} + +struct target_cluster_ops dlm_cluster_ops = { + .name = "dlm", + .owner = THIS_MODULE, + .cg_group = &cluster_cfg.cg_group, + + .init = target_init_dlm, + .cleanup = target_cleanup_dlm, + .caw_lock = target_caw_lock_dlm, + .caw_unlock = target_caw_unlock_dlm, +}; + +static int __init target_cluster_dlm_module_init(void) +{ + config_group_init_type_name(&cluster_cfg.cg_group, + dlm_cluster_ops.name, + &ci_cluster_config); + + return target_cluster_impl_register(&dlm_cluster_ops); +} + +static void __exit target_cluster_dlm_module_exit(void) +{ + target_cluster_impl_unregister(&dlm_cluster_ops); +} + +MODULE_DESCRIPTION("TCM Cluster implementation over DLM"); +MODULE_AUTHOR("Dmitry Bogdanov "); +MODULE_LICENSE("GPL"); + +module_init(target_cluster_dlm_module_init); +module_exit(target_cluster_dlm_module_exit); From patchwork Mon Nov 29 09:00:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5845C19F2C for ; Wed, 3 Aug 2022 16:31:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238530AbiHCQbt (ORCPT ); Wed, 3 Aug 2022 12:31:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238455AbiHCQaY (ORCPT ); Wed, 3 Aug 2022 12:30:24 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3405A4D4C2; Wed, 3 Aug 2022 09:30:03 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9EA7B412F7; Wed, 3 Aug 2022 16:30:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544199; x=1661358600; bh=7RN/zxZr+Ny4BQZWDFA8hJi+M a8n+gx9BFIwRKD77sU=; b=hONNWuWrG0oZD8oVwh0p09z+fzezas0wM8ojUrQvY uJQWPS9iLRBezVvZ8dTIbFgigiK0J2LPgpEd1NgVak4bs2IQehH19V1uH5cggu2y lAGxAzcrGQ6MAIpC7/aaX3jMXQVyrh0HP+aGFj5I1sDJh489HGyXZgZmUZRxbOCH eA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hN1wkk2GzYSa; Wed, 3 Aug 2022 19:29:59 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 368EA412CC; Wed, 3 Aug 2022 19:29:23 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:23 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:22 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 38/48] target: cluster: store PR data in DLM cluster Date: Mon, 29 Nov 2021 12:00:46 +0300 Message-ID: <20220803162857.27770-39-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Follow the sequence bellow to sync all PR data throughout nodes in cluster: pr_lock() Handle PR OUT locally if (success) pr_sync() pr_unlock() where pr_sync() is a synchronical function. That will garantee an atomic update of PR data in a cluster. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 247 +++++++++++++++++++++++- drivers/target/target_core_device.c | 8 + drivers/target/target_core_fabric_lib.c | 1 + drivers/target/target_core_internal.h | 2 - drivers/target/target_core_pr.c | 7 + include/target/target_core_base.h | 6 + 6 files changed, 268 insertions(+), 3 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 2edae188811d..14465764eaaf 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -5,12 +5,21 @@ #include #include #include +#include +#include +#include "target_core_pr.h" #include "dlm_ckv.h" struct target_cluster_data { struct dlm_ckv_bucket *bucket; u32 local_nodeid; + struct mutex pr_lock_mutex; + struct dlm_ckv_lock *pr_lock; + struct dlm_ckv_kv *pr_data; + int reserved_node_id; + struct dlm_ckv_kv **pr_reg_kv; + size_t pr_reg_kv_len; }; struct target_lksb { @@ -60,6 +69,44 @@ static const struct config_item_type ci_cluster_config = { }; static struct target_cluster_config cluster_cfg; +struct pr_lvb { + u32 nr_registrants; + u32 pr_generation; + u8 version; + u8 pr_is_set; + u8 pr_type; + u8 pr_scope; + u8 pr_aptpl; + u8 reserved[3]; + u32 reserved_by_nodeid; +}; + +struct pr_reg_lvb { + u64 key; + u16 rtpi; + u8 version; + u8 is_holder; + u8 is_all_tg_pt; + u8 tid[PR_REG_TID_LEN]; +} __packed ; /* to fit in 256 bytes */ + +struct async_group { + refcount_t pending; + atomic_t status; + struct completion compl; +}; + +static int pr_reg_realloc(struct target_cluster_data *cluster_data, + size_t nr_registrants); + +static void group_compl_cb(void *arg, int res) +{ + struct async_group *grp = arg; + + atomic_cmpxchg(&grp->status, 0, res); + if (refcount_dec_and_test(&grp->pending)) + complete(&grp->compl); +} static int target_init_dlm(struct se_device *dev) { @@ -68,6 +115,7 @@ static int target_init_dlm(struct se_device *dev) int err = 0; BUG_ON(dev->cluster_data); + BUILD_BUG_ON(sizeof(struct pr_reg_lvb) > DLM_CKV_VALUE_MAX_SIZE); snprintf(ls_name, sizeof(ls_name), "%s%s", dev->t10_wwn.model, dev->t10_wwn.unit_serial); @@ -85,16 +133,42 @@ static int target_init_dlm(struct se_device *dev) dev->cluster_data = NULL; return err; } + + mutex_init(&cluster_data->pr_lock_mutex); + cluster_data->pr_lock = dlm_ckv_create_lock(cluster_data->bucket, "pr_lock"); + if (!cluster_data->pr_lock) + goto fail; + + cluster_data->pr_data = dlm_ckv_create_kv(cluster_data->bucket, "pr_data"); + if (!cluster_data->pr_data) + goto fail; + dev->cluster_data = cluster_data; return err; +fail: + if (cluster_data->pr_lock) + dlm_ckv_free_lock(cluster_data->pr_lock); + if (cluster_data->pr_data) + dlm_ckv_free_kv(cluster_data->pr_data); + dlm_ckv_close_bucket(cluster_data->bucket); + kfree(cluster_data); + return -EIO; } static int target_cleanup_dlm(struct se_device *dev) { struct target_cluster_data *cluster_data = dev->cluster_data; int res; + int i; + + for (i = 0; i < cluster_data->pr_reg_kv_len; i++) + if (cluster_data->pr_reg_kv[i]) + dlm_ckv_free_kv(cluster_data->pr_reg_kv[i]); + kfree(cluster_data->pr_reg_kv); + dlm_ckv_free_lock(cluster_data->pr_lock); + dlm_ckv_free_kv(cluster_data->pr_data); res = dlm_ckv_close_bucket(cluster_data->bucket); if (res) pr_err("TARGET_CORE[%d]: closing bucket failed: %d\n", @@ -142,7 +216,175 @@ static int target_caw_unlock_dlm(struct se_device *dev, void *lock) return res; } -struct target_cluster_ops dlm_cluster_ops = { +static int target_pr_lock_dlm(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + int res = -ENOMEM; + + mutex_lock(&cluster_data->pr_lock_mutex); + res = dlm_ckv_lock_get(cluster_data->pr_lock); + if (res) { + pr_warn("target_dlm_lock_wait failed with %d\n", res); + mutex_unlock(&cluster_data->pr_lock_mutex); + } + + return res; +} + +static int target_pr_unlock_dlm(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + int res; + + res = dlm_ckv_lock_release(cluster_data->pr_lock); + mutex_unlock(&cluster_data->pr_lock_mutex); + + return res; +} + +static int target_get_nr_registrants(struct se_device *dev) +{ + struct t10_pr_registration *pr_reg; + int count = 0; + + list_for_each_entry(pr_reg, + &dev->t10_pr.registration_list, pr_reg_list) + count++; + + return count; +} + +static int pr_reg_realloc(struct target_cluster_data *cluster_data, + size_t nr_registrants) +{ + char key_name[16]; + void *p; + int res = 0; + int i; + + if (nr_registrants > cluster_data->pr_reg_kv_len) { + p = krealloc(cluster_data->pr_reg_kv, + nr_registrants * sizeof(struct dlm_ckv_kv *), + GFP_KERNEL | __GFP_ZERO); + if (!p) { + res = -ENOMEM; + goto done; + } + cluster_data->pr_reg_kv = p; + for (i = cluster_data->pr_reg_kv_len; + i < nr_registrants; i++) { + snprintf(key_name, sizeof(key_name), "pr_reg.%d", i); + if (!cluster_data->pr_reg_kv[i]) + cluster_data->pr_reg_kv[i] = dlm_ckv_create_kv( + cluster_data->bucket, + key_name); + if (!cluster_data->pr_reg_kv[i]) { + res = -ENOMEM; + goto fail_create; + } + } + cluster_data->pr_reg_kv_len = nr_registrants; + } +done: + return res; +fail_create: + /* free newly created objects */ + for (i = cluster_data->pr_reg_kv_len; + i < nr_registrants; i++) { + if (!cluster_data->pr_reg_kv[i]) + dlm_ckv_free_kv(cluster_data->pr_reg_kv[i]); + } + return res; +} + +static int target_pr_sync_dlm(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + struct t10_pr_registration *pr_reg; + struct pr_reg_lvb pr_reg_data; + struct async_group grp; + struct pr_lvb pr_data; + int i = 0; + int res; + + spin_lock(&dev->t10_pr.registration_lock); + + pr_data.version = 1; + pr_data.pr_generation = atomic_read(&dev->t10_pr.pr_generation); + pr_data.nr_registrants = target_get_nr_registrants(dev); + pr_data.pr_is_set = !!dev->dev_pr_res_holder; + pr_data.pr_aptpl = dev->t10_pr.pr_aptpl_active; + if (dev->dev_pr_res_holder) { + pr_data.pr_type = dev->dev_pr_res_holder->pr_res_type; + pr_data.pr_scope = dev->dev_pr_res_holder->pr_res_scope; + } + pr_data.reserved_by_nodeid = cluster_data->reserved_node_id; + + spin_unlock(&dev->t10_pr.registration_lock); + + res = dlm_ckv_set(cluster_data->pr_data, (char *)&pr_data, + sizeof(pr_data)); + if (res) + goto done; + + + res = pr_reg_realloc(cluster_data, pr_data.nr_registrants); + if (res) + goto done; + + if (pr_data.nr_registrants == 0) + goto skip_pr_reg; + + refcount_set(&grp.pending, 1); /* 1 for a loop */ + atomic_set(&grp.status, 0); + init_completion(&grp.compl); + + list_for_each_entry(pr_reg, + &dev->t10_pr.registration_list, pr_reg_list) { + + pr_reg_data.key = pr_reg->pr_res_key; + pr_reg_data.rtpi = pr_reg->tg_pt_sep_rtpi; + pr_reg_data.version = 1; + pr_reg_data.is_holder = pr_reg->pr_res_holder; + pr_reg_data.is_all_tg_pt = !!pr_reg->pr_reg_all_tg_pt; + target_get_pr_transport_id(pr_reg, pr_reg_data.tid); + + refcount_inc(&grp.pending); + res = dlm_ckv_set_async(cluster_data->pr_reg_kv[i], + (char *)&pr_reg_data, + sizeof(pr_reg_data), + group_compl_cb, + &grp); + if (res) { + pr_err("TARGET_CORE[%d]: set pr_reg_%d failed: %d\n", + dev->dev_index, i, res); + refcount_dec(&grp.pending); + break; + } + i++; + } + group_compl_cb(&grp, 0); + res = wait_for_completion_timeout(&grp.compl, 60 * HZ); + if (!res) { + pr_err("TARGET_CORE[%d]: timeout of waiting for dlm_ckv_set_async\n", + dev->dev_index); + goto done; + } + res = atomic_read(&grp.status); + if (res) { + pr_err("TARGET_CORE[%d]: fail of group for dlm_ckv_set_async %d\n", + dev->dev_index, res); + goto done; + } + +skip_pr_reg: + +done: + + return res; +} + +const struct target_cluster_ops dlm_cluster_ops = { .name = "dlm", .owner = THIS_MODULE, .cg_group = &cluster_cfg.cg_group, @@ -151,6 +393,9 @@ struct target_cluster_ops dlm_cluster_ops = { .cleanup = target_cleanup_dlm, .caw_lock = target_caw_lock_dlm, .caw_unlock = target_caw_unlock_dlm, + .pr_lock = target_pr_lock_dlm, + .pr_unlock = target_pr_unlock_dlm, + .pr_sync = target_pr_sync_dlm, }; static int __init target_cluster_dlm_module_init(void) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 9c7f4dc1b85d..f5da3fc17ad1 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -782,6 +782,11 @@ static int target_caw_unlock_nodlm(struct se_device *dev, void *lock) return 0; } +int target_dummy_nodlm(struct se_device *dev) +{ + return 0; +} + const struct target_cluster_ops nodlm_cluster_ops = { .name = "single", .owner = NULL, @@ -789,6 +794,9 @@ const struct target_cluster_ops nodlm_cluster_ops = { .cleanup = target_cleanup_nodlm, .caw_lock = target_caw_lock_nodlm, .caw_unlock = target_caw_unlock_nodlm, + .pr_lock = target_dummy_nodlm, + .pr_unlock = target_dummy_nodlm, + .pr_sync = target_dummy_nodlm, }; struct se_device *target_alloc_device(struct se_hba *hba, const char *name) diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 0eb03d7b9e78..8232d872db35 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -382,6 +382,7 @@ int target_get_pr_transport_id( return len; } +EXPORT_SYMBOL(target_get_pr_transport_id); int target_cmp_pr_transport_id( struct t10_pr_registration *pr_reg, diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 58261c716776..8669ff6f48e9 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -107,8 +107,6 @@ int target_fabric_setup_cits(struct target_fabric_configfs *); /* target_core_fabric_lib.c */ int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg); -int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, - unsigned char *buf); int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, unsigned char *tid); int target_gen_pr_transport_id(struct t10_pr_registration *pr_reg, diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index a984987f0a82..a9fe192f5fe4 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -3560,6 +3560,8 @@ target_scsi3_emulate_pr_out(struct se_cmd *cmd) return TCM_PARAMETER_LIST_LENGTH_ERROR; } + if (dev->cl_ops->pr_lock(dev)) + return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; /* * (core_scsi3_emulate_pro_* function parameters * are defined by spc4r17 Table 174: @@ -3598,9 +3600,14 @@ target_scsi3_emulate_pr_out(struct se_cmd *cmd) default: pr_err("Unknown PERSISTENT_RESERVE_OUT service" " action: 0x%02x\n", sa); + dev->cl_ops->pr_unlock(dev); return TCM_INVALID_CDB_FIELD; } + if (!ret) + dev->cl_ops->pr_sync(dev); + dev->cl_ops->pr_unlock(dev); + if (!ret) target_complete_cmd(cmd, SAM_STAT_GOOD); return ret; diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index fbeb4777654b..9edf3c1ac204 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -787,6 +787,9 @@ struct target_cluster_ops { int (*cleanup)(struct se_device *dev); void *(*caw_lock)(struct se_device *dev, u64 lba); int (*caw_unlock)(struct se_device *dev, void *lock); + int (*pr_lock)(struct se_device *dev); + int (*pr_unlock)(struct se_device *dev); + int (*pr_sync)(struct se_device *dev); }; struct se_device { @@ -999,4 +1002,7 @@ static inline void target_free_tag(struct se_session *sess, struct se_cmd *cmd) int target_cluster_impl_register(const struct target_cluster_ops *ops); void target_cluster_impl_unregister(const struct target_cluster_ops *ops); +int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, + unsigned char *buf); + #endif /* TARGET_CORE_BASE_H */ From patchwork Mon Dec 6 13:39:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F321C25B06 for ; Wed, 3 Aug 2022 16:31:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236822AbiHCQb4 (ORCPT ); Wed, 3 Aug 2022 12:31:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238548AbiHCQaj (ORCPT ); Wed, 3 Aug 2022 12:30:39 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 486084D80E; Wed, 3 Aug 2022 09:30:05 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id C8E144136D; Wed, 3 Aug 2022 16:30:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544201; x=1661358602; bh=pOcEOZn7zbrsXU9QO0/YYkQvP 8Qm6scl2kk4JEHpRg4=; b=SWs8hly5oo4vgaiCMG/7lW462sYfhXNQ0wy/uOSTl TTxslmXiXnLXLjabbiZYlt21MuQs5+oda2e1/q8nQ11t1FKos1NvZkY25WWwOVLH LGRnexgOIp2Kgsfn0UOglmHji7DAXV0YU88ifNtf6B5AIvcTys21hwwtTlqT0pX9 qQ= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IgCiEk_zx9-V; Wed, 3 Aug 2022 19:30:01 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 75EC4412EE; Wed, 3 Aug 2022 19:29:23 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:23 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:22 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 39/48] target: cluster: read PR data from cluster Date: Mon, 6 Dec 2021 16:39:54 +0300 Message-ID: <20220803162857.27770-40-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Notify other nodes in cluster that PR data updated. On notification do the following under PR cluster lock to read PR data: Update an existing registrants. Remove an outdated registrants (absent in cluster data). Create new registrants. Update Reservation status. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 304 ++++++++++++++++++++++++ drivers/target/target_core_device.c | 1 + drivers/target/target_core_fabric_lib.c | 3 +- drivers/target/target_core_internal.h | 5 - drivers/target/target_core_pr.c | 8 +- drivers/target/target_core_pr.h | 1 + include/target/target_core_base.h | 5 + 7 files changed, 320 insertions(+), 7 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 14465764eaaf..6b7cb3175e3a 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -17,6 +17,7 @@ struct target_cluster_data { struct mutex pr_lock_mutex; struct dlm_ckv_lock *pr_lock; struct dlm_ckv_kv *pr_data; + struct dlm_ckv_notify *pr_sync_notify; int reserved_node_id; struct dlm_ckv_kv **pr_reg_kv; size_t pr_reg_kv_len; @@ -96,6 +97,7 @@ struct async_group { struct completion compl; }; +static void target_pr_sync_cb(void *arg); static int pr_reg_realloc(struct target_cluster_data *cluster_data, size_t nr_registrants); @@ -143,10 +145,17 @@ static int target_init_dlm(struct se_device *dev) if (!cluster_data->pr_data) goto fail; + cluster_data->pr_sync_notify = dlm_ckv_create_notification( + cluster_data->bucket, "pr_sync", target_pr_sync_cb); + if (!cluster_data->pr_sync_notify) + goto fail; + dev->cluster_data = cluster_data; return err; fail: + if (cluster_data->pr_sync_notify) + dlm_ckv_free_notification(cluster_data->pr_sync_notify); if (cluster_data->pr_lock) dlm_ckv_free_lock(cluster_data->pr_lock); if (cluster_data->pr_data) @@ -167,6 +176,7 @@ static int target_cleanup_dlm(struct se_device *dev) dlm_ckv_free_kv(cluster_data->pr_reg_kv[i]); kfree(cluster_data->pr_reg_kv); + dlm_ckv_free_notification(cluster_data->pr_sync_notify); dlm_ckv_free_lock(cluster_data->pr_lock); dlm_ckv_free_kv(cluster_data->pr_data); res = dlm_ckv_close_bucket(cluster_data->bucket); @@ -379,11 +389,305 @@ static int target_pr_sync_dlm(struct se_device *dev) skip_pr_reg: + /* request to update PR data on other nodes in cluster */ + dlm_ckv_notify(cluster_data->pr_sync_notify); + done: return res; } +static int target_update_pr_reg(struct se_device *dev, + struct t10_pr_registration *pr_reg, + const struct pr_lvb *pr_data, + const struct pr_reg_lvb *pr_reg_data, + struct t10_pr_registration **pr_reg_res_holder) +{ + /* update existing registrant */ + pr_reg->pr_res_key = pr_reg_data->key; + pr_reg->pr_reg_all_tg_pt = pr_reg_data->is_all_tg_pt; + + if (pr_reg_data->is_holder) + *pr_reg_res_holder = pr_reg; + + return 0; +} + +static struct t10_pr_registration * +target_create_pr_reg(struct se_device *dev, + struct pr_lvb *pr_data, + struct pr_reg_lvb *pr_reg_data) +{ + struct t10_pr_registration *pr_reg = NULL; + struct se_lun *lun, *lun_tmp; + struct se_lun_acl *lacl_tmp; + struct se_node_acl *nacl_tmp; + struct se_dev_entry *deve; + char i_buf[PR_REG_ISID_ID_LEN]; + u8 initiatorname[PR_REG_ISID_LEN]; + char *isid = NULL; + + target_parse_pr_out_transport_id(pr_reg_data->tid, initiatorname, &isid); + + spin_lock(&dev->se_port_lock); + list_for_each_entry_safe(lun, lun_tmp, &dev->dev_sep_list, lun_dev_link) { + if (!percpu_ref_tryget_live(&lun->lun_ref)) + continue; + + if (lun->lun_tpg->tpg_rtpi != pr_reg_data->rtpi) { + percpu_ref_put(&lun->lun_ref); + continue; + } + spin_unlock(&dev->se_port_lock); + + spin_lock(&lun->lun_deve_lock); + list_for_each_entry(deve, &lun->lun_deve_list, lun_link) { + if (!deve->se_lun_acl) + continue; + + lacl_tmp = deve->se_lun_acl; + nacl_tmp = lacl_tmp->se_lun_nacl; + + if (strcmp(nacl_tmp->initiatorname, initiatorname)) + continue; + + kref_get(&deve->pr_kref); + pr_reg = __core_scsi3_do_alloc_registration( + dev, nacl_tmp, + nacl_tmp->initiatorname, lun->unpacked_lun, + pr_reg_data->rtpi, + deve, lacl_tmp->mapped_lun, + isid, + pr_reg_data->key, + pr_reg_data->is_all_tg_pt, + pr_data->pr_aptpl); + + percpu_ref_put(&lun->lun_ref); + spin_unlock(&lun->lun_deve_lock); + goto out; + } + percpu_ref_put(&lun->lun_ref); + spin_unlock(&lun->lun_deve_lock); + spin_lock(&dev->se_port_lock); + } + spin_unlock(&dev->se_port_lock); +out: + + if (!pr_reg) { + /* target not yet configured, + * allocate registration like for APTPL case + */ + pr_reg = __core_scsi3_do_alloc_registration( + dev, NULL, initiatorname, 0, + pr_reg_data->rtpi, + NULL, 0, + isid, + pr_reg_data->key, + pr_reg_data->is_all_tg_pt, + pr_data->pr_aptpl); + } + + if (pr_reg) { + spin_lock(&dev->t10_pr.registration_lock); + memcpy(pr_reg->pr_tid, pr_reg_data->tid, PR_REG_TID_LEN); + list_add_tail(&pr_reg->pr_reg_list, + &dev->t10_pr.registration_list); + + /* + * Drop deve->pr_kref obtained in __core_scsi3_do_alloc_registration() + */ + rcu_read_lock(); + deve = pr_reg->pr_reg_deve; + if (deve) { + set_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); + kref_put(&deve->pr_kref, target_pr_kref_release); + pr_reg->pr_reg_deve = NULL; + } + rcu_read_unlock(); + + memset(&i_buf[0], 0, PR_REG_ISID_ID_LEN); + core_pr_dump_initiator_port(pr_reg, i_buf, + PR_REG_ISID_ID_LEN); + pr_debug("SPC-3 PR Service Action: REGISTER Initiator Node: %s%s\n", + initiatorname, i_buf); + pr_debug("SPC-3 PR registration on Target Port: %d\n", + pr_reg->tg_pt_sep_rtpi); + pr_debug("SPC-3 PR for %s TCM Subsystem %s Object Target Port(s)\n", + (pr_reg->pr_reg_all_tg_pt) ? "ALL" : "SINGLE", + dev->transport->name); + pr_debug("SPC-3 PR SA Res Key: 0x%016llx PRgeneration: 0x%08x APTPL: %d\n", + pr_reg->pr_res_key, + pr_reg->pr_res_generation, + pr_reg->pr_reg_aptpl); + spin_unlock(&dev->t10_pr.registration_lock); + return pr_reg; + } + + pr_warn("TARGET_CORE[%d]: PR data from cluster is invalid: could not find local nacl/deve for Initiator %s - RTPI %d\n", + dev->dev_index, initiatorname, pr_reg_data->rtpi); + + return NULL; +} + +static void target_pr_sync_cb(void *arg) +{ + struct se_device *dev = arg; + struct target_cluster_data *cluster_data = dev->cluster_data; + struct t10_pr_registration *pr_reg, *pr_reg_tmp; + struct t10_pr_registration *pr_reg_res_holder = NULL; + struct t10_pr_registration *pr_prev_res_holder = NULL; + struct pr_reg_lvb *pr_reg_data = NULL; + LIST_HEAD(to_be_deleted_list); + struct async_group grp; + struct pr_lvb pr_data; + bool res_to_delete = false; + bool was_held; + u8 was_type; + u8 was_scope; + bool found; + int i = 0; + int res; + + res = dlm_ckv_get(cluster_data->pr_data, (char *)&pr_data, sizeof(pr_data)); + if (res) + goto done; + + if (!pr_data.version) { + pr_info("TARGET_CORE[%d]: PR data from cluster is invalid\n", + dev->dev_index); + goto done; + } + + pr_reg_data = kzalloc(sizeof(struct pr_reg_lvb) * pr_data.nr_registrants, + GFP_KERNEL); + if (!pr_reg_data) { + res = -ENOMEM; + goto done; + } + + res = pr_reg_realloc(cluster_data, pr_data.nr_registrants); + if (res) + goto done; + + if (pr_data.nr_registrants == 0) + goto skip_pr_reg; + + refcount_set(&grp.pending, 1); /* 1 for a loop */ + atomic_set(&grp.status, 0); + init_completion(&grp.compl); + + for (i = 0; i < pr_data.nr_registrants; ++i) { + refcount_inc(&grp.pending); + res = dlm_ckv_get_async(cluster_data->pr_reg_kv[i], (char *)(pr_reg_data + i), + sizeof(struct pr_reg_lvb), group_compl_cb, &grp); + if (res) { + refcount_dec(&grp.pending); + break; + } + } + group_compl_cb(&grp, 0); + res = wait_for_completion_timeout(&grp.compl, 60 * HZ); + if (!res) { + pr_err("TARGET_CORE[%d]: timeout of waiting for dlm_ckv_get_async\n", + dev->dev_index); + goto done; + } + res = atomic_read(&grp.status); + if (res) { + pr_err("TARGET_CORE[%d]: fail of group for dlm_ckv_get_async %d\n", + dev->dev_index, res); + goto done; + } + +skip_pr_reg: + /* + * Update existing registrations + */ + spin_lock(&dev->t10_pr.registration_lock); + + was_held = !!dev->dev_pr_res_holder; + pr_prev_res_holder = dev->dev_pr_res_holder; + if (was_held) { + was_scope = pr_prev_res_holder->pr_res_scope; + was_type = pr_prev_res_holder->pr_res_type; + } + + list_for_each_entry_safe(pr_reg, pr_reg_tmp, &dev->t10_pr.registration_list, + pr_reg_list) { + found = false; + + for (i = 0; i < pr_data.nr_registrants; ++i) { + if (!pr_reg_data[i].version) + continue; + + if (pr_reg->tg_pt_sep_rtpi == pr_reg_data[i].rtpi && + !target_cmp_pr_transport_id(pr_reg, pr_reg_data[i].tid)) { + found = true; + /* mark existing registrants */ + pr_reg_data[i].version = 0; + target_update_pr_reg(dev, pr_reg, &pr_data, + &pr_reg_data[i], + &pr_reg_res_holder); + break; + } + } + + if (!found) + list_move_tail(&pr_reg->pr_reg_list, &to_be_deleted_list); + } + + /* deregister obsolete entries */ + list_for_each_entry_safe(pr_reg, pr_reg_tmp, &to_be_deleted_list, + pr_reg_list) { + if (dev->dev_pr_res_holder != pr_reg) + __core_scsi3_free_registration(dev, pr_reg, NULL, 0); + else + res_to_delete = true; + } + spin_unlock(&dev->t10_pr.registration_lock); + + /* register new entries */ + for (i = 0; i < pr_data.nr_registrants; ++i) { + /* skip existing registrants */ + if (!pr_reg_data[i].version) + continue; + + pr_reg = target_create_pr_reg(dev, &pr_data, &pr_reg_data[i]); + if (!pr_reg) + pr_err("TARGET_CORE[%d]: can not create new registration\n", + dev->dev_index); + else if (pr_reg_data[i].is_holder) + pr_reg_res_holder = pr_reg; + } + + /* update general data */ + atomic_set(&dev->t10_pr.pr_generation, pr_data.pr_generation); + dev->t10_pr.pr_aptpl_active = pr_data.pr_aptpl; + + /* update reservation */ + spin_lock(&dev->dev_reservation_lock); + if (pr_prev_res_holder && + ((pr_reg_res_holder != pr_prev_res_holder) || + (was_type != pr_reg_res_holder->pr_res_type))) + __core_scsi3_complete_pro_release(dev, pr_prev_res_holder, 0, 0); + + if (pr_reg_res_holder) + __core_scsi3_set_reservation(dev, pr_reg_res_holder, + pr_data.pr_scope, pr_data.pr_type); + spin_unlock(&dev->dev_reservation_lock); + + if (res_to_delete) { + spin_lock(&dev->t10_pr.registration_lock); + __core_scsi3_free_registration(dev, pr_prev_res_holder, NULL, 0); + spin_unlock(&dev->t10_pr.registration_lock); + } + + core_scsi3_update_and_write_aptpl(dev, dev->t10_pr.pr_aptpl_active); + +done: + kfree(pr_reg_data); +} + const struct target_cluster_ops dlm_cluster_ops = { .name = "dlm", .owner = THIS_MODULE, diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index f5da3fc17ad1..297bf5985f48 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -355,6 +355,7 @@ void target_pr_kref_release(struct kref *kref) pr_kref); complete(&deve->pr_comp); } +EXPORT_SYMBOL(target_pr_kref_release); /* * Establish UA condition on SCSI device - all LUNs diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 8232d872db35..2937bf4f6865 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -396,7 +396,7 @@ int target_cmp_pr_transport_id( return memcmp(pr_reg->pr_tid, tid, len1); } - +EXPORT_SYMBOL(target_cmp_pr_transport_id); int target_gen_pr_transport_id( struct t10_pr_registration *pr_reg, @@ -444,3 +444,4 @@ u32 target_parse_pr_out_transport_id( *port_nexus_ptr = NULL; return 24; } +EXPORT_SYMBOL(target_parse_pr_out_transport_id); diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 8669ff6f48e9..89f0ddca35ed 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -65,7 +65,6 @@ extern struct t10_alua_lu_gp *default_lu_gp; /* target_core_device.c */ struct se_dev_entry *core_get_se_deve_from_rtpi(struct se_node_acl *, u16); -void target_pr_kref_release(struct kref *); void core_free_device_list_for_node(struct se_node_acl *, struct se_portal_group *); void core_update_device_list_access(u64, bool, struct se_node_acl *); @@ -107,14 +106,10 @@ int target_fabric_setup_cits(struct target_fabric_configfs *); /* target_core_fabric_lib.c */ int target_get_pr_transport_id_len(struct t10_pr_registration *pr_reg); -int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, - unsigned char *tid); int target_gen_pr_transport_id(struct t10_pr_registration *pr_reg, int proto_id, const char *initiatorname, unsigned char *isid); -u32 target_parse_pr_out_transport_id( - const char *buf, char *initiatorname, char **port_nexus_ptr); /* target_core_hba.c */ struct se_hba *core_alloc_hba(const char *, u32, u32); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index a9fe192f5fe4..fe2f64826e9e 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -52,6 +52,7 @@ void core_pr_dump_initiator_port( snprintf(buf, size, ",i,0x%s", pr_reg->pr_reg_isid); } +EXPORT_SYMBOL(core_pr_dump_initiator_port); enum register_type { REGISTER, @@ -677,6 +678,7 @@ struct t10_pr_registration *__core_scsi3_do_alloc_registration( return pr_reg; } +EXPORT_SYMBOL(__core_scsi3_do_alloc_registration); static int core_scsi3_lunacl_depend_item(struct se_dev_entry *); static void core_scsi3_lunacl_undepend_item(struct se_dev_entry *); @@ -1296,6 +1298,7 @@ void __core_scsi3_free_registration( */ list_add_tail(&pr_reg->pr_reg_abort_list, preempt_and_abort_list); } +EXPORT_SYMBOL(__core_scsi3_free_registration); void core_scsi3_free_all_registrations( struct se_device *dev) @@ -1925,7 +1928,7 @@ static int __core_scsi3_write_aptpl_to_file( * Clear the APTPL metadata if APTPL has been disabled, otherwise * write out the updated metadata to struct file for this SCSI device. */ -static sense_reason_t core_scsi3_update_and_write_aptpl(struct se_device *dev, bool aptpl) +sense_reason_t core_scsi3_update_and_write_aptpl(struct se_device *dev, bool aptpl) { unsigned char *buf; int rc, len = PR_APTPL_BUF_LEN; @@ -1965,6 +1968,7 @@ static sense_reason_t core_scsi3_update_and_write_aptpl(struct se_device *dev, b pr_debug("SPC-3 PR: Set APTPL Bit Activated\n"); return 0; } +EXPORT_SYMBOL(core_scsi3_update_and_write_aptpl); static sense_reason_t core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key, @@ -2214,6 +2218,7 @@ void __core_scsi3_set_reservation(struct se_device *dev, pr_reg->pr_iport, i_buf); } +EXPORT_SYMBOL(__core_scsi3_set_reservation); static sense_reason_t core_scsi3_pro_reserve(struct se_cmd *cmd, int type, int scope, u64 res_key) @@ -2430,6 +2435,7 @@ void __core_scsi3_complete_pro_release( */ pr_reg->pr_res_holder = pr_reg->pr_res_type = pr_reg->pr_res_scope = 0; } +EXPORT_SYMBOL(__core_scsi3_complete_pro_release); static sense_reason_t core_scsi3_emulate_pro_release(struct se_cmd *cmd, int type, int scope, diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index 589fdac5470f..e7eff50a01f2 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -68,6 +68,7 @@ extern int core_scsi3_alloc_aptpl_registration( extern int core_scsi3_check_aptpl_registration(struct se_device *, struct se_portal_group *, struct se_lun *, struct se_node_acl *, u64); +sense_reason_t core_scsi3_update_and_write_aptpl(struct se_device *dev, bool aptpl); extern void core_scsi3_free_all_registrations(struct se_device *); extern unsigned char *core_scsi3_pr_dump_type(int); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 9edf3c1ac204..b0f0a6b93f6e 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -1004,5 +1004,10 @@ void target_cluster_impl_unregister(const struct target_cluster_ops *ops); int target_get_pr_transport_id(struct t10_pr_registration *pr_reg, unsigned char *buf); +u32 target_parse_pr_out_transport_id( + const char *buf, char *initiatorname, char **port_nexus_ptr); +int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, + unsigned char *tid); +void target_pr_kref_release(struct kref *kref); #endif /* TARGET_CORE_BASE_H */ From patchwork Fri Dec 24 09:52:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E66DFC19F29 for ; Wed, 3 Aug 2022 16:31:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238087AbiHCQb6 (ORCPT ); Wed, 3 Aug 2022 12:31:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238545AbiHCQaj (ORCPT ); Wed, 3 Aug 2022 12:30:39 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D22545079; Wed, 3 Aug 2022 09:30:06 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id EA42141373; Wed, 3 Aug 2022 16:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544203; x=1661358604; bh=1bLe2bmEe6OtncXjy7yrZAf4g VDDxB/SLGnZp0Uh5Gw=; b=IXdS4qrfffxQKjPxMvScNC+RsHcvrXvR2/y4WZwo8 WQJMeyqnE6PJE+zUZTxtlO/eoBfJ93v2m7ERahFfu7JyHTxbVbPC/sk9ig4Jah09 3yFtbA50T7gK6pUwNuTMKMDBViJJAMl7pF1to9OhPnFBc0uTWF37FeDgnsiO5pjP uc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hxvtP1K1Fjci; Wed, 3 Aug 2022 19:30:03 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id DA4A2412E3; Wed, 3 Aug 2022 19:29:23 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:23 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:23 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 40/48] target: cluster: sync PR for dynamic acls Date: Fri, 24 Dec 2021 12:52:06 +0300 Message-ID: <20220803162857.27770-41-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Dynamic acls are not present in remote target, so use NULL for nacl pointer for such registrants. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 10 ++++++++++ drivers/target/target_core_pr.c | 17 +++++++++++------ 2 files changed, 21 insertions(+), 6 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 6b7cb3175e3a..6dbe1a0b21e2 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -466,6 +466,16 @@ target_create_pr_reg(struct se_device *dev, spin_unlock(&lun->lun_deve_lock); goto out; } + /* deve not found - that is a dynamic acl */ + pr_reg = __core_scsi3_do_alloc_registration( + dev, NULL, initiatorname, lun->unpacked_lun, + pr_reg_data->rtpi, + NULL, lun->unpacked_lun, + isid, + pr_reg_data->key, + pr_reg_data->is_all_tg_pt, + pr_data->pr_aptpl); + percpu_ref_put(&lun->lun_ref); spin_unlock(&lun->lun_deve_lock); spin_lock(&dev->se_port_lock); diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index fe2f64826e9e..22f2aa91cf2e 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -644,10 +644,13 @@ struct t10_pr_registration *__core_scsi3_do_alloc_registration( * * Otherwise, locate se_dev_entry now and obtain a reference until * registration completes in __core_scsi3_add_registration(). + * + * NULL nacl means that this registration is for dynamic acl on a remote + * target - no need to find a deve. */ if (dest_deve) { pr_reg->pr_reg_deve = dest_deve; - } else { + } else if (nacl) { rcu_read_lock(); pr_reg->pr_reg_deve = target_nacl_find_deve(nacl, mapped_lun); if (!pr_reg->pr_reg_deve) { @@ -1270,11 +1273,13 @@ void __core_scsi3_free_registration( cpu_relax(); } - rcu_read_lock(); - deve = target_nacl_find_deve(nacl, pr_reg->pr_res_mapped_lun); - if (deve) - clear_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); - rcu_read_unlock(); + if (nacl) { + rcu_read_lock(); + deve = target_nacl_find_deve(nacl, pr_reg->pr_res_mapped_lun); + if (deve) + clear_bit(DEF_PR_REG_ACTIVE, &deve->deve_flags); + rcu_read_unlock(); + } spin_lock(&pr_tmpl->registration_lock); pr_debug("SPC-3 PR Service Action: UNREGISTER Initiator Node: %s%s\n", From patchwork Fri Dec 17 09:27:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFAEEC19F2D for ; Wed, 3 Aug 2022 16:31:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236041AbiHCQby (ORCPT ); Wed, 3 Aug 2022 12:31:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238563AbiHCQaj (ORCPT ); Wed, 3 Aug 2022 12:30:39 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A708C4F1A4; Wed, 3 Aug 2022 09:30:07 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 047C241374; Wed, 3 Aug 2022 16:30:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544204; x=1661358605; bh=jvuc0gW1ZdHDie47Gs0pEbRvc vgWdNhP9Gt5CHLqipY=; b=HJGWRc+RwzdyLdXtsOJ4k9GyvSwzOUB8BJo4b8Cg8 vq81FWml6ve6L7h0cEHHpbXN0IMKIdAzb8as4npD/CHvaOPZUsSySf0uSvuykyyj JMpdNqrvQq59gbuSCIxeTaoLqbXQLrAXzzP+iFxkJCoeGLUN+zo11+1xRnV6xOI4 wE= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4wQcWWR4r1bI; Wed, 3 Aug 2022 19:30:04 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 38DDF41291; Wed, 3 Aug 2022 19:29:24 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:24 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:23 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 41/48] target: cluster: sync-up PR data on cluster join Date: Fri, 17 Dec 2021 12:27:25 +0300 Message-ID: <20220803162857.27770-42-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Read PR data from cluster on joining to the cluster. acl/lun/tpg are supposed to be configured already. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 6dbe1a0b21e2..16391a8661fa 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -152,6 +152,11 @@ static int target_init_dlm(struct se_device *dev) dev->cluster_data = cluster_data; + /* initial sync-up on joining the cluster */ + dlm_ckv_lock_get(cluster_data->pr_lock); + target_pr_sync_cb(dev); + dlm_ckv_lock_release(cluster_data->pr_lock); + return err; fail: if (cluster_data->pr_sync_notify) From patchwork Thu Dec 16 10:20:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6259C3F6B0 for ; Wed, 3 Aug 2022 16:31:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236105AbiHCQbw (ORCPT ); Wed, 3 Aug 2022 12:31:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238574AbiHCQat (ORCPT ); Wed, 3 Aug 2022 12:30:49 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE8F34F65E; Wed, 3 Aug 2022 09:30:08 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 35E6741378; Wed, 3 Aug 2022 16:30:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544205; x=1661358606; bh=O/iIPyEyPOsenWf7AyLBCWTes vHKmHvcvCNKpQOR8uo=; b=eX8v/l0lMH13N6dAIbiop1YOdo8frZTIsWITeQYox RnqxBep+O0vIKOCKlRhVvvWML020GUI66U/YzktJaFITD/SMp+jxGuGUgK+ehJB3 unZE/pw9rVBUSCNAgwVbIwcusDbpFA0jdOD63yqsj2RTYjAR66O8MJpK1Lz7pdjP L8= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id umlnsauycaoX; Wed, 3 Aug 2022 19:30:05 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 81217412E6; Wed, 3 Aug 2022 19:29:24 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:24 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:23 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 42/48] target: cluster: sync SPC-2 reservations Date: Thu, 16 Dec 2021 13:20:31 +0300 Message-ID: <20220803162857.27770-43-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Reservation status is stored in DLM_CKV as part of PR data. If a reservation holder node is left the cluster other nodes will clear reservation status locally. Signed-off-by: Dmitry Bogdanov --- drivers/target/dlm_ckv.c | 16 ++++++- drivers/target/dlm_ckv.h | 4 +- drivers/target/target_cluster_dlm.c | 59 +++++++++++++++++++++++++- drivers/target/target_core_device.c | 10 +++++ drivers/target/target_core_pr.c | 22 +++++++--- drivers/target/target_core_pr.h | 1 + drivers/target/target_core_tmr.c | 4 +- drivers/target/target_core_transport.c | 2 +- include/target/target_core_base.h | 1 + 9 files changed, 104 insertions(+), 15 deletions(-) diff --git a/drivers/target/dlm_ckv.c b/drivers/target/dlm_ckv.c index 417159f18fc6..f4e7bc410a08 100644 --- a/drivers/target/dlm_ckv.c +++ b/drivers/target/dlm_ckv.c @@ -49,6 +49,7 @@ struct dlm_ckv_bucket { size_t num_nodes; int nodeid[64]; void *userarg; + dlm_ckv_nodeleft_cb nodeleft_cb; struct completion sync_compl; struct workqueue_struct *notify_wq; }; @@ -79,7 +80,12 @@ static void dlm_ckv_recover_prep(void *arg) static void dlm_ckv_recover_slot(void *arg, struct dlm_slot *slot) { - pr_info("nodeid %d left the cluster\n", slot->nodeid); + struct dlm_ckv_bucket *bucket = arg; + + pr_debug("nodeid %d left the cluster\n", slot->nodeid); + + if (bucket->nodeleft_cb) + bucket->nodeleft_cb(bucket->userarg, slot->nodeid); } /* dlm calls after recover_slot and after it completes lock recovery */ @@ -667,7 +673,8 @@ static void bucket_release(struct kref *ref) } struct dlm_ckv_bucket * -dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) +dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg, + dlm_ckv_nodeleft_cb nodeleft_cb, int *local_nodeid) { struct dlm_ckv_bucket *bucket; int name_len = strlen(name); @@ -677,6 +684,9 @@ dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) if (!name) return ERR_PTR(-EINVAL); + if (!local_nodeid) + return ERR_PTR(-EINVAL); + if (name_len > DLM_LOCKSPACE_LEN) return ERR_PTR(-EINVAL); @@ -684,6 +694,7 @@ dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) kref_init(&bucket->refcount); bucket->userarg = userarg; + bucket->nodeleft_cb = nodeleft_cb; init_completion(&bucket->sync_compl); bucket->notify_wq = alloc_ordered_workqueue("notify_wq-%s", 0, name); @@ -708,6 +719,7 @@ dlm_ckv_open_bucket(const char *name, const char *cluster_name, void *userarg) goto fail_init; } + *local_nodeid = bucket->local_nodeid; return bucket; fail_init: diff --git a/drivers/target/dlm_ckv.h b/drivers/target/dlm_ckv.h index e8045917067e..c5a69486a3bf 100644 --- a/drivers/target/dlm_ckv.h +++ b/drivers/target/dlm_ckv.h @@ -14,7 +14,9 @@ typedef void (*dlm_ckv_nodeleft_cb)(void *arg, int nodeid); struct dlm_ckv_bucket *dlm_ckv_open_bucket(const char *name, const char *cluster_name, - void *userarg); + void *userarg, + dlm_ckv_nodeleft_cb nodeleft_cb, + int *local_nodeid); int dlm_ckv_close_bucket(struct dlm_ckv_bucket *bucket); struct dlm_ckv_lock * diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 16391a8661fa..4c4d337e6a1a 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -98,6 +98,7 @@ struct async_group { }; static void target_pr_sync_cb(void *arg); +static void target_nodeleft_cb(void *arg, int nodeid); static int pr_reg_realloc(struct target_cluster_data *cluster_data, size_t nr_registrants); @@ -128,7 +129,8 @@ static int target_init_dlm(struct se_device *dev) cluster_data->bucket = dlm_ckv_open_bucket(ls_name, cluster_cfg.cluster_name, - dev); + dev, target_nodeleft_cb, + &cluster_data->local_nodeid); if (!cluster_data->bucket) { err = -EIO; kfree(cluster_data); @@ -679,6 +681,27 @@ static void target_pr_sync_cb(void *arg) atomic_set(&dev->t10_pr.pr_generation, pr_data.pr_generation); dev->t10_pr.pr_aptpl_active = pr_data.pr_aptpl; + /* update SPC-2 reservation */ + cluster_data->reserved_node_id = pr_data.reserved_by_nodeid; + spin_lock(&dev->dev_reservation_lock); + if (cluster_data->reserved_node_id == cluster_data->local_nodeid && + dev->reservation_holder == NULL) { + pr_err("TARGET_CORE[%d]:SCP-2 reservation is out of sync. Cluster thinks that we are reservation holder\n", + dev->dev_index); + } + if (cluster_data->reserved_node_id != cluster_data->local_nodeid && + dev->reservation_holder != NULL) { + pr_err("TARGET_CORE[%d]: SPC-2 reservation is out of sync. Possible split-brain. Cluster thinks that reservation holder is other node\n", + dev->dev_index); + dev->reservation_holder = NULL; + } + if (cluster_data->reserved_node_id != 0) + dev->dev_reservation_flags |= DRF_SPC2_RESERVATIONS; + else + target_release_reservation(dev); + + spin_unlock(&dev->dev_reservation_lock); + /* update reservation */ spin_lock(&dev->dev_reservation_lock); if (pr_prev_res_holder && @@ -703,6 +726,39 @@ static void target_pr_sync_cb(void *arg) kfree(pr_reg_data); } +static void +target_spc2_reserve(struct se_device *dev, struct se_session *sess) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + + target_pr_lock_dlm(dev); + + if (sess) { + target_get_reservation(dev, sess); + cluster_data->reserved_node_id = cluster_data->local_nodeid; + } else { + target_release_reservation(dev); + cluster_data->reserved_node_id = 0; + } + + target_pr_sync_dlm(dev); + target_pr_unlock_dlm(dev); +} + +static void target_nodeleft_cb(void *arg, int nodeid) +{ + struct se_device *dev = arg; + struct target_cluster_data *cluster_data = dev->cluster_data; + + if (cluster_data->reserved_node_id == nodeid && + dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { + pr_info("TARGET_CORE[%d]: SPC-2 reservation holder left the cluster\n", + dev->dev_index); + dev->reservation_holder = NULL; + cluster_data->reserved_node_id = 0; + } +} + const struct target_cluster_ops dlm_cluster_ops = { .name = "dlm", .owner = THIS_MODULE, @@ -715,6 +771,7 @@ const struct target_cluster_ops dlm_cluster_ops = { .pr_lock = target_pr_lock_dlm, .pr_unlock = target_pr_unlock_dlm, .pr_sync = target_pr_sync_dlm, + .reserve = target_spc2_reserve, }; static int __init target_cluster_dlm_module_init(void) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 297bf5985f48..9f01a28ef72f 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -788,6 +788,15 @@ int target_dummy_nodlm(struct se_device *dev) return 0; } +static void target_reserve2_nodlm(struct se_device *dev, struct se_session *sess) +{ + if (sess) { + target_get_reservation(dev, sess); + } else { + target_release_reservation(dev); + } +} + const struct target_cluster_ops nodlm_cluster_ops = { .name = "single", .owner = NULL, @@ -798,6 +807,7 @@ const struct target_cluster_ops nodlm_cluster_ops = { .pr_lock = target_dummy_nodlm, .pr_unlock = target_dummy_nodlm, .pr_sync = target_dummy_nodlm, + .reserve = target_reserve2_nodlm, }; struct se_device *target_alloc_device(struct se_hba *hba, const char *name) diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 22f2aa91cf2e..930f0bebd75d 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -192,6 +192,7 @@ void target_release_reservation(struct se_device *dev) dev->dev_reservation_flags &= ~DRF_SPC2_RESERVATIONS_WITH_ISID; } } +EXPORT_SYMBOL(target_release_reservation); sense_reason_t target_scsi2_reservation_release(struct se_cmd *cmd) @@ -222,7 +223,7 @@ target_scsi2_reservation_release(struct se_cmd *cmd) if (dev->dev_res_bin_isid != sess->sess_bin_isid) goto out_unlock; - target_release_reservation(dev); + dev->cl_ops->reserve(dev, NULL); tpg = sess->se_tpg; pr_debug("SCSI-2 Released reservation for %s LUN: %llu ->" " MAPPED LUN: %llu for %s\n", @@ -237,6 +238,17 @@ target_scsi2_reservation_release(struct se_cmd *cmd) return 0; } +void target_get_reservation(struct se_device *dev, struct se_session *sess) +{ + dev->reservation_holder = sess; + dev->dev_reservation_flags |= DRF_SPC2_RESERVATIONS; + if (sess->sess_bin_isid != 0) { + dev->dev_res_bin_isid = sess->sess_bin_isid; + dev->dev_reservation_flags |= DRF_SPC2_RESERVATIONS_WITH_ISID; + } +} +EXPORT_SYMBOL(target_get_reservation); + sense_reason_t target_scsi2_reservation_reserve(struct se_cmd *cmd) { @@ -284,12 +296,8 @@ target_scsi2_reservation_reserve(struct se_cmd *cmd) goto out_unlock; } - dev->reservation_holder = sess; - dev->dev_reservation_flags |= DRF_SPC2_RESERVATIONS; - if (sess->sess_bin_isid != 0) { - dev->dev_res_bin_isid = sess->sess_bin_isid; - dev->dev_reservation_flags |= DRF_SPC2_RESERVATIONS_WITH_ISID; - } + dev->cl_ops->reserve(dev, sess); + pr_debug("SCSI-2 Reserved %s LUN: %llu -> MAPPED LUN: %llu" " for %s\n", tpg->se_tpg_tfo->fabric_name, cmd->se_lun->unpacked_lun, cmd->orig_fe_lun, diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index e7eff50a01f2..a3a6013dc530 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -59,6 +59,7 @@ extern struct kmem_cache *t10_pr_reg_cache; extern void core_pr_dump_initiator_port(struct t10_pr_registration *, char *, u32); extern void target_release_reservation(struct se_device *dev); +extern void target_get_reservation(struct se_device *dev, struct se_session *sess); extern sense_reason_t target_scsi2_reservation_release(struct se_cmd *); extern sense_reason_t target_scsi2_reservation_reserve(struct se_cmd *); extern int core_scsi3_alloc_aptpl_registration( diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 2db45efedd97..26df0f685499 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -406,9 +406,7 @@ int core_tmr_lun_reset( * LOGICAL UNIT RESET */ if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { - spin_lock(&dev->dev_reservation_lock); - target_release_reservation(dev); - spin_unlock(&dev->dev_reservation_lock); + dev->cl_ops->reserve(dev, NULL); pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); } diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index d3cbba294510..47b44a1e6925 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -612,7 +612,7 @@ static int target_release_res(struct se_device *dev, void *data) struct se_session *sess = data; if (dev->reservation_holder == sess) - target_release_reservation(dev); + dev->cl_ops->reserve(dev, NULL); return 0; } diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index b0f0a6b93f6e..c49ddf0828f6 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -790,6 +790,7 @@ struct target_cluster_ops { int (*pr_lock)(struct se_device *dev); int (*pr_unlock)(struct se_device *dev); int (*pr_sync)(struct se_device *dev); + void (*reserve)(struct se_device *dev, struct se_session *sess); }; struct se_device { From patchwork Mon Feb 28 10:37:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A52F9C19F29 for ; Wed, 3 Aug 2022 16:31:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230034AbiHCQbv (ORCPT ); Wed, 3 Aug 2022 12:31:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238596AbiHCQa6 (ORCPT ); Wed, 3 Aug 2022 12:30:58 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48DA04F6A7; Wed, 3 Aug 2022 09:30:09 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id B40E74137B; Wed, 3 Aug 2022 16:30:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544206; x=1661358607; bh=IxlaNYqM1m4BES/XxCSOI3nqy Jy6bGDMyNjTNNedogQ=; b=Te+cy6/K+xpI46IdSTE9m+NapWoWuMWVonQ3LW6DI GX1A2R+aW6Byurm/VDe3Hr14flM8yNeTooa3qms6Yo1qS2ZZ9lO9x4sDlNDD8qCd 7eTztDa6ZZSnTniqt6XshTXENy3ZjsAPSzn/p9FTE2b5x60Jw/6NhMrrmEHzUiNv rA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u4JiokXhGnpG; Wed, 3 Aug 2022 19:30:06 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1C6AD412D2; Wed, 3 Aug 2022 19:29:25 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:25 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:24 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 43/48] target: cluster: allocate UAs on PR sync Date: Mon, 28 Feb 2022 13:37:58 +0300 Message-ID: <20220803162857.27770-44-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Establish Unit Attention condition on the peer nodes in the cluster on PR OUT commands according to SPC-4. Pass PR OUT Service Action to pr_sync to allow peer nodes to know the reason of the update of PR state. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 115 +++++++++++++++++++++++++++- drivers/target/target_core_device.c | 7 +- drivers/target/target_core_pr.c | 2 +- drivers/target/target_core_ua.c | 1 + include/target/target_core_base.h | 2 +- 5 files changed, 121 insertions(+), 6 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 4c4d337e6a1a..cb44f6c0f0ac 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -7,6 +7,7 @@ #include #include #include +#include "target_core_ua.h" #include "target_core_pr.h" #include "dlm_ckv.h" @@ -78,7 +79,8 @@ struct pr_lvb { u8 pr_type; u8 pr_scope; u8 pr_aptpl; - u8 reserved[3]; + u8 pro_sa; + u8 reserved[2]; u32 reserved_by_nodeid; }; @@ -314,7 +316,7 @@ static int pr_reg_realloc(struct target_cluster_data *cluster_data, return res; } -static int target_pr_sync_dlm(struct se_device *dev) +static int target_pr_sync_dlm(struct se_device *dev, u8 pro_sa) { struct target_cluster_data *cluster_data = dev->cluster_data; struct t10_pr_registration *pr_reg; @@ -328,6 +330,7 @@ static int target_pr_sync_dlm(struct se_device *dev) pr_data.version = 1; pr_data.pr_generation = atomic_read(&dev->t10_pr.pr_generation); + pr_data.pro_sa = pro_sa; pr_data.nr_registrants = target_get_nr_registrants(dev); pr_data.pr_is_set = !!dev->dev_pr_res_holder; pr_data.pr_aptpl = dev->t10_pr.pr_aptpl_active; @@ -546,6 +549,19 @@ target_create_pr_reg(struct se_device *dev, return NULL; } +static void target_allocate_pr_ua(struct se_device *dev, u8 asc) +{ + struct t10_pr_registration *pr_reg; + + list_for_each_entry(pr_reg, &dev->t10_pr.registration_list, pr_reg_list) { + target_ua_allocate_lun( + pr_reg->pr_reg_nacl, + pr_reg->pr_res_mapped_lun, + 0x2A, + asc); + } +} + static void target_pr_sync_cb(void *arg) { struct se_device *dev = arg; @@ -558,6 +574,9 @@ static void target_pr_sync_cb(void *arg) struct async_group grp; struct pr_lvb pr_data; bool res_to_delete = false; + struct se_node_acl *pr_reg_nacl; + u64 pr_res_mapped_lun; + bool reg_deleted = false; bool was_held; u8 was_type; u8 was_scope; @@ -656,10 +675,41 @@ static void target_pr_sync_cb(void *arg) /* deregister obsolete entries */ list_for_each_entry_safe(pr_reg, pr_reg_tmp, &to_be_deleted_list, pr_reg_list) { + reg_deleted = true; + pr_reg_nacl = pr_reg->pr_reg_nacl; + pr_res_mapped_lun = pr_reg->pr_res_mapped_lun; + if (dev->dev_pr_res_holder != pr_reg) __core_scsi3_free_registration(dev, pr_reg, NULL, 0); else res_to_delete = true; + + switch (pr_data.pro_sa) { + case PRO_CLEAR: + /* + * establish a unit attention condition for the initiator + * port associated with every registered I_T nexus other + * than the I_T nexus on which the PERSISTENT RESERVE OUT + * command with CLEAR service action was received, with + * the additional sense code set to RESERVATIONS PREEMPTED + */ + target_ua_allocate_lun(pr_reg_nacl, + pr_res_mapped_lun, 0x2A, + ASCQ_2AH_RESERVATIONS_PREEMPTED); + break; + case PRO_PREEMPT_AND_ABORT: + case PRO_REGISTER_AND_IGNORE_EXISTING_KEY: + /* + * establish a unit attention condition for the initiator + * port associated with every I_T nexus that lost its + * persistent reservation and/or registration, with the + * additional sense code set to REGISTRATIONS PREEMPTED; + */ + target_ua_allocate_lun(pr_reg_nacl, + pr_res_mapped_lun, 0x2A, + ASCQ_2AH_REGISTRATIONS_PREEMPTED); + break; + } } spin_unlock(&dev->t10_pr.registration_lock); @@ -677,6 +727,65 @@ static void target_pr_sync_cb(void *arg) pr_reg_res_holder = pr_reg; } + switch (pr_data.pro_sa) { + case PRO_REGISTER_AND_IGNORE_EXISTING_KEY: + case PRO_REGISTER: + /* + * If the I_T nexus is the reservation holder and the persistent + * reservation is of a type other than all registrants, then + * the device server shall also release the persistent reservation + * + * If the persistent reservation is a registrants only + * type, the device server shall establish a unit + * attention condition for the initiator port associated + * with every registered I_T nexus except for the I_T + * nexus on which the PERSISTENT RESERVE OUT command was + * received, with the additional sense code set to + * RESERVATIONS RELEASED. + */ + if (reg_deleted && was_held && + (was_type == PR_TYPE_WRITE_EXCLUSIVE_REGONLY || + was_type == PR_TYPE_EXCLUSIVE_ACCESS_REGONLY)) { + target_allocate_pr_ua(dev, + ASCQ_2AH_RESERVATIONS_RELEASED); + } + break; + case PRO_RELEASE: + /* + * if the released persistent reservation is either a registrants + * only type or an all registrants type persistent reservation, + * then the device server shall establish a unit attention + * condition for the initiator port associated with every + * registered I_T nexus other than I_T nexus on which the + * PERSISTENT RESERVE OUT command with RELEASE service action was + * received, with the additional sense code set to RESERVATIONS RELEASED + */ + if ((was_type == PR_TYPE_WRITE_EXCLUSIVE_REGONLY) || + (was_type == PR_TYPE_EXCLUSIVE_ACCESS_REGONLY) || + (was_type == PR_TYPE_WRITE_EXCLUSIVE_ALLREG) || + (was_type == PR_TYPE_EXCLUSIVE_ACCESS_ALLREG)) { + target_allocate_pr_ua(dev, + ASCQ_2AH_RESERVATIONS_RELEASED); + } + break; + case PRO_PREEMPT: + case PRO_PREEMPT_AND_ABORT: + /* + * if the type or scope has changed, then for every I_T nexus + * whose reservation key was not removed, except for the I_T nexus + * on which the PERSISTENT RESERVE OUT command was received, the + * device server shall establish a unit attention condition for + * the initiator port associated with that I_T nexus, with the + * additional sense code set to RESERVATIONS RELEASED. I + */ + if (pr_prev_res_holder && pr_reg_res_holder && + pr_prev_res_holder != pr_reg_res_holder && + (was_type != pr_reg_res_holder->pr_res_type || + was_scope != pr_reg_res_holder->pr_res_scope)) + target_allocate_pr_ua(dev, ASCQ_2AH_RESERVATIONS_RELEASED); + break; + } + /* update general data */ atomic_set(&dev->t10_pr.pr_generation, pr_data.pr_generation); dev->t10_pr.pr_aptpl_active = pr_data.pr_aptpl; @@ -741,7 +850,7 @@ target_spc2_reserve(struct se_device *dev, struct se_session *sess) cluster_data->reserved_node_id = 0; } - target_pr_sync_dlm(dev); + target_pr_sync_dlm(dev, -1); target_pr_unlock_dlm(dev); } diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 9f01a28ef72f..4f36d12d2213 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -788,6 +788,11 @@ int target_dummy_nodlm(struct se_device *dev) return 0; } +int target_prsync_nodlm(struct se_device *dev, u8 pro_sa) +{ + return 0; +} + static void target_reserve2_nodlm(struct se_device *dev, struct se_session *sess) { if (sess) { @@ -806,7 +811,7 @@ const struct target_cluster_ops nodlm_cluster_ops = { .caw_unlock = target_caw_unlock_nodlm, .pr_lock = target_dummy_nodlm, .pr_unlock = target_dummy_nodlm, - .pr_sync = target_dummy_nodlm, + .pr_sync = target_prsync_nodlm, .reserve = target_reserve2_nodlm, }; diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 930f0bebd75d..fd89647c5ac7 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -3624,7 +3624,7 @@ target_scsi3_emulate_pr_out(struct se_cmd *cmd) } if (!ret) - dev->cl_ops->pr_sync(dev); + dev->cl_ops->pr_sync(dev, sa); dev->cl_ops->pr_unlock(dev); if (!ret) diff --git a/drivers/target/target_core_ua.c b/drivers/target/target_core_ua.c index 4276690fb6cb..68a7339611f3 100644 --- a/drivers/target/target_core_ua.c +++ b/drivers/target/target_core_ua.c @@ -171,6 +171,7 @@ void target_ua_allocate_lun(struct se_node_acl *nacl, core_scsi3_ua_allocate(deve, asc, ascq); rcu_read_unlock(); } +EXPORT_SYMBOL(target_ua_allocate_lun); void core_scsi3_ua_release_all( struct se_dev_entry *deve) diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index c49ddf0828f6..6c99fb4aa151 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -789,7 +789,7 @@ struct target_cluster_ops { int (*caw_unlock)(struct se_device *dev, void *lock); int (*pr_lock)(struct se_device *dev); int (*pr_unlock)(struct se_device *dev); - int (*pr_sync)(struct se_device *dev); + int (*pr_sync)(struct se_device *dev, u8 pro_sa); void (*reserve)(struct se_device *dev, struct se_session *sess); }; From patchwork Fri Mar 4 12:35:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43E5FC3F6B0 for ; Wed, 3 Aug 2022 16:32:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238363AbiHCQcM (ORCPT ); Wed, 3 Aug 2022 12:32:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236806AbiHCQbd (ORCPT ); Wed, 3 Aug 2022 12:31:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 088922AEB; Wed, 3 Aug 2022 09:30:13 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 841414137C; Wed, 3 Aug 2022 16:30:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544211; x=1661358612; bh=dDqzX1mALs+TsBue/pJSZwyxv N2QpHIjrzwOQ3a8ERA=; b=JsN6AIVhk40wrlTO8qrPb2xByyKjPlqpHJox2iAyQ W++Ho4co2+rmaBJlgpQHqCS4aisvjJO+kWfYREoRpANI9HzwoMwOpZsHKe5U5Era nn9ML33RylqdROrU8PdhNtwUy3fvw2Ch50NGtqXiv4UQKEEffr/Z5gftRevLnF+G Ns= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g1-BKzSFUBFd; Wed, 3 Aug 2022 19:30:11 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1CDF0412FE; Wed, 3 Aug 2022 19:29:26 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:26 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:24 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 44/48] target: cluster: support PR OUT preempt and abort Date: Fri, 4 Mar 2022 15:35:36 +0300 Message-ID: <20220803162857.27770-45-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Abort commands in the preempted I_T nexuses on all nodes in cluster by PR OUT (PREEMPT AND ABORT) command. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 17 +++++++++++++++-- drivers/target/target_core_internal.h | 2 -- drivers/target/target_core_pr.c | 3 ++- drivers/target/target_core_pr.h | 3 +++ drivers/target/target_core_tmr.c | 1 + include/target/target_core_base.h | 3 +++ 6 files changed, 24 insertions(+), 5 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index cb44f6c0f0ac..91f8ffc7ad2f 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -571,6 +571,7 @@ static void target_pr_sync_cb(void *arg) struct t10_pr_registration *pr_prev_res_holder = NULL; struct pr_reg_lvb *pr_reg_data = NULL; LIST_HEAD(to_be_deleted_list); + LIST_HEAD(preempt_and_abort_list); struct async_group grp; struct pr_lvb pr_data; bool res_to_delete = false; @@ -680,7 +681,9 @@ static void target_pr_sync_cb(void *arg) pr_res_mapped_lun = pr_reg->pr_res_mapped_lun; if (dev->dev_pr_res_holder != pr_reg) - __core_scsi3_free_registration(dev, pr_reg, NULL, 0); + __core_scsi3_free_registration(dev, pr_reg, + (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) ? + &preempt_and_abort_list : NULL, 0); else res_to_delete = true; @@ -825,10 +828,20 @@ static void target_pr_sync_cb(void *arg) if (res_to_delete) { spin_lock(&dev->t10_pr.registration_lock); - __core_scsi3_free_registration(dev, pr_prev_res_holder, NULL, 0); + __core_scsi3_free_registration(dev, pr_prev_res_holder, + (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) ? + &preempt_and_abort_list : NULL, 0); spin_unlock(&dev->t10_pr.registration_lock); } + if (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) { + core_local_lun_reset(dev, NULL, &preempt_and_abort_list, NULL); + + core_scsi3_release_preempt_and_abort( + &preempt_and_abort_list, + pr_reg_res_holder); + } + core_scsi3_update_and_write_aptpl(dev, dev->t10_pr.pr_aptpl_active); done: diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 89f0ddca35ed..fc7c2a9d9507 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -119,8 +119,6 @@ int core_delete_hba(struct se_hba *); void core_tmr_abort_task(struct se_device *, struct se_tmr_req *, struct se_session *); int core_tmr_lun_reset(struct se_device *dev, struct se_tmr_req *tmr); -void core_local_lun_reset(struct se_device *swc, struct se_tmr_req *tmr, - struct list_head *preempt_and_abort_list, struct se_cmd *prout_cmd); /* target_core_tpg.c */ extern struct spinlock g_tpg_lock; diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index fd89647c5ac7..a476dc3aa03f 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -2725,7 +2725,7 @@ static void __core_scsi3_complete_pro_preempt( preempt_and_abort_list); } -static void core_scsi3_release_preempt_and_abort( +void core_scsi3_release_preempt_and_abort( struct list_head *preempt_and_abort_list, struct t10_pr_registration *pr_reg_holder) { @@ -2747,6 +2747,7 @@ static void core_scsi3_release_preempt_and_abort( kmem_cache_free(t10_pr_reg_cache, pr_reg); } } +EXPORT_SYMBOL(core_scsi3_release_preempt_and_abort); static sense_reason_t core_scsi3_pro_preempt(struct se_cmd *cmd, int type, int scope, u64 res_key, diff --git a/drivers/target/target_core_pr.h b/drivers/target/target_core_pr.h index a3a6013dc530..82b4a270f711 100644 --- a/drivers/target/target_core_pr.h +++ b/drivers/target/target_core_pr.h @@ -99,6 +99,9 @@ void __core_scsi3_free_registration(struct se_device *dev, struct t10_pr_registration *pr_reg, struct list_head *preempt_and_abort_list, int dec_holders); +void core_scsi3_release_preempt_and_abort( + struct list_head *preempt_and_abort_list, + struct t10_pr_registration *pr_reg_holder); #endif /* TARGET_CORE_PR_H */ diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 26df0f685499..6984bfa976a4 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -394,6 +394,7 @@ void core_local_lun_reset( (preempt_and_abort_list) ? "Preempt" : "TMR", dev->transport->name); } +EXPORT_SYMBOL(core_local_lun_reset); int core_tmr_lun_reset( struct se_device *dev, diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 6c99fb4aa151..af3c972c3376 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -1010,5 +1010,8 @@ u32 target_parse_pr_out_transport_id( int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, unsigned char *tid); void target_pr_kref_release(struct kref *kref); +void core_local_lun_reset(struct se_device *swc, struct se_tmr_req *tmr, + struct list_head *preempt_and_abort_list, struct se_cmd *prout_cmd); + #endif /* TARGET_CORE_BASE_H */ From patchwork Fri Mar 11 08:40:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E651C19F2D for ; Wed, 3 Aug 2022 16:32:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238213AbiHCQcN (ORCPT ); Wed, 3 Aug 2022 12:32:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237606AbiHCQbd (ORCPT ); Wed, 3 Aug 2022 12:31:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E6416342; Wed, 3 Aug 2022 09:30:15 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id AE436412FE; Wed, 3 Aug 2022 16:30:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544212; x=1661358613; bh=EyVD6bNDYi6Hci8/RaMsy90vn T1nj8IM+Qzvs9n7IKU=; b=mdSuoobvPkqdGfO04V80E1BlFFQ6ouwb5erMdDKEw LYDMXPsLI3EExm7o74Vr2WB9QtFgV6sjY4tXdLl6X0TaaDis7gx5dLMJhYkbVoeC gXwrEmyT8P9bxaMx7IdJKVkvPsuPGa0/0lZNOvgslyvKQ8AepbreCVMBEboKA+0N CY= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A8pKwWcY2DeP; Wed, 3 Aug 2022 19:30:12 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 799334127C; Wed, 3 Aug 2022 19:29:26 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:26 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:25 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 45/48] target: cluster: add reset cluster function Date: Fri, 11 Mar 2022 11:40:31 +0300 Message-ID: <20220803162857.27770-46-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Introduce `reset' function pointer in target cluster ops. For no cluster case do just clear SCSI2 reservation status. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_core_device.c | 9 +++++++++ drivers/target/target_core_tmr.c | 9 +-------- include/target/target_core_base.h | 1 + 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 4f36d12d2213..7d0d889961c2 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -802,6 +802,14 @@ static void target_reserve2_nodlm(struct se_device *dev, struct se_session *sess } } +static void target_reset_nodlm(struct se_device *dev) +{ + if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { + target_release_reservation(dev); + pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); + } +} + const struct target_cluster_ops nodlm_cluster_ops = { .name = "single", .owner = NULL, @@ -813,6 +821,7 @@ const struct target_cluster_ops nodlm_cluster_ops = { .pr_unlock = target_dummy_nodlm, .pr_sync = target_prsync_nodlm, .reserve = target_reserve2_nodlm, + .reset = target_reset_nodlm, }; struct se_device *target_alloc_device(struct se_hba *hba, const char *name) diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 6984bfa976a4..878401ab8145 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -402,14 +402,7 @@ int core_tmr_lun_reset( { core_local_lun_reset(dev, tmr, NULL, NULL); - /* - * Clear any legacy SPC-2 reservation when called during - * LOGICAL UNIT RESET - */ - if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { - dev->cl_ops->reserve(dev, NULL); - pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); - } + dev->cl_ops->reset(dev); target_dev_ua_allocate(dev, 0x29, ASCQ_29H_BUS_DEVICE_RESET_FUNCTION_OCCURRED); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index af3c972c3376..c8900da705b6 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -791,6 +791,7 @@ struct target_cluster_ops { int (*pr_unlock)(struct se_device *dev); int (*pr_sync)(struct se_device *dev, u8 pro_sa); void (*reserve)(struct se_device *dev, struct se_session *sess); + void (*reset)(struct se_device *dev); }; struct se_device { From patchwork Fri Mar 11 09:30:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1EAEC25B07 for ; Wed, 3 Aug 2022 16:32:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238391AbiHCQcO (ORCPT ); Wed, 3 Aug 2022 12:32:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237776AbiHCQbd (ORCPT ); Wed, 3 Aug 2022 12:31:33 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E82D6348; Wed, 3 Aug 2022 09:30:15 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id C9E704137B; Wed, 3 Aug 2022 16:30:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544212; x=1661358613; bh=Wot/84Y8dChAbb4bp4eHumjPs XKgO/6Gr7sNnnEq4MA=; b=U437+bpn7GZM34qWNg8dPwEXrGs5VJTTVxCzxVRQn zgl1p3CE/rGMCe12mag5Y6MB2GFehZcqf4IPyMOQn4+THfRX0Q1kV9buIILhX51G qvp/csQPAnFLu8rS7k6ZWPXjAsmpG3Gsjwmt52R6WMID2TQ+9M+S9DV8fidkfrXP gk= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BQh7v7g582Ze; Wed, 3 Aug 2022 19:30:12 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id C7BF9412C5; Wed, 3 Aug 2022 19:29:26 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:26 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:26 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 46/48] target: cluster: implement LUN reset in DLM cluster Date: Fri, 11 Mar 2022 12:30:19 +0300 Message-ID: <20220803162857.27770-47-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Since LUN RESET clears SCSI2 Reservation (cluster wide data of persistence reservation status) make LUN RESET just one of reasons to update the Persistence Reservation status as well as Service Actions of PR OUT. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 31 ++++++++++++++++++++++++++- drivers/target/target_core_device.c | 1 + drivers/target/target_core_internal.h | 1 - include/target/target_core_base.h | 2 +- 4 files changed, 32 insertions(+), 3 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 91f8ffc7ad2f..546e28062b8f 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -99,6 +99,9 @@ struct async_group { struct completion compl; }; +#define PR_SYNC_REASON_RESERVE 10 +#define PR_SYNC_REASON_RESET 11 + static void target_pr_sync_cb(void *arg); static void target_nodeleft_cb(void *arg, int nodeid); static int pr_reg_realloc(struct target_cluster_data *cluster_data, @@ -842,6 +845,15 @@ static void target_pr_sync_cb(void *arg) pr_reg_res_holder); } + if (pr_data.pro_sa == PR_SYNC_REASON_RESET) { + core_local_lun_reset(dev, NULL, NULL, NULL); + + target_dev_ua_allocate(dev, 0x29, + ASCQ_29H_BUS_DEVICE_RESET_FUNCTION_OCCURRED); + + atomic_long_inc(&dev->num_resets); + } + core_scsi3_update_and_write_aptpl(dev, dev->t10_pr.pr_aptpl_active); done: @@ -863,7 +875,23 @@ target_spc2_reserve(struct se_device *dev, struct se_session *sess) cluster_data->reserved_node_id = 0; } - target_pr_sync_dlm(dev, -1); + target_pr_sync_dlm(dev, PR_SYNC_REASON_RESERVE); + target_pr_unlock_dlm(dev); +} + +static void +target_dlm_lun_reset(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + + target_pr_lock_dlm(dev); + if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS) { + target_release_reservation(dev); + cluster_data->reserved_node_id = 0; + pr_debug("LUN_RESET: SCSI-2 Released reservation\n"); + } + + target_pr_sync_dlm(dev, PR_SYNC_REASON_RESET); target_pr_unlock_dlm(dev); } @@ -894,6 +922,7 @@ const struct target_cluster_ops dlm_cluster_ops = { .pr_unlock = target_pr_unlock_dlm, .pr_sync = target_pr_sync_dlm, .reserve = target_spc2_reserve, + .reset = target_dlm_lun_reset, }; static int __init target_cluster_dlm_module_init(void) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 7d0d889961c2..6fe9f40b86f5 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -375,6 +375,7 @@ void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq) } spin_unlock(&dev->se_port_lock); } +EXPORT_SYMBOL(target_dev_ua_allocate); static void target_luns_data_has_changed(struct se_node_acl *nacl, struct se_dev_entry *new, diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index fc7c2a9d9507..7c495dddcb4b 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -93,7 +93,6 @@ int target_configure_device(struct se_device *dev); void target_free_device(struct se_device *); int target_for_each_device(int (*fn)(struct se_device *dev, void *data), void *data); -void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq); /* target_core_configfs.c */ extern struct configfs_item_operations target_core_dev_item_ops; diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index c8900da705b6..4dad7837f603 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -1013,6 +1013,6 @@ int target_cmp_pr_transport_id(struct t10_pr_registration *pr_reg, void target_pr_kref_release(struct kref *kref); void core_local_lun_reset(struct se_device *swc, struct se_tmr_req *tmr, struct list_head *preempt_and_abort_list, struct se_cmd *prout_cmd); - +void target_dev_ua_allocate(struct se_device *dev, u8 asc, u8 ascq); #endif /* TARGET_CORE_BASE_H */ From patchwork Fri Mar 25 06:35:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78789C3F6B0 for ; Wed, 3 Aug 2022 16:32:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238410AbiHCQcX (ORCPT ); Wed, 3 Aug 2022 12:32:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238236AbiHCQbf (ORCPT ); Wed, 3 Aug 2022 12:31:35 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFB1A95B9; Wed, 3 Aug 2022 09:30:16 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 71FC941306; Wed, 3 Aug 2022 16:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544213; x=1661358614; bh=TvgIU7pD4ryCcfsdLEnvvPmpr 25Q4u6JZGeMIGkWvsg=; b=YevSKbvdHKx65w8h1hpsbHkC59FNHheo+dUYUT8Zf yVOw+l5GgjvO8otm0MxwItcM8TnTYYgZY3NBpQ4Tb9LrCG43FbQME4AtJ4BtV3S3 Qt3SeTiA3hJuteVKTZtIoEVxX+qCQHsUXh0EuIIUoQ+/YIIKgRbTbyaVjpOUaz5x Yg= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oIdCX-8JyHok; Wed, 3 Aug 2022 19:30:13 +0300 (MSK) Received: from T-EXCH-01.corp.yadro.com (t-exch-01.corp.yadro.com [172.17.10.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1D3504128A; Wed, 3 Aug 2022 19:29:27 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-01.corp.yadro.com (172.17.10.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:27 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:26 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 47/48] target: cluster: split cluster sync function Date: Fri, 25 Mar 2022 09:35:45 +0300 Message-ID: <20220803162857.27770-48-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Split cluster sync function to two parts: 1. Read cluster data. 2. Apply cluster data. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 117 ++++++++++++++++++---------- 1 file changed, 74 insertions(+), 43 deletions(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index 546e28062b8f..ff0c207be141 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -565,61 +565,53 @@ static void target_allocate_pr_ua(struct se_device *dev, u8 asc) } } -static void target_pr_sync_cb(void *arg) +static int target_dlm_read_cluster_data( + struct se_device *dev, + struct pr_lvb *pr_data, + struct pr_reg_lvb **pr_reg_datap) { - struct se_device *dev = arg; struct target_cluster_data *cluster_data = dev->cluster_data; - struct t10_pr_registration *pr_reg, *pr_reg_tmp; - struct t10_pr_registration *pr_reg_res_holder = NULL; - struct t10_pr_registration *pr_prev_res_holder = NULL; struct pr_reg_lvb *pr_reg_data = NULL; - LIST_HEAD(to_be_deleted_list); - LIST_HEAD(preempt_and_abort_list); struct async_group grp; - struct pr_lvb pr_data; - bool res_to_delete = false; - struct se_node_acl *pr_reg_nacl; - u64 pr_res_mapped_lun; - bool reg_deleted = false; - bool was_held; - u8 was_type; - u8 was_scope; - bool found; + int res = 0; int i = 0; - int res; - res = dlm_ckv_get(cluster_data->pr_data, (char *)&pr_data, sizeof(pr_data)); + res = dlm_ckv_get(cluster_data->pr_data, (char *)pr_data, + sizeof(struct pr_lvb)); if (res) goto done; - if (!pr_data.version) { + if (!pr_data->version) { pr_info("TARGET_CORE[%d]: PR data from cluster is invalid\n", dev->dev_index); goto done; } - pr_reg_data = kzalloc(sizeof(struct pr_reg_lvb) * pr_data.nr_registrants, + if (pr_data->nr_registrants == 0) + goto done; + + *pr_reg_datap = kzalloc(sizeof(struct pr_reg_lvb) * pr_data->nr_registrants, GFP_KERNEL); + pr_reg_data = *pr_reg_datap; if (!pr_reg_data) { res = -ENOMEM; goto done; } - res = pr_reg_realloc(cluster_data, pr_data.nr_registrants); + res = pr_reg_realloc(cluster_data, pr_data->nr_registrants); if (res) goto done; - if (pr_data.nr_registrants == 0) - goto skip_pr_reg; - refcount_set(&grp.pending, 1); /* 1 for a loop */ atomic_set(&grp.status, 0); init_completion(&grp.compl); - for (i = 0; i < pr_data.nr_registrants; ++i) { + for (i = 0; i < pr_data->nr_registrants; ++i) { refcount_inc(&grp.pending); - res = dlm_ckv_get_async(cluster_data->pr_reg_kv[i], (char *)(pr_reg_data + i), - sizeof(struct pr_reg_lvb), group_compl_cb, &grp); + res = dlm_ckv_get_async(cluster_data->pr_reg_kv[i], + (char *)(pr_reg_data + i), + sizeof(struct pr_reg_lvb), + group_compl_cb, &grp); if (res) { refcount_dec(&grp.pending); break; @@ -639,7 +631,33 @@ static void target_pr_sync_cb(void *arg) goto done; } -skip_pr_reg: +done: + if (res) + kfree(pr_reg_data); + return res; +} + +static void target_dlm_apply_cluster_data( + struct se_device *dev, + struct pr_lvb *pr_data, + struct pr_reg_lvb *pr_reg_data) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + struct t10_pr_registration *pr_reg, *pr_reg_tmp; + struct t10_pr_registration *pr_reg_res_holder = NULL; + struct t10_pr_registration *pr_prev_res_holder = NULL; + LIST_HEAD(to_be_deleted_list); + LIST_HEAD(preempt_and_abort_list); + bool res_to_delete = false; + struct se_node_acl *pr_reg_nacl; + u64 pr_res_mapped_lun; + bool reg_deleted = false; + bool was_held; + u8 was_type; + u8 was_scope; + bool found; + int i = 0; + /* * Update existing registrations */ @@ -656,7 +674,7 @@ static void target_pr_sync_cb(void *arg) pr_reg_list) { found = false; - for (i = 0; i < pr_data.nr_registrants; ++i) { + for (i = 0; i < pr_data->nr_registrants; ++i) { if (!pr_reg_data[i].version) continue; @@ -665,7 +683,7 @@ static void target_pr_sync_cb(void *arg) found = true; /* mark existing registrants */ pr_reg_data[i].version = 0; - target_update_pr_reg(dev, pr_reg, &pr_data, + target_update_pr_reg(dev, pr_reg, pr_data, &pr_reg_data[i], &pr_reg_res_holder); break; @@ -685,12 +703,12 @@ static void target_pr_sync_cb(void *arg) if (dev->dev_pr_res_holder != pr_reg) __core_scsi3_free_registration(dev, pr_reg, - (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) ? + (pr_data->pro_sa == PRO_PREEMPT_AND_ABORT) ? &preempt_and_abort_list : NULL, 0); else res_to_delete = true; - switch (pr_data.pro_sa) { + switch (pr_data->pro_sa) { case PRO_CLEAR: /* * establish a unit attention condition for the initiator @@ -720,12 +738,12 @@ static void target_pr_sync_cb(void *arg) spin_unlock(&dev->t10_pr.registration_lock); /* register new entries */ - for (i = 0; i < pr_data.nr_registrants; ++i) { + for (i = 0; i < pr_data->nr_registrants; ++i) { /* skip existing registrants */ if (!pr_reg_data[i].version) continue; - pr_reg = target_create_pr_reg(dev, &pr_data, &pr_reg_data[i]); + pr_reg = target_create_pr_reg(dev, pr_data, &pr_reg_data[i]); if (!pr_reg) pr_err("TARGET_CORE[%d]: can not create new registration\n", dev->dev_index); @@ -733,7 +751,7 @@ static void target_pr_sync_cb(void *arg) pr_reg_res_holder = pr_reg; } - switch (pr_data.pro_sa) { + switch (pr_data->pro_sa) { case PRO_REGISTER_AND_IGNORE_EXISTING_KEY: case PRO_REGISTER: /* @@ -793,11 +811,11 @@ static void target_pr_sync_cb(void *arg) } /* update general data */ - atomic_set(&dev->t10_pr.pr_generation, pr_data.pr_generation); - dev->t10_pr.pr_aptpl_active = pr_data.pr_aptpl; + atomic_set(&dev->t10_pr.pr_generation, pr_data->pr_generation); + dev->t10_pr.pr_aptpl_active = pr_data->pr_aptpl; /* update SPC-2 reservation */ - cluster_data->reserved_node_id = pr_data.reserved_by_nodeid; + cluster_data->reserved_node_id = pr_data->reserved_by_nodeid; spin_lock(&dev->dev_reservation_lock); if (cluster_data->reserved_node_id == cluster_data->local_nodeid && dev->reservation_holder == NULL) { @@ -826,18 +844,18 @@ static void target_pr_sync_cb(void *arg) if (pr_reg_res_holder) __core_scsi3_set_reservation(dev, pr_reg_res_holder, - pr_data.pr_scope, pr_data.pr_type); + pr_data->pr_scope, pr_data->pr_type); spin_unlock(&dev->dev_reservation_lock); if (res_to_delete) { spin_lock(&dev->t10_pr.registration_lock); __core_scsi3_free_registration(dev, pr_prev_res_holder, - (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) ? + (pr_data->pro_sa == PRO_PREEMPT_AND_ABORT) ? &preempt_and_abort_list : NULL, 0); spin_unlock(&dev->t10_pr.registration_lock); } - if (pr_data.pro_sa == PRO_PREEMPT_AND_ABORT) { + if (pr_data->pro_sa == PRO_PREEMPT_AND_ABORT) { core_local_lun_reset(dev, NULL, &preempt_and_abort_list, NULL); core_scsi3_release_preempt_and_abort( @@ -845,7 +863,7 @@ static void target_pr_sync_cb(void *arg) pr_reg_res_holder); } - if (pr_data.pro_sa == PR_SYNC_REASON_RESET) { + if (pr_data->pro_sa == PR_SYNC_REASON_RESET) { core_local_lun_reset(dev, NULL, NULL, NULL); target_dev_ua_allocate(dev, 0x29, @@ -856,10 +874,23 @@ static void target_pr_sync_cb(void *arg) core_scsi3_update_and_write_aptpl(dev, dev->t10_pr.pr_aptpl_active); -done: kfree(pr_reg_data); } +static void target_pr_sync_cb(void *arg) +{ + struct pr_reg_lvb *pr_reg_data = NULL; + struct se_device *dev = arg; + struct pr_lvb pr_data; + int res; + + res = target_dlm_read_cluster_data(dev, &pr_data, &pr_reg_data); + if (res) + return; + + target_dlm_apply_cluster_data(dev, &pr_data, pr_reg_data); +} + static void target_spc2_reserve(struct se_device *dev, struct se_session *sess) { From patchwork Fri Mar 25 07:58:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Bogdanov X-Patchwork-Id: 595235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDF94C19F29 for ; Wed, 3 Aug 2022 16:32:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238481AbiHCQcY (ORCPT ); Wed, 3 Aug 2022 12:32:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238222AbiHCQbf (ORCPT ); Wed, 3 Aug 2022 12:31:35 -0400 Received: from mta-01.yadro.com (mta-02.yadro.com [89.207.88.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94BE07663; Wed, 3 Aug 2022 09:30:16 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 3D10C4137E; Wed, 3 Aug 2022 16:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received:received; s= mta-01; t=1659544213; x=1661358614; bh=/qy+AU5xyZkgMhCzOHh14ktfG ZtI8a6kol2zDDHecjw=; b=NSlHc3aUjNGuJIbNapupwKQqgUm7U9KZvydOI3sVr I6kajW9ViQTF2hzrITGYuSCwqAEY3k5adV4/14u/aHusaZlspXq8u39nX4hxgRPU HDZ3mS5SBtC7Ba6fT7NElhKAzVEdxmX7SzMvEvO6F+caMfK1/Y+9QgLgb0uyWzeg tc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tNjZMERgxICD; Wed, 3 Aug 2022 19:30:13 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 6DE6341306; Wed, 3 Aug 2022 19:29:27 +0300 (MSK) Received: from T-EXCH-08.corp.yadro.com (172.17.11.58) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Wed, 3 Aug 2022 19:29:27 +0300 Received: from NB-591.corp.yadro.com (10.199.18.20) by T-EXCH-08.corp.yadro.com (172.17.11.58) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.9; Wed, 3 Aug 2022 19:29:26 +0300 From: Dmitry Bogdanov To: Martin Petersen , CC: , , Dmitry Bogdanov Subject: [RFC PATCH 48/48] target: cluster: request data on initial sync Date: Fri, 25 Mar 2022 10:58:34 +0300 Message-ID: <20220803162857.27770-49-d.bogdanov@yadro.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803162857.27770-1-d.bogdanov@yadro.com> References: <20220803162857.27770-1-d.bogdanov@yadro.com> MIME-Version: 1.0 X-Originating-IP: [10.199.18.20] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-08.corp.yadro.com (172.17.11.58) Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org If there is no cluster data in cluster (master node died), request it from alive nodes. Signed-off-by: Dmitry Bogdanov --- drivers/target/target_cluster_dlm.c | 36 ++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/target/target_cluster_dlm.c b/drivers/target/target_cluster_dlm.c index ff0c207be141..39754d3671b7 100644 --- a/drivers/target/target_cluster_dlm.c +++ b/drivers/target/target_cluster_dlm.c @@ -19,6 +19,7 @@ struct target_cluster_data { struct dlm_ckv_lock *pr_lock; struct dlm_ckv_kv *pr_data; struct dlm_ckv_notify *pr_sync_notify; + struct dlm_ckv_notify *init_sync_notify; int reserved_node_id; struct dlm_ckv_kv **pr_reg_kv; size_t pr_reg_kv_len; @@ -101,7 +102,10 @@ struct async_group { #define PR_SYNC_REASON_RESERVE 10 #define PR_SYNC_REASON_RESET 11 +#define PR_SYNC_REASON_INIT 13 +static void target_initial_sync(struct se_device *dev); +static void target_initial_sync_cb(void *arg); static void target_pr_sync_cb(void *arg); static void target_nodeleft_cb(void *arg, int nodeid); static int pr_reg_realloc(struct target_cluster_data *cluster_data, @@ -157,15 +161,22 @@ static int target_init_dlm(struct se_device *dev) if (!cluster_data->pr_sync_notify) goto fail; + cluster_data->init_sync_notify = dlm_ckv_create_notification( + cluster_data->bucket, "init_sync", target_initial_sync_cb); + if (!cluster_data->init_sync_notify) + goto fail; + dev->cluster_data = cluster_data; /* initial sync-up on joining the cluster */ dlm_ckv_lock_get(cluster_data->pr_lock); - target_pr_sync_cb(dev); + target_initial_sync(dev); dlm_ckv_lock_release(cluster_data->pr_lock); return err; fail: + if (cluster_data->init_sync_notify) + dlm_ckv_free_notification(cluster_data->init_sync_notify); if (cluster_data->pr_sync_notify) dlm_ckv_free_notification(cluster_data->pr_sync_notify); if (cluster_data->pr_lock) @@ -188,6 +199,7 @@ static int target_cleanup_dlm(struct se_device *dev) dlm_ckv_free_kv(cluster_data->pr_reg_kv[i]); kfree(cluster_data->pr_reg_kv); + dlm_ckv_free_notification(cluster_data->init_sync_notify); dlm_ckv_free_notification(cluster_data->pr_sync_notify); dlm_ckv_free_lock(cluster_data->pr_lock); dlm_ckv_free_kv(cluster_data->pr_data); @@ -410,6 +422,13 @@ static int target_pr_sync_dlm(struct se_device *dev, u8 pro_sa) return res; } +static void target_initial_sync_cb(void *arg) +{ + struct se_device *dev = arg; + + target_pr_sync_dlm(dev, PR_SYNC_REASON_INIT); +} + static int target_update_pr_reg(struct se_device *dev, struct t10_pr_registration *pr_reg, const struct pr_lvb *pr_data, @@ -584,6 +603,7 @@ static int target_dlm_read_cluster_data( if (!pr_data->version) { pr_info("TARGET_CORE[%d]: PR data from cluster is invalid\n", dev->dev_index); + res = -ENOLCK; goto done; } @@ -877,6 +897,20 @@ static void target_dlm_apply_cluster_data( kfree(pr_reg_data); } +static void target_initial_sync(struct se_device *dev) +{ + struct target_cluster_data *cluster_data = dev->cluster_data; + struct pr_reg_lvb *pr_reg_data = NULL; + struct pr_lvb pr_data; + int res; + + res = target_dlm_read_cluster_data(dev, &pr_data, &pr_reg_data); + if (!res) + target_dlm_apply_cluster_data(dev, &pr_data, pr_reg_data); + if (res == -ENOLCK) + dlm_ckv_notify(cluster_data->init_sync_notify); +} + static void target_pr_sync_cb(void *arg) { struct pr_reg_lvb *pr_reg_data = NULL;