From patchwork Fri Feb 26 08:32:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 388003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2C63C433DB for ; Fri, 26 Feb 2021 08:35:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 82F7464E20 for ; Fri, 26 Feb 2021 08:35:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbhBZIf3 (ORCPT ); Fri, 26 Feb 2021 03:35:29 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:9469 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229845AbhBZIfX (ORCPT ); Fri, 26 Feb 2021 03:35:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1614329689; x=1645865689; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+mj9k8DOW2ynjWjt3zJCuC4pRcPAUXqtzLGwdtsx21E=; b=QwNqoMw4zyG62bcIjjC/y8ZjNzpd6GpzUPopDu5YkXaZepoLADmk6U5c K8++FxJcX9fFXy5g316j0hGwfQUfH7wnLJHZdvFs2SK5TwQ3qsqZTGjNw spTSmHfEpojoejC1Nu29H4BcQ7dTxkB1XtJWdPdio+Q2bZrRMPX2bsr21 PB7dXimttm0c5f8s4GSRkVnprKV4ek6sYQ7XIINL+NOL+vAMwNZLMHZ2m 6G2CBjnWaafWfTMNgBEOGkooNeHjjKLdIPSot37yxCqFcWYgQy+0a82PH gPtZ5Lb5rNvVYZ6VwhpUs1MIReH9vKQrnHCCLoBQT1xUx8D6A9LatDpZO Q==; IronPort-SDR: EdYKVSNaKKBP4XVEQ7mMQ1e92vAggWsg/LXDNRBljwO1CbdwXws2V5+Hp87BAletj7a7cBkCHY mvdDcyMXWOsUvcATLsCyfRGQITJRp1W6YQbBed+B65T1Qs8bInX+5bahG1ijcHNTYgMlilyEyE 3i7e8/fvfwVpyUJtZqAEZBhdcCbzfTmwvYSg8FW55ZSBj6EQ5oY9LphALOfKeGQrvbNyJyCWLB vqBPmj5DRN3BOaG6NWaoKmeQDuIgr7KTbK09mEvx5MiRWvNqJqayWkoDIBsCq/nbL71Fw4ntHl 2UM= X-IronPort-AV: E=Sophos;i="5.81,208,1610380800"; d="scan'208";a="265100132" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 26 Feb 2021 16:52:54 +0800 IronPort-SDR: m6Gs0gfumMKMoMrDncaRMS7DR9sp5Nw94RwMojOFHskVbVsrJ0L/a4Y/LPzWvJFM4ZOhMijfqQ pJtYu1Nb6vugVVCstrp6aHQYYLL9KqI+kOMdenZzZGiYONVrYwGmPianABks6Rw+eqv8rS2RJJ VTR4PE2uv+bsce9pFjTqkXz87XVeG57ycs7Z+RIIAcJJOhWdssIGA4IvJ8tk3QH3YCVYhtbLZ1 IS2X6YmLi/fYFL3amZQfXfv5HL+p5+oPXbgFj2+S7z8tUuhv0JI9syWJ7YKJU77Law+AROqFFr kCDh+LCtuGm0VLWpD8UrJp8e Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 00:17:20 -0800 IronPort-SDR: uJ3hqNSeyqH1HF3zIc0JwbmSGDbh6pvMtx1NEvgibR25EbjGnJRogn6BMU+Lu8R+6DVT4bViht pKsV47+aMIDco+SIPKrfQ59VNfR5/tq4kGvhyblVN6Co3RaurMo9JYog+4zCaVJ1xxJOPStyg+ 3RtO+TI5EtiYfclCdtqnPlVltPqrrGkGUIpHLBFyz6CiG3e9ee7QtGqgYSuqd64LaGVymw/4bA bvEybD/3PfWZSAfRKB69HibdkmB8iVgx+ZETHUS9pFcClsoikcN0idlcLtg+AjFzt7CfplgriI 8og= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 26 Feb 2021 00:34:02 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v4 2/9] scsi: ufshpb: Add host control mode support to rsp_upiu Date: Fri, 26 Feb 2021 10:32:53 +0200 Message-Id: <20210226083300.30934-3-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210226083300.30934-1-avri.altman@wdc.com> References: <20210226083300.30934-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In device control mode, the device may recommend the host to either activate or inactivate a region, and the host should follow. Meaning those are not actually recommendations, but more of instructions. On the contrary, in host control mode, the recommendation protocol is slightly changed: a) The device may only recommend the host to update a subregion of an already-active region. And, b) The device may *not* recommend to inactivate a region. Furthermore, in host control mode, the host may choose not to follow any of the device's recommendations. However, in case of a recommendation to update an active and clean subregion, it is better to follow those recommendation because otherwise the host has no other way to know that some internal relocation took place. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 34 +++++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 2 ++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index d9ea0cddc3c4..044fec9854a0 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -166,6 +166,8 @@ static void ufshpb_set_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, else set_bit_len = cnt; + set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + if (rgn->rgn_state != HPB_RGN_INACTIVE && srgn->srgn_state == HPB_SRGN_VALID) bitmap_set(srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); @@ -235,6 +237,11 @@ static bool ufshpb_test_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, return false; } +static inline bool is_rgn_dirty(struct ufshpb_region *rgn) +{ + return test_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); +} + static int ufshpb_fill_ppn_from_page(struct ufshpb_lu *hpb, struct ufshpb_map_ctx *mctx, int pos, int len, u64 *ppn_buf) @@ -717,6 +724,7 @@ static void ufshpb_put_map_req(struct ufshpb_lu *hpb, static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, struct ufshpb_subregion *srgn) { + struct ufshpb_region *rgn; u32 num_entries = hpb->entries_per_srgn; if (!srgn->mctx) { @@ -730,6 +738,10 @@ static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, num_entries = hpb->last_srgn_entries; bitmap_zero(srgn->mctx->ppn_dirty, num_entries); + + rgn = hpb->rgn_tbl + srgn->rgn_idx; + clear_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + return 0; } @@ -1257,6 +1269,18 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, srgn_i = be16_to_cpu(rsp_field->hpb_active_field[i].active_srgn); + rgn = hpb->rgn_tbl + rgn_i; + if (hpb->is_hcm && + (rgn->rgn_state != HPB_RGN_ACTIVE || is_rgn_dirty(rgn))) { + /* + * in host control mode, subregion activation + * recommendations are only allowed to active regions. + * Also, ignore recommendations for dirty regions - the + * host will make decisions concerning those by himself + */ + continue; + } + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "activate(%d) region %d - %d\n", i, rgn_i, srgn_i); @@ -1264,7 +1288,6 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, ufshpb_update_active_info(hpb, rgn_i, srgn_i); spin_unlock(&hpb->rsp_list_lock); - rgn = hpb->rgn_tbl + rgn_i; srgn = rgn->srgn_tbl + srgn_i; /* blocking HPB_READ */ @@ -1275,6 +1298,14 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_active_cnt++; } + if (hpb->is_hcm) { + /* + * in host control mode the device is not allowed to inactivate + * regions + */ + goto out; + } + for (i = 0; i < rsp_field->inactive_rgn_cnt; i++) { rgn_i = be16_to_cpu(rsp_field->hpb_inactive_field[i]); dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, @@ -1299,6 +1330,7 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_inactive_cnt++; } +out: dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "Noti: #ACT %u #INACT %u\n", rsp_field->active_rgn_cnt, rsp_field->inactive_rgn_cnt); diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 14b7ba9bda3a..8119b1a3d1e5 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -119,6 +119,8 @@ struct ufshpb_region { /* below information is used by lru */ struct list_head list_lru_rgn; + unsigned long rgn_flags; +#define RGN_FLAG_DIRTY 0 }; #define for_each_sub_region(rgn, i, srgn) \ From patchwork Fri Feb 26 08:32:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 388001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B753AC433E0 for ; Fri, 26 Feb 2021 08:36:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 700B364EE3 for ; Fri, 26 Feb 2021 08:36:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230041AbhBZIgy (ORCPT ); Fri, 26 Feb 2021 03:36:54 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:9535 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229989AbhBZIgB (ORCPT ); Fri, 26 Feb 2021 03:36:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1614329746; x=1645865746; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uTq6wIUb+TSt9CNKgvD5aCPIirS1X26O3rqisHDbZ78=; b=lVx95mHbQRrle46Mz7gZ+OtkcyYtOYEkyqP5YA782oMWxulMu5TbTL0F iw3EGu33mnbmZM18MZf5GxxwV6jDgH+oDwVReExtKAl7V8F5giGJFb27F mK0isGWNWA2P07ee7lxE1kNJu5JRkW/+5tl3rywbzpf6MZHveNQGcIqV6 46/W1qSXcao7fPl3gzJv0Sopiw3LvrkQw103QvanP9x7TEqcMUjwqv2tn s+eL2CdKsUZb8z7gxrgUZAve8MCoyklmhVee88NQ7gaI2g1wiTl8VUuJR KsMzwwu07+3Zyl3DAJyRW4dHnH1j6VoEO7QicUO4WWgaXaJWKgWIqqEEa A==; IronPort-SDR: x3Pfa2K7nVMj4RWiXERlwVJv0zud9LeaqYdY9THI61/fM+RFOBls5xH0J5JQN7PpRuxNAO8Xg0 edEXoHg/DzT785R1woDGu8ggNTvkBbR8KRbjhMgpq9GoM+GgDYN85MAfkRWT8Q5eDnWZa6QHVB 1C4rbb71vsoPBIaebVUjDlOk54U8q+Ostb2DSsE3aOjGNc9WHQhv+bK/BmhhYEif4/K1upHUXz WAXSpHGY9wJ7AROWFw1S3uvaUnNUu9MyYfUCGXSlR/wCDY2PYuh/7q7OQQtXc+XgMoj11dYRsU HDA= X-IronPort-AV: E=Sophos;i="5.81,208,1610380800"; d="scan'208";a="265100156" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 26 Feb 2021 16:53:10 +0800 IronPort-SDR: RI/Y9JIJSw00UzHbAODUXIf7qlb0u9u+e663FuSG5G30YYePuNkl3aqnKizhOSM4Z38RHzWr2A kJdibRYNzAGtgZBSwtq8zU1VpjYSalZxeK5Do9BzeaC1nzjjQVtiZx885GtNYCbH04So6HQ49B 9Q+4SzTHeIkQqFSL82Ee487mMWk6Pgw6hXoqcGIlelneLRuQBj3wrZGGWRtEWQpVpZJmvB7bnh /SVzsaTU81hWsggECfB/iRUNEXeGkmS0fXoKoP3L+Vf/QGh4kNEmp97CA5dVe/6jSRmz+9SP80 T5Pcco4pbnK3Dso9+L6ndcon Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 00:17:30 -0800 IronPort-SDR: YJ4cSZnaQoSNhQWB+qcNDsofbro6ikRca2IVDnuSMev05rxwvQURvivrJ5/xE6sQ1nLaN3T8+u 7FQauBA7O8ffp97ahByQQEge95orb56th0GX9Md/SJT4FbNQG6WKAL2qVGw/noT82TGfPe11sb zU9z9AwJZgq6R5nei7mxiGPOKpmTDBIE0NQP1x4OXEM/m96sq7OT0QK/mE8VKV2BN8HJi2Hlfq vpJpLXDfDC7biGGCpwqqjAfucBgE3wayEn0/TlZAI2o0c1cLa4cbub+h4i2NjUMClWUMvpbX6Q JBM= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 26 Feb 2021 00:34:13 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v4 3/9] scsi: ufshpb: Add region's reads counter Date: Fri, 26 Feb 2021 10:32:54 +0200 Message-Id: <20210226083300.30934-4-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210226083300.30934-1-avri.altman@wdc.com> References: <20210226083300.30934-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host control mode, reads are the major source of activation trials. Keep track of those reads counters, for both active as well inactive regions. We reset the read counter upon write - we are only interested in "clean" reads. less intuitive however, is that we also reset it upon region's deactivation. Region deactivation is often due to the fact that eviction took place: a region become active on the expense of another. This is happening when the max-active-regions limit has crossed. If we don’t reset the counter, we will trigger a lot of trashing of the HPB database, since few reads (or even one) to the region that was deactivated, will trigger a re-activation trial. Keep those counters normalized, as we are using those reads as a comparative score, to make various decisions. If during consecutive normalizations an active region has exhaust its reads - inactivate it. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 108 ++++++++++++++++++++++++++++++++------ drivers/scsi/ufs/ufshpb.h | 7 +++ 2 files changed, 100 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 044fec9854a0..717ccb66b33e 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -16,6 +16,8 @@ #include "ufshpb.h" #include "../sd.h" +#define ACTIVATION_THRESHOLD 4 /* 4 IOs */ + /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; static mempool_t *ufshpb_mctx_pool; @@ -554,6 +556,21 @@ static int ufshpb_issue_pre_req(struct ufshpb_lu *hpb, struct scsi_cmnd *cmd, return ret; } +static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, + int srgn_idx) +{ + struct ufshpb_region *rgn; + struct ufshpb_subregion *srgn; + + rgn = hpb->rgn_tbl + rgn_idx; + srgn = rgn->srgn_tbl + srgn_idx; + + list_del_init(&rgn->list_inact_rgn); + + if (list_empty(&srgn->list_act_srgn)) + list_add_tail(&srgn->list_act_srgn, &hpb->lh_act_srgn); +} + /* * This function will set up HPB read command using host-side L2P map data. */ @@ -600,12 +617,45 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) ufshpb_set_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, transfer_len); spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + if (hpb->is_hcm) { + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = 0; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + return 0; } if (!ufshpb_is_support_chunk(hpb, transfer_len)) return 0; + if (hpb->is_hcm) { + bool activate = false; + /* + * in host control mode, reads are the main source for + * activation trials. + */ + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads++; + if (rgn->reads == ACTIVATION_THRESHOLD) + activate = true; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + if (activate) { + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_active_info(hpb, rgn_idx, srgn_idx); + hpb->stats.rb_active_cnt++; + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, + "activate region %d-%d\n", rgn_idx, srgn_idx); + } + + /* keep those counters normalized */ + if (rgn->reads > hpb->entries_per_srgn && + !test_and_set_bit(WORK_PENDING, &hpb->work_data_bits)) + schedule_work(&hpb->ufshpb_normalization_work); + } + spin_lock_irqsave(&hpb->rgn_state_lock, flags); if (ufshpb_test_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, transfer_len)) { @@ -745,21 +795,6 @@ static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, return 0; } -static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, - int srgn_idx) -{ - struct ufshpb_region *rgn; - struct ufshpb_subregion *srgn; - - rgn = hpb->rgn_tbl + rgn_idx; - srgn = rgn->srgn_tbl + srgn_idx; - - list_del_init(&rgn->list_inact_rgn); - - if (list_empty(&srgn->list_act_srgn)) - list_add_tail(&srgn->list_act_srgn, &hpb->lh_act_srgn); -} - static void ufshpb_update_inactive_info(struct ufshpb_lu *hpb, int rgn_idx) { struct ufshpb_region *rgn; @@ -1079,6 +1114,14 @@ static void __ufshpb_evict_region(struct ufshpb_lu *hpb, ufshpb_cleanup_lru_info(lru_info, rgn); + if (hpb->is_hcm) { + unsigned long flags; + + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = 0; + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + for_each_sub_region(rgn, srgn_idx, srgn) ufshpb_purge_active_subregion(hpb, srgn); } @@ -1523,6 +1566,36 @@ static void ufshpb_run_inactive_region_list(struct ufshpb_lu *hpb) spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); } +static void ufshpb_normalization_work_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb; + int rgn_idx; + + hpb = container_of(work, struct ufshpb_lu, ufshpb_normalization_work); + + for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { + struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; + + if (rgn->reads) { + unsigned long flags; + + spin_lock_irqsave(&rgn->rgn_lock, flags); + rgn->reads = (rgn->reads >> 1); + spin_unlock_irqrestore(&rgn->rgn_lock, flags); + } + + if (rgn->rgn_state != HPB_RGN_ACTIVE || rgn->reads) + continue; + + /* if region is active but has no reads - inactivate it */ + spin_lock(&hpb->rsp_list_lock); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + spin_unlock(&hpb->rsp_list_lock); + } + + clear_bit(WORK_PENDING, &hpb->work_data_bits); +} + static void ufshpb_map_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, map_work); @@ -1913,6 +1986,9 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); + if (hpb->is_hcm) + INIT_WORK(&hpb->ufshpb_normalization_work, + ufshpb_normalization_work_handler); hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -2012,6 +2088,8 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { + if (hpb->is_hcm) + cancel_work_sync(&hpb->ufshpb_normalization_work); cancel_work_sync(&hpb->map_work); } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 8119b1a3d1e5..2fbe928ae7fd 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -121,6 +121,10 @@ struct ufshpb_region { struct list_head list_lru_rgn; unsigned long rgn_flags; #define RGN_FLAG_DIRTY 0 + + /* region reads - for host mode */ + spinlock_t rgn_lock; + unsigned int reads; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -211,6 +215,9 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; + struct work_struct ufshpb_normalization_work; + unsigned long work_data_bits; +#define WORK_PENDING 0 /* pinned region information */ u32 lu_pinned_start; From patchwork Fri Feb 26 08:32:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 388002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51BC9C433E0 for ; Fri, 26 Feb 2021 08:36:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04C9864E20 for ; Fri, 26 Feb 2021 08:36:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230169AbhBZIfx (ORCPT ); Fri, 26 Feb 2021 03:35:53 -0500 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:28206 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230215AbhBZIfo (ORCPT ); Fri, 26 Feb 2021 03:35:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1614328543; x=1645864543; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3uOjSmOzjY24VNbjOX25A85YrvCh/iOBT1jHqLJB7pU=; b=NQL/+uT0S3lgza/G7mq9MwoAfWVXIoQRcaNuhokxs1u7+TzEf5bcFhlw MZaV0y23ew2e2y8ytELTHPmZpogz6//OGahSkF+a7j/OnIoPVuPZnMzlr Cnyvo26SmYhR5EL0eLWET0KdyDcRORHjswXp2OjHjoQmAVa2c4Zm8rIqw xYE4Bhi80dAiamr6vNJGYxYDtk0YUupnPGcuwI92UBtvw1xfgw2pHa5YX pDKaPeXoQC77EvySUKp+hovha0zRMRz/MNifMFyMBMPCxP/d82C5CfnNt GBjc/pvxEQx+ioYRStSqxQMgto5MSgDh6lKmcCAsuKqUK3xcRu+6aHXdG A==; IronPort-SDR: uQh6wbbH6kmOIukTx91PoRmSnnqdeBNsvImasfTkkGY+132t1xJ6ZuaH/ZMj5DgaPLXdhKP3Hv hR3IwsAHDGSK6jR37ZIHZW4nvThZlY9aXiUnsTNeCuAZhRsoK4Msr6MA36/pEHyGcAI6BMOiUm MBfK+9XD2z0UtlAw6U1OJ7kmWy6dF2TK12awO8etWOcVIvyjqfRIO9QSUwsXfC2HhywAYAOXC/ a+BXESwqMFwMEDeId66lHFNOor2eCDi1ngb/ru/XRLXVmVgG3mZII9RTBwhkWl4FtmRUZxldNB Y2Q= X-IronPort-AV: E=Sophos;i="5.81,208,1610380800"; d="scan'208";a="165357041" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 26 Feb 2021 16:34:27 +0800 IronPort-SDR: HrDXkslQFKxFHzHEIVJmKLXAG8sAc5hoUxODX+ptRZEvZ9HE2fIBxS2mDLMB+zAKXLNy25iWiL Hx+TJAn911ottkK/3GLastTI79kgKYQXMb/bUmxLDpuUESeubpQvnMkXI/BlPWdVXqVCphUV1j t6lg+ZfISrfC6Rsq7DhGJauAABAbZMxh0T8E8NOrQKo3/NGJ+jw5KB3e/+RvqiSaP2D7ANqYma 806njrIDysnvzEGterleMp4j8ikqKVxeaqy2BNJPOB8kx4divhGjwBLXWwjrRTqifQ5pUvmHkV 2nPjEbbB46+0p7c7MPh0mn6z Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 00:15:47 -0800 IronPort-SDR: eubKs3W8xT23Z8jNrEmRNgAYGnsiD7dWenQgNHEndpVCRMRP4+5t5gRQrJ3vz6+9FSjv9ZPEqH b0Km3e/0DGhauYMPjlV3eG6LohKbDUI7nVQm2o42cwcF7BSPK9BVS6/X5KlPzyvT6kfiVGmrsG p9CMchWI25dIvzkyp0XNx+yGBDdrqnswkOKILyGP16vWtlDCCziR1EKY4fzBuilsUXueCWlhVr quq6HXWP2MB3KRrF97b62RMcVWfrp7mgvBQ50Qs3aN4YA+sqNzNki/cZEszsdv2qz0wBvRuf2b l1A= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 26 Feb 2021 00:34:22 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v4 4/9] scsi: ufshpb: Make eviction depends on region's reads Date: Fri, 26 Feb 2021 10:32:55 +0200 Message-Id: <20210226083300.30934-5-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210226083300.30934-1-avri.altman@wdc.com> References: <20210226083300.30934-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host mode, eviction is considered an extreme measure. verify that the entering region has enough reads, and the exiting region has much less reads. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 717ccb66b33e..44e56a6d7102 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,6 +17,7 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 4 /* 4 IOs */ +#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 6) /* 256 IOs */ /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -1051,6 +1052,13 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) if (ufshpb_check_srgns_issue_state(hpb, rgn)) continue; + /* + * in host control mode, verify that the exiting region + * has less reads + */ + if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + continue; + victim_rgn = rgn; break; } @@ -1236,7 +1244,7 @@ static int ufshpb_issue_map_req(struct ufshpb_lu *hpb, static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) { - struct ufshpb_region *victim_rgn; + struct ufshpb_region *victim_rgn = NULL; struct victim_select_info *lru_info = &hpb->lru_info; unsigned long flags; int ret = 0; @@ -1264,6 +1272,16 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * because the device could detect this region * by not issuing HPB_READ */ + + /* + * in host control mode, verify that the entering + * region has enough reads + */ + if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + ret = -EACCES; + goto out; + } + victim_rgn = ufshpb_victim_lru_info(hpb); if (!victim_rgn) { dev_warn(&hpb->sdev_ufs_lu->sdev_dev, From patchwork Fri Feb 26 08:32:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 388000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F45FC433E0 for ; Fri, 26 Feb 2021 08:37:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0847B64E20 for ; Fri, 26 Feb 2021 08:37:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230100AbhBZIhK (ORCPT ); Fri, 26 Feb 2021 03:37:10 -0500 Received: from esa1.hgst.iphmx.com ([68.232.141.245]:11508 "EHLO esa1.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230188AbhBZIgM (ORCPT ); Fri, 26 Feb 2021 03:36:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1614328571; x=1645864571; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5o1Joy+YObhraRwBiKgOJuCuzFTW6xDSF5NHVSdNyk0=; b=RFB424/GPzPGGoesrVnyB6Zp7NSJQbLdvCwTJ5jjb7qZjMKnQtzu70Ga RAL/uHfMUxJeyjAZ/VNnQL6WFF6boqu5jbjZ8aCfdcIhjrbp7iFGuSgFZ 4IAWSTDiq2g0RhQC6fl7Y6iSs7b2iEvRqaa1+J/ztIOC7r58o8tJsh2jX 4fGaRBpNW3w3/RXH/vH3tGmPccO9punFjnlJqyBhDpRoxcaDPGlc7eUa3 Lj9GeF4RUCqUBKDlELTzoaxGLdzwSLhPqpGIVlr0Mqoe57ajS8YCxb/TU lP7SLTTkHNAN6ZPvyb96LMdtCL9CE/ADaSRtnb2szAvgurWmnVXgCUhWn A==; IronPort-SDR: jGZvRZVlh8jTkihstFnH/R7WUN+odmMdhyfub6qmSlMf25PUCdS57I8NCy0d2UvjOOBHnvBK+I njN8dRNi9Al9IE1uJPI1Ih0q0AzwEAQG2Qkcj4GZQxCu6Uf6ZyhPuIGn5Vv0lJfTSJV5d6UweE Ls7JyhpzhdtyU/0s4KYWoYAT6ADMUaRn9S5LfR+M05y05F4ycX8J4IvKqA0I8EKX4+sYlgU1mk k1BEc6hFvZJFLdMtFsF1Mjud7BVmpVKRrzjmypUIXGsTD7TyqvsoXCVmRTnxt1hYq+pqyFZBdY 1kU= X-IronPort-AV: E=Sophos;i="5.81,208,1610380800"; d="scan'208";a="271435340" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 26 Feb 2021 16:35:02 +0800 IronPort-SDR: 2ddLZ/0RoRyDjxoNmwSsVJB/jzq/qkJGk+L+u6Bkgt7pVLZXuoC8mnH0znps0zjPLiPuNTtKOU mpjJqYTbnLjMxPBPlaRUTNlPom+4aXl30eGuBJmUsuhIeWBj2cWmcWSt+5I8kitQ5nNoheszPb /mB3hOkUR/dhqt2A0Li46ZhGKWly1GvsrkigHCuKTxA2kDv48rwqNR9SU7fFnEfNq0go9uEkRy cMSLwNwrlRi2IpJD0iecwx30efP3KYv/wZ560ngTMYpWyHCM6DHT+LF0Fy+92fi6k48pJ9rend 2gSrVc1hIP2rohMcRRa6NDUG Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 00:18:16 -0800 IronPort-SDR: H/OmzIMgJ2nT9kd5lxl+YH0xaxPmxpOI7e82RGj5U9U6AG7AiUuKPkrx/Qj0HjP/VQdmU7bkev sLRPH+KOKGHCJDf9kMicQFMwn/RORu64T4kuO5CEqLOrTt84GG4fnr22t4BRTE4ISDG688ndmb EXB08adlZRu0PkrpwIEPJMANgj2Fbpxw4R0ta+ciJLeem+kdIVfAzg+vxfl9umclzhtkdy/VWD pL5+C/kj2UJUfhmdAVlpkxU8PIlXAdq97oTCKG2d/GxZ3sYFLq9EtuXBL2vQcL/+JPlrh6EN5K ZGk= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 26 Feb 2021 00:34:58 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v4 8/9] scsi: ufshpb: Add support for host control mode Date: Fri, 26 Feb 2021 10:32:59 +0200 Message-Id: <20210226083300.30934-9-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210226083300.30934-1-avri.altman@wdc.com> References: <20210226083300.30934-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Support devices that report they are using host control mode. Signed-off-by: Avri Altman --- drivers/scsi/ufs/ufshpb.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 5b76341fd558..86f4720f4f0d 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -2573,12 +2573,6 @@ void ufshpb_get_dev_info(struct ufs_hba *hba, u8 *desc_buf) u32 max_hpb_sigle_cmd = 0; hpb_dev_info->control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; - if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) { - dev_err(hba->dev, "%s: host control mode is not supported.\n", - __func__); - hpb_dev_info->hpb_disabled = true; - return; - } version = get_unaligned_be16(desc_buf + DEVICE_DESC_PARAM_HPB_VER); if ((version != HPB_SUPPORT_VERSION) && From patchwork Fri Feb 26 08:33:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 387999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D931FC433DB for ; Fri, 26 Feb 2021 08:38:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D23464EE7 for ; Fri, 26 Feb 2021 08:38:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230006AbhBZIhR (ORCPT ); Fri, 26 Feb 2021 03:37:17 -0500 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:52835 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230122AbhBZIgP (ORCPT ); Fri, 26 Feb 2021 03:36:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1614328574; x=1645864574; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XzEuJ0bjvB4QfBo7WhmbY0snmseA3+UXNSyqutCePg8=; b=Gmxf0cKwl3q4XMsgDVWcVgnMWYXghDGMLPxC6ztKfwio/LjK3PSvyLuq h5ls2zx/lQkZnbFMOJqebicDkkWh+cKOhNqCtrtNnKnNTq8TnwSZnAleD rI7/0HVWIX63gcxjbtb6Pqqg2L4UrTFPpq28Cme8Swe/aDAy3CnfXjpKE bkC1ipu+PGQk7s9bkZpblqtMgk4dwXHweR+AvDq6X0xvJ4A2vgln05/4R ZeroZK2RDJ2ewxqDP5HftcwiiuUe563Eyvl1akuY2GfuEEStjywgb10vj ub6gh/dKcxk4vF9iItqFoGQZ8RmqHJufWQHYeepEjwsZ125ebWP4BAp9T A==; IronPort-SDR: 9ArIrNgnBMUrnr/JcY22SXplFz8nDi5R2t1XTq/neigbXRuKil/O5dMct36wv4aTFaLma0K4/d WSpwD3jfa672HeyigeSYQf1RtuU9QK7eX3QUsUv0+UkH63Dx02NO6Wn0CKE4iym5pnp8Wwaqa3 0v3eZ4dnBRXXFTFsA916uEqmnuBwUKlQOMY+zWX7/TtkmHo/TJNFnXUHw8c1dpBdZf5qaHsYZC BRR5T1qZyA8EvGe2nRXtZ+KTIZpGcqd7bvN1higmJ0meQcuPAEJaMDPzr/JvW/oZaP4xKE12GQ T64= X-IronPort-AV: E=Sophos;i="5.81,208,1610380800"; d="scan'208";a="160859719" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 26 Feb 2021 16:35:09 +0800 IronPort-SDR: 331hAUSDZ7jS8/y7jdpssfw83nohfKEMU8jhbdWJlPwvIbJVXMJ1hTcAieQUXyFY6eu9OShW7k uxZcyJWkpzYkNH/RI5ORNhDFd69P5QrOGjoz7p2dSjNMiyDrqaNADivLDX3e2J84HfoM14emOs K8lbBzW0EpSK8DtfaT+wcB0KKvV+AuEUnY7MBxyPRN5g6bfhzmhzdeMngdaTlV/MDsFQW2uCaR cS6xZHgjQMMext3g8zuPLFgSGljY2BUocAE/TOigVbzOLg0RWeBtc6bQoOA3P4hjreZe/zOtlH OK6gU+E7fEeZk5LZcpkGq6tT Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2021 00:18:22 -0800 IronPort-SDR: u3qggXPBISht++ka76ehj0azDnjwaC51rtxDICy0fNdeQkJqrjLo9ddLsgaPRMzVAItKt4TsDA yBhfidZzp8rlh06tWwuuUq5Tux7J+mBiGLX2ZDYg53WWe9LOb0spCJWmQDwCxoNZ4rlv3YvfCh bzzN725JcjIp/rV38CHMwhjcWZIkGV6/ngddPnpfl2GnC0tvGt8LcViJGCZQUQh+9XOb8H7A9X cBwZJN2DPovvSB3Z9F9QZmDlvjMcTQaU/KCRR0sirOisr0Zk0fwtR/RgAoNkOni/kgnO+/r4ZK GU0= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 26 Feb 2021 00:35:05 -0800 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v4 9/9] scsi: ufshpb: Make host mode parameters configurable Date: Fri, 26 Feb 2021 10:33:00 +0200 Message-Id: <20210226083300.30934-10-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210226083300.30934-1-avri.altman@wdc.com> References: <20210226083300.30934-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org We can make use of this commit, to elaborate some more of the host control mode logic, explaining what role play each and every variable. While at it, allow those parameters to be configurable. Signed-off-by: Avri Altman --- Documentation/ABI/testing/sysfs-driver-ufs | 67 ++++++ drivers/scsi/ufs/ufshpb.c | 253 +++++++++++++++++++-- drivers/scsi/ufs/ufshpb.h | 18 ++ 3 files changed, 324 insertions(+), 14 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs index 0017eaf89cbe..d9eca371c507 100644 --- a/Documentation/ABI/testing/sysfs-driver-ufs +++ b/Documentation/ABI/testing/sysfs-driver-ufs @@ -1322,3 +1322,70 @@ Description: This entry shows the maximum HPB data size for using single HPB === ======== The file is read only. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/activation_thld +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, reads are the major source of activation + trials. once this threshold hs met, the region is added to the + "to-be-activated" list. Since we reset the read counter upon + write, this include sending a rb command updating the region + ppn as well. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/normalization_factor +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, We think of the regions as "buckets". + Those buckets are being filled with reads, and emptied on write. + We use entries_per_srgn - the amount of blocks in a subregion as + our bucket size. This applies because HPB1.0 only concern a + single-block reads. Once the bucket size is crossed, we trigger + a normalization work - not only to avoid overflow, but mainly + because we want to keep those counters normalized, as we are + using those reads as a comparative score, to make various decisions. + The normalization is dividing (shift right) the read counter by + the normalization_factor. If during consecutive normalizations + an active region has exhaust its reads - inactivate it. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_enter +Date: February 2021 +Contact: Avri Altman +Description: Region deactivation is often due to the fact that eviction took + place: a region become active on the expense of another. This is + happening when the max-active-regions limit has crossed. + In host mode, eviction is considered an extreme measure. We + want to verify that the entering region has enough reads, and + the exiting region has much less reads. eviction_thld_enter is + the min reads that a region must have in order to be considered + as a candidate to evict other region. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_exit +Date: February 2021 +Contact: Avri Altman +Description: same as above for the exiting region. A region is consider to + be a candidate to be evicted, only if it has less reads than + eviction_thld_exit. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_ms +Date: February 2021 +Contact: Avri Altman +Description: In order not to hang on to “cold” regions, we shall inactivate + a region that has no READ access for a predefined amount of + time - read_timeout_ms. If read_timeout_ms has expired, and the + region is dirty - it is less likely that we can make any use of + HPB-READing it. So we inactivate it. Still, deactivation has + its overhead, and we may still benefit from HPB-READing this + region if it is clean - see read_timeout_expiries. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_expiries +Date: February 2021 +Contact: Avri Altman +Description: if the region read timeout has expired, but the region is clean, + just re-wind its timer for another spin. Do that as long as it + is clean and did not exhaust its read_timeout_expiries threshold. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/timeout_polling_interval_ms +Date: February 2021 +Contact: Avri Altman +Description: the frequency in which the delayed worker that checks the + read_timeouts is awaken. diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 86f4720f4f0d..7387c6d0f663 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,7 +17,6 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 4 /* 4 IOs */ -#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 6) /* 256 IOs */ #define READ_TO_MS 1000 #define READ_TO_EXPIRIES 100 #define POLLING_INTERVAL_MS 200 @@ -642,7 +641,7 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) */ spin_lock_irqsave(&rgn->rgn_lock, flags); rgn->reads++; - if (rgn->reads == ACTIVATION_THRESHOLD) + if (rgn->reads == hpb->params.activation_thld) activate = true; spin_unlock_irqrestore(&rgn->rgn_lock, flags); if (activate || @@ -1035,6 +1034,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) struct victim_select_info *lru_info; struct ufshpb_region *rgn; unsigned long flags; + unsigned int poll; LIST_HEAD(expired_list); hpb = container_of(dwork, struct ufshpb_lu, ufshpb_read_to_work); @@ -1056,7 +1056,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) list_add(&rgn->list_expired_rgn, &expired_list); else rgn->read_timeout = ktime_add_ms(ktime_get(), - READ_TO_MS); + hpb->params.read_timeout_ms); } } @@ -1074,8 +1074,9 @@ static void ufshpb_read_to_handler(struct work_struct *work) clear_bit(TIMEOUT_WORK_PENDING, &hpb->work_data_bits); + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); } static void ufshpb_add_lru_info(struct victim_select_info *lru_info, @@ -1085,8 +1086,11 @@ static void ufshpb_add_lru_info(struct victim_select_info *lru_info, list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); if (rgn->hpb->is_hcm) { - rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); - rgn->read_timeout_expiries = READ_TO_EXPIRIES; + rgn->read_timeout = + ktime_add_ms(ktime_get(), + rgn->hpb->params.read_timeout_ms); + rgn->read_timeout_expiries = + rgn->hpb->params.read_timeout_expiries; } } @@ -1115,7 +1119,8 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) * in host control mode, verify that the exiting region * has less reads */ - if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + if (hpb->is_hcm && + rgn->reads > hpb->params.eviction_thld_exit) continue; victim_rgn = rgn; @@ -1346,7 +1351,8 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * in host control mode, verify that the entering * region has enough reads */ - if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + if (hpb->is_hcm && + rgn->reads < hpb->params.eviction_thld_enter) { ret = -EACCES; goto out; } @@ -1696,8 +1702,10 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb; int rgn_idx; + u8 factor; hpb = container_of(work, struct ufshpb_lu, ufshpb_normalization_work); + factor = hpb->params.normalization_factor; for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; @@ -1706,7 +1714,7 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) unsigned long flags; spin_lock_irqsave(&rgn->rgn_lock, flags); - rgn->reads = (rgn->reads >> 1); + rgn->reads = (rgn->reads >> factor); spin_unlock_irqrestore(&rgn->rgn_lock, flags); } @@ -2028,8 +2036,216 @@ requeue_timeout_ms_store(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RW(requeue_timeout_ms); +ufshpb_sysfs_param_show_func(activation_thld); +static ssize_t +activation_thld_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.activation_thld = val; + + return count; +} +static DEVICE_ATTR_RW(activation_thld); + +ufshpb_sysfs_param_show_func(normalization_factor); +static ssize_t +normalization_factor_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0 || val > ilog2(hpb->entries_per_srgn)) + return -EINVAL; + + hpb->params.normalization_factor = val; + + return count; +} +static DEVICE_ATTR_RW(normalization_factor); + +ufshpb_sysfs_param_show_func(eviction_thld_enter); +static ssize_t +eviction_thld_enter_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.eviction_thld_exit) + return -EINVAL; + + hpb->params.eviction_thld_enter = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_enter); + +ufshpb_sysfs_param_show_func(eviction_thld_exit); +static ssize_t +eviction_thld_exit_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.activation_thld) + return -EINVAL; + + hpb->params.eviction_thld_exit = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_exit); + +ufshpb_sysfs_param_show_func(read_timeout_ms); +static ssize_t +read_timeout_ms_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.read_timeout_ms = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_ms); + +ufshpb_sysfs_param_show_func(read_timeout_expiries); +static ssize_t +read_timeout_expiries_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.read_timeout_expiries = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_expiries); + +ufshpb_sysfs_param_show_func(timeout_polling_interval_ms); +static ssize_t +timeout_polling_interval_ms_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.timeout_polling_interval_ms = val; + + return count; +} +static DEVICE_ATTR_RW(timeout_polling_interval_ms); + +static void ufshpb_hcm_param_init(struct ufshpb_lu *hpb) +{ + hpb->params.activation_thld = ACTIVATION_THRESHOLD; + hpb->params.normalization_factor = 1; + hpb->params.eviction_thld_enter = (ACTIVATION_THRESHOLD << 6); + hpb->params.eviction_thld_exit = (ACTIVATION_THRESHOLD << 5); + hpb->params.read_timeout_ms = READ_TO_MS; + hpb->params.read_timeout_expiries = READ_TO_EXPIRIES; + hpb->params.timeout_polling_interval_ms = POLLING_INTERVAL_MS; +} + static struct attribute *hpb_dev_param_attrs[] = { &dev_attr_requeue_timeout_ms.attr, + &dev_attr_activation_thld.attr, + &dev_attr_normalization_factor.attr, + &dev_attr_eviction_thld_enter.attr, + &dev_attr_eviction_thld_exit.attr, + &dev_attr_read_timeout_ms.attr, + &dev_attr_read_timeout_expiries.attr, + &dev_attr_timeout_polling_interval_ms.attr, NULL, }; @@ -2104,6 +2320,8 @@ static void ufshpb_stat_init(struct ufshpb_lu *hpb) static void ufshpb_param_init(struct ufshpb_lu *hpb) { hpb->params.requeue_timeout_ms = HPB_REQUEUE_TIME_MS; + if (hpb->is_hcm) + ufshpb_hcm_param_init(hpb); } static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) @@ -2160,9 +2378,13 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); - if (hpb->is_hcm) + if (hpb->is_hcm) { + unsigned int poll; + + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); + } return 0; @@ -2342,10 +2564,13 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); - if (hpb->is_hcm) - schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + if (hpb->is_hcm) { + unsigned int poll = + hpb->params.timeout_polling_interval_ms; + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(poll)); + } } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 8d14f01b0e7b..9703fe9807ad 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -180,8 +180,26 @@ struct victim_select_info { atomic_t active_cnt; }; +/** + * ufshpb_params - ufs hpb parameters + * @requeue_timeout_ms - requeue threshold of wb command (0x2) + * @activation_thld - min reads [IOs] to activate/update a region + * @normalization_factor - shift right the region's reads + * @eviction_thld_enter - min reads [IOs] for the entering region in eviction + * @eviction_thld_exit - max reads [IOs] for the exiting region in eviction + * @read_timeout_ms - timeout [ms] from the last read IO to the region + * @read_timeout_expiries - amount of allowable timeout expireis + * @timeout_polling_interval_ms - frequency in which timeouts are checked + */ struct ufshpb_params { unsigned int requeue_timeout_ms; + unsigned int activation_thld; + unsigned int normalization_factor; + unsigned int eviction_thld_enter; + unsigned int eviction_thld_exit; + unsigned int read_timeout_ms; + unsigned int read_timeout_expiries; + unsigned int timeout_polling_interval_ms; }; struct ufshpb_stats {