From patchwork Mon Jan 25 07:08:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61879C433DB for ; Mon, 25 Jan 2021 07:37:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 214CC20727 for ; Mon, 25 Jan 2021 07:37:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727272AbhAYHgq (ORCPT ); Mon, 25 Jan 2021 02:36:46 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33742 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727243AbhAYHWD (ORCPT ); Mon, 25 Jan 2021 02:22:03 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 6C3637A51; Sun, 24 Jan 2021 23:08:22 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 6C3637A51 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558502; bh=rLB4tGyOrnDJIrcxPjan3TVZfTPm4xTW0I3JrAtphE0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J0EpNd4SMwus8z+hqrKg0u9OorWrldQJ2vxwpv5x/YpNLIRaZuhOre6w2cjq6r8Mh /3JH+2dWGij2fcY7aydY3oJ3ypwV+6s0P2GSvG7Sg//35EC5XIj43VtDgB2Wv7Qc3z J/+AghA7OAq6n9x6ayLQZDfz+0JmjlPtOkseMrYk= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 02/15] bnxt_en: Define macros for the various health register states. Date: Mon, 25 Jan 2021 02:08:08 -0500 Message-Id: <1611558501-11022-3-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Define macros to check for the various states in the lower 16 bits of the health register. Replace the C code that checks for these values with the newly defined macros. Reviewed-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 10 ++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c | 7 +++---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index d68065367cf2..a1dd80a0fcf6 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1534,9 +1534,19 @@ struct bnxt_fw_reporter_ctx { #define BNXT_FW_HEALTH_WIN_OFF(reg) (BNXT_FW_HEALTH_WIN_BASE + \ ((reg) & BNXT_GRC_OFFSET_MASK)) +#define BNXT_FW_STATUS_HEALTH_MSK 0xffff #define BNXT_FW_STATUS_HEALTHY 0x8000 #define BNXT_FW_STATUS_SHUTDOWN 0x100000 +#define BNXT_FW_IS_HEALTHY(sts) (((sts) & BNXT_FW_STATUS_HEALTH_MSK) ==\ + BNXT_FW_STATUS_HEALTHY) + +#define BNXT_FW_IS_BOOTING(sts) (((sts) & BNXT_FW_STATUS_HEALTH_MSK) < \ + BNXT_FW_STATUS_HEALTHY) + +#define BNXT_FW_IS_ERR(sts) (((sts) & BNXT_FW_STATUS_HEALTH_MSK) > \ + BNXT_FW_STATUS_HEALTHY) + struct bnxt { void __iomem *bar0; void __iomem *bar1; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c index 6b7b69ed62db..90a31b4a3020 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c @@ -44,21 +44,20 @@ static int bnxt_fw_reporter_diagnose(struct devlink_health_reporter *reporter, struct netlink_ext_ack *extack) { struct bnxt *bp = devlink_health_reporter_priv(reporter); - u32 val, health_status; + u32 val; int rc; if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) return 0; val = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG); - health_status = val & 0xffff; - if (health_status < BNXT_FW_STATUS_HEALTHY) { + if (BNXT_FW_IS_BOOTING(val)) { rc = devlink_fmsg_string_pair_put(fmsg, "Description", "Not yet completed initialization"); if (rc) return rc; - } else if (health_status > BNXT_FW_STATUS_HEALTHY) { + } else if (BNXT_FW_IS_ERR(val)) { rc = devlink_fmsg_string_pair_put(fmsg, "Description", "Encountered fatal error and cannot recover"); if (rc) From patchwork Mon Jan 25 07:08:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4016EC433E0 for ; Mon, 25 Jan 2021 07:43:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB6E420727 for ; Mon, 25 Jan 2021 07:43:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727081AbhAYHmp (ORCPT ); Mon, 25 Jan 2021 02:42:45 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33568 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727208AbhAYHUk (ORCPT ); Mon, 25 Jan 2021 02:20:40 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 908857DC5; Sun, 24 Jan 2021 23:08:22 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 908857DC5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558502; bh=bUiy+zPCUYxmqywN6AB3sPmoRYCpg7U2j6RH2rBAZrA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QjRaQH3v7eDmZwWcxFLgU1KNtShHITB6LEq52sJcBb50gkoNtcguFgFgj7PZ2zruu VfBxVH5zBLgglqpD8fF7PFjtIb8Y8pGofecboC1i0ShaAPrfIRsQbOD1z5PC3/Hrcs Fjl9BZzs2BQdlGM7cUzBvx8B5T4eTVo4UIo7sT2U= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 03/15] bnxt_en: handle CRASH_NO_MASTER during bnxt_open() Date: Mon, 25 Jan 2021 02:08:09 -0500 Message-Id: <1611558501-11022-4-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Edwin Peer Add missing support for handling NO_MASTER crashes while ports are administratively down (ifdown). On some SoC platforms, the driver needs to assist the firmware to recover from a crash via OP-TEE. This is performed in a similar fashion to what is done during driver probe. Signed-off-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 67 +++++++++++++---------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 5daef6801512..c091a1023188 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -9337,6 +9337,37 @@ static int bnxt_hwrm_shutdown_link(struct bnxt *bp) static int bnxt_fw_init_one(struct bnxt *bp); +static int bnxt_fw_reset_via_optee(struct bnxt *bp) +{ +#ifdef CONFIG_TEE_BNXT_FW + int rc = tee_bnxt_fw_load(); + + if (rc) + netdev_err(bp->dev, "Failed FW reset via OP-TEE, rc=%d\n", rc); + + return rc; +#else + netdev_err(bp->dev, "OP-TEE not supported\n"); + return -ENODEV; +#endif +} + +static int bnxt_try_recover_fw(struct bnxt *bp) +{ + if (bp->fw_health && bp->fw_health->status_reliable) { + u32 sts = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG); + + netdev_err(bp->dev, "Firmware not responding, status: 0x%x\n", + sts); + if (sts & FW_STATUS_REG_CRASHED_NO_MASTER) { + netdev_warn(bp->dev, "Firmware recover via OP-TEE requested\n"); + return bnxt_fw_reset_via_optee(bp); + } + } + + return -ENODEV; +} + static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) { struct hwrm_func_drv_if_change_output *resp = bp->hwrm_cmd_resp_addr; @@ -9356,6 +9387,10 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) if (!rc) flags = le32_to_cpu(resp->flags); mutex_unlock(&bp->hwrm_cmd_lock); + if (rc && up) { + rc = bnxt_try_recover_fw(bp); + fw_reset = true; + } if (rc) return rc; @@ -11183,21 +11218,6 @@ static void bnxt_init_dflt_coal(struct bnxt *bp) bp->stats_coal_ticks = BNXT_DEF_STATS_COAL_TICKS; } -static int bnxt_fw_reset_via_optee(struct bnxt *bp) -{ -#ifdef CONFIG_TEE_BNXT_FW - int rc = tee_bnxt_fw_load(); - - if (rc) - netdev_err(bp->dev, "Failed FW reset via OP-TEE, rc=%d\n", rc); - - return rc; -#else - netdev_err(bp->dev, "OP-TEE not supported\n"); - return -ENODEV; -#endif -} - static int bnxt_fw_init_one_p1(struct bnxt *bp) { int rc; @@ -11206,19 +11226,10 @@ static int bnxt_fw_init_one_p1(struct bnxt *bp) rc = bnxt_hwrm_ver_get(bp); bnxt_try_map_fw_health_reg(bp); if (rc) { - if (bp->fw_health && bp->fw_health->status_reliable) { - u32 sts = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG); - - netdev_err(bp->dev, - "Firmware not responding, status: 0x%x\n", - sts); - if (sts & FW_STATUS_REG_CRASHED_NO_MASTER) { - netdev_warn(bp->dev, "Firmware recover via OP-TEE requested\n"); - rc = bnxt_fw_reset_via_optee(bp); - if (!rc) - rc = bnxt_hwrm_ver_get(bp); - } - } + rc = bnxt_try_recover_fw(bp); + if (rc) + return rc; + rc = bnxt_hwrm_ver_get(bp); if (rc) return rc; } From patchwork Mon Jan 25 07:08:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EF05C433DB for ; Mon, 25 Jan 2021 07:25:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA79B224F9 for ; Mon, 25 Jan 2021 07:25:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727025AbhAYHYH (ORCPT ); Mon, 25 Jan 2021 02:24:07 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33744 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727246AbhAYHWD (ORCPT ); Mon, 25 Jan 2021 02:22:03 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 0A5F380DC; Sun, 24 Jan 2021 23:08:23 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 0A5F380DC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558503; bh=xPmMZajMlx3+8nw/gIUg2y17dg/AfR1xFH/XzNqReyk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=johBacmnoH35HW35jQcbzTHpzj3KnJIMAhZzInZGpm/ITvwIAPpiJncz4TDZtninH ntIH6U2nptvP0C7ue5gtplsF6OoTkbl2dJ8s8wmr0PqeZ8JpT5cGVYrT02UTSJb5Cj jZ1UyQbTfcqvRbBuLBIoG7lQnRLKIj6N3VRs4maM= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 06/15] bnxt_en: Add an upper bound for all firmware command timeouts. Date: Mon, 25 Jan 2021 02:08:12 -0500 Message-Id: <1611558501-11022-7-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Vasundhara Volam The timeout period for firmware messages is passed to the driver from the firmware in the response of the first command. This timeout period is multiplied by a factor for certain long running commands such as NVRAM commands. In some cases, the timeout period can become really long and it can cause hung task warnings if firmware has crashed or is not responding. To avoid such long delays, cap all firmware commands to a max timeout value of 40 seconds. Reviewed-by: Edwin Peer Signed-off-by: Vasundhara Volam Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 ++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 2fb9873e0162..c06c5f81f087 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -4425,6 +4425,8 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len, if (!timeout) timeout = DFLT_HWRM_CMD_TIMEOUT; + /* Limit timeout to an upper limit */ + timeout = min(timeout, HWRM_CMD_MAX_TIMEOUT); /* convert timeout to usec */ timeout *= 1000; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 867b1d3a134e..cbb338baab07 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -656,6 +656,7 @@ struct nqe_cn { #define BNXT_HWRM_MAX_REQ_LEN (bp->hwrm_max_req_len) #define BNXT_HWRM_SHORT_REQ_LEN sizeof(struct hwrm_short_input) #define DFLT_HWRM_CMD_TIMEOUT 500 +#define HWRM_CMD_MAX_TIMEOUT 40000 #define SHORT_HWRM_CMD_TIMEOUT 20 #define HWRM_CMD_TIMEOUT (bp->hwrm_cmd_timeout) #define HWRM_RESET_TIMEOUT ((HWRM_CMD_TIMEOUT) * 4) From patchwork Mon Jan 25 07:08:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EBA7C433E0 for ; Mon, 25 Jan 2021 07:30:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B71FA22ADC for ; Mon, 25 Jan 2021 07:30:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726714AbhAYHaF (ORCPT ); Mon, 25 Jan 2021 02:30:05 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33756 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727234AbhAYHWf (ORCPT ); Mon, 25 Jan 2021 02:22:35 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 5359080E2; Sun, 24 Jan 2021 23:08:23 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 5359080E2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558503; bh=hX1ehjN/bcERG+Clw4VFk1IdjWQOc/g+BIBciaBagjo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NfS6W2vuk+AWAVZxC1LQUfPFZhP3/964BqPEeMV5GwQ8lgT7UPJvWQxo5ZK4ChwQ2 zqMgsLGeGTdwn5u8eVeUb14JAQvHBTI+ranq/0yU8XQ5ebVTbhQ44zvWfIB1oMLsUZ hrlK5BKjKINXiggXW5eIT3Ky3d2CKWQAXjiLkFB0= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 08/15] bnxt_en: attempt to reinitialize after aborted reset Date: Mon, 25 Jan 2021 02:08:14 -0500 Message-Id: <1611558501-11022-9-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Edwin Peer Drawing a hard line on aborted resets prevents a NIC open in some scenarios that may otherwise be recoverable. For example, if a firmware recovery happened while a PF was down and an attempt was made to bring up an associated VF in this state, then it was impossible to ever bring up this VF without a rebind or reload of its driver. Attempt to reinitialize the firmware when an aborted reset (or failed init after a reset) is discovered during open - it may succeed. Also take care to allow the user to retry opening the NIC even after an aborted reset. Signed-off-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 29 +++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index c8c25f7644ae..7f30a9fee0c8 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -9768,6 +9768,25 @@ static void bnxt_preset_reg_win(struct bnxt *bp) static int bnxt_init_dflt_ring_mode(struct bnxt *bp); +static int bnxt_reinit_after_abort(struct bnxt *bp) +{ + int rc; + + if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) + return -EBUSY; + + rc = bnxt_fw_init_one(bp); + if (!rc) { + bnxt_clear_int_mode(bp); + rc = bnxt_init_int_mode(bp); + if (!rc) { + clear_bit(BNXT_STATE_ABORT_ERR, &bp->state); + set_bit(BNXT_STATE_FW_RESET_DET, &bp->state); + } + } + return rc; +} + static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init) { int rc = 0; @@ -9926,8 +9945,14 @@ static int bnxt_open(struct net_device *dev) int rc; if (test_bit(BNXT_STATE_ABORT_ERR, &bp->state)) { - netdev_err(bp->dev, "A previous firmware reset did not complete, aborting\n"); - return -ENODEV; + rc = bnxt_reinit_after_abort(bp); + if (rc) { + if (rc == -EBUSY) + netdev_err(bp->dev, "A previous firmware reset has not completed, aborting\n"); + else + netdev_err(bp->dev, "Failed to reinitialize after aborted firmware reset\n"); + return -ENODEV; + } } rc = bnxt_hwrm_if_change(bp, true); From patchwork Mon Jan 25 07:08:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C746AC433DB for ; Mon, 25 Jan 2021 07:53:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 78231207B6 for ; Mon, 25 Jan 2021 07:53:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727313AbhAYHnX (ORCPT ); Mon, 25 Jan 2021 02:43:23 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33572 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727214AbhAYHUk (ORCPT ); Mon, 25 Jan 2021 02:20:40 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 9C0D980F0; Sun, 24 Jan 2021 23:08:23 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 9C0D980F0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558503; bh=UjAp/KEeiD2Pkc7/tKQbUFdrJ7OM+R/kmkRk30knHbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=egsgpRvpm5xzMb5Xcg9Se/6NFWsyASGawqBBNI+wK6J8u4/Jc79mWgOJwITZJViUf o+84waMHoP9c+2EXrGSL0s9rzBCocIjYoMnvEbVWNlRC4aINce7CzuiYPCU15tT6Xi 4JhkyX+XmuuQ1GrP82V6Jl8GhSi5AgqUzgcYJ3E4= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 10/15] bnxt_en: Add bnxt_fw_reset_timeout() helper. Date: Mon, 25 Jan 2021 02:08:16 -0500 Message-Id: <1611558501-11022-11-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This code to check if we have reached the maximum wait time after firmware reset is used multiple times. Add a helper function to do this. Reviewed-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index c35a5d497c1e..98caac9fbdee 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -11503,6 +11503,12 @@ static void bnxt_reset_all(struct bnxt *bp) bp->fw_reset_timestamp = jiffies; } +static bool bnxt_fw_reset_timeout(struct bnxt *bp) +{ + return time_after(jiffies, bp->fw_reset_timestamp + + (bp->fw_reset_max_dsecs * HZ / 10)); +} + static void bnxt_fw_reset_task(struct work_struct *work) { struct bnxt *bp = container_of(work, struct bnxt, fw_reset_task.work); @@ -11524,8 +11530,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) bp->fw_reset_timestamp)); goto fw_reset_abort; } else if (n > 0) { - if (time_after(jiffies, bp->fw_reset_timestamp + - (bp->fw_reset_max_dsecs * HZ / 10))) { + if (bnxt_fw_reset_timeout(bp)) { clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state); bp->fw_reset_state = 0; netdev_err(bp->dev, "Firmware reset aborted, bnxt_get_registered_vfs() returns %d\n", @@ -11554,8 +11559,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) val = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG); if (!(val & BNXT_FW_STATUS_SHUTDOWN) && - !time_after(jiffies, bp->fw_reset_timestamp + - (bp->fw_reset_max_dsecs * HZ / 10))) { + !bnxt_fw_reset_timeout(bp)) { bnxt_queue_fw_reset_work(bp, HZ / 5); return; } @@ -11597,8 +11601,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) bp->hwrm_cmd_timeout = SHORT_HWRM_CMD_TIMEOUT; rc = __bnxt_hwrm_ver_get(bp, true); if (rc) { - if (time_after(jiffies, bp->fw_reset_timestamp + - (bp->fw_reset_max_dsecs * HZ / 10))) { + if (bnxt_fw_reset_timeout(bp)) { netdev_err(bp->dev, "Firmware reset aborted\n"); goto fw_reset_abort_status; } From patchwork Mon Jan 25 07:08:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6050C433DB for ; Mon, 25 Jan 2021 07:35:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 702B422ADC for ; Mon, 25 Jan 2021 07:35:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727278AbhAYHbs (ORCPT ); Mon, 25 Jan 2021 02:31:48 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33760 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727248AbhAYHWf (ORCPT ); Mon, 25 Jan 2021 02:22:35 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id C083280F1; Sun, 24 Jan 2021 23:08:23 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com C083280F1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558503; bh=ElocyARdvFBM3y0xyz8B6lfvZALUbYxHCcaWqeLZSRE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lmBV8/bBXnwZ0TlzMMCIqUqqbF6FBTCeflsCqugDlB1mxZlNjxKyWSbyIptHk+rOk Qh6b1Ngiaq0RSNR7lnBJsbn9KstotIoq+NOLJsbZCYzLM3HM4qjPNw7M1rhlsU7ROl 3sPU932jZ5xaLI4srbd7QY+wAVf+evNq0J+dINS8= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 11/15] bnxt_en: Add a new BNXT_STATE_NAPI_DISABLED flag to keep track of NAPI state. Date: Mon, 25 Jan 2021 02:08:17 -0500 Message-Id: <1611558501-11022-12-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Up until now, we don't need to keep track of this state because NAPI is always enabled once and disabled once during bring up and shutdown. For better error recovery in subsequent patches, we want to quiesce the device earlier during fatal error conditions. The normal shutdown sequence will disable NAPI again and the flag will prevent disabling NAPI twice. Reviewed-by: Pavan Chebbi Reviewed-by: Andy Gospodarek Reviewed-by: Edwin Peer Reviewed-by: Vasundhara Volam Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 98caac9fbdee..83846b50042a 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -8836,7 +8836,8 @@ static void bnxt_disable_napi(struct bnxt *bp) { int i; - if (!bp->bnapi) + if (!bp->bnapi || + test_and_set_bit(BNXT_STATE_NAPI_DISABLED, &bp->state)) return; for (i = 0; i < bp->cp_nr_rings; i++) { @@ -8853,6 +8854,7 @@ static void bnxt_enable_napi(struct bnxt *bp) { int i; + clear_bit(BNXT_STATE_NAPI_DISABLED, &bp->state); for (i = 0; i < bp->cp_nr_rings; i++) { struct bnxt_napi *bnapi = bp->bnapi[i]; struct bnxt_cp_ring_info *cpr; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index bd36f00ef28c..4ef6888acdc6 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1809,6 +1809,7 @@ struct bnxt { #define BNXT_STATE_FW_FATAL_COND 6 #define BNXT_STATE_DRV_REGISTERED 7 #define BNXT_STATE_PCI_CHANNEL_IO_FROZEN 8 +#define BNXT_STATE_NAPI_DISABLED 9 #define BNXT_NO_FW_ACCESS(bp) \ (test_bit(BNXT_STATE_FW_FATAL_COND, &(bp)->state) || \ From patchwork Mon Jan 25 07:08:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 370482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47EB8C433DB for ; Mon, 25 Jan 2021 07:45:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA82422C9C for ; Mon, 25 Jan 2021 07:45:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727327AbhAYHnb (ORCPT ); Mon, 25 Jan 2021 02:43:31 -0500 Received: from lpdvacalvio01.broadcom.com ([192.19.229.182]:33584 "EHLO relay.smtp-ext.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727221AbhAYHUk (ORCPT ); Mon, 25 Jan 2021 02:20:40 -0500 Received: from localhost.swdvt.lab.broadcom.net (dhcp-10-13-253-90.swdvt.lab.broadcom.net [10.13.253.90]) by relay.smtp-ext.broadcom.com (Postfix) with ESMTP id 1B0F080F5; Sun, 24 Jan 2021 23:08:24 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 relay.smtp-ext.broadcom.com 1B0F080F5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1611558504; bh=K40O4k5HjQw55d1yHGE6XCcdG+22ODoapSiZ/o3GHUE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AJVLXod3l5/dizt9ZC4BT+MRTHLOocDpsJcASrO/y/4pVqjBDEnJGLdaSfH/wD807 Nbgmb1mX+75Dn2J+TLPZVc8CfWyDpT6W4mUPynEAst3js0Tah+2Au4H5ujvYcgEyYh jSnNH90fJWdjc/g598EGaz4mkuAcYo9vU4bKtr7k= From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com Subject: [PATCH net-next 13/15] bnxt_en: Improve firmware fatal error shutdown sequence. Date: Mon, 25 Jan 2021 02:08:19 -0500 Message-Id: <1611558501-11022-14-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> References: <1611558501-11022-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In the event of a fatal firmware error, firmware will notify the host and then it will proceed to do core reset when it sees that all functions have disabled Bus Master. To prevent Master Aborts and other hard errors, we need to quiesce all activities in addition to disabling Bus Master before the chip goes into core reset. Reviewed-by: Edwin Peer Reviewed-by: Vasundhara Volam Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 80dab4e622ab..e7abb3b7ed68 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -10905,11 +10905,18 @@ static void bnxt_rx_ring_reset(struct bnxt *bp) static void bnxt_fw_reset_close(struct bnxt *bp) { bnxt_ulp_stop(bp); - /* When firmware is fatal state, disable PCI device to prevent - * any potential bad DMAs before freeing kernel memory. + /* When firmware is in fatal state, quiesce device and disable + * bus master to prevent any potential bad DMAs before freeing + * kernel memory. */ - if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) + if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) { + bnxt_tx_disable(bp); + bnxt_disable_napi(bp); + bnxt_disable_int_sync(bp); + bnxt_free_irq(bp); + bnxt_clear_int_mode(bp); pci_disable_device(bp->pdev); + } __bnxt_close_nic(bp, true, false); bnxt_clear_int_mode(bp); bnxt_hwrm_func_drv_unrgtr(bp);