From patchwork Sun Oct 4 19:22:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 289009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MIME_HEADER_CTYPE_ONLY, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, T_TVD_MIME_NO_HEADERS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 383EFC46466 for ; Sun, 4 Oct 2020 19:23:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E0298205ED for ; Sun, 4 Oct 2020 19:23:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="KIimXRHE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726731AbgJDTXk (ORCPT ); Sun, 4 Oct 2020 15:23:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726294AbgJDTXd (ORCPT ); Sun, 4 Oct 2020 15:23:33 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 640A7C0613D0 for ; Sun, 4 Oct 2020 12:23:33 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id t14so4334255pgl.10 for ; Sun, 04 Oct 2020 12:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HUmvziSH10+NSBA4K69lu3ErQwLVzy6PHh7gve51sqI=; b=KIimXRHEAbj/dCXmHsHKEvtmtYvHmXnE/N0mVDd4x2gQHPo397byeSf45Zpni/VLm+ jt2RCrjunk0peWE6eA4JfcKR8JLe8HKOA8ZEQoVUAFZrpheIyTXUIUpZNYXDfgIS9KMF j/gXoepD+nYXHsSG9M54ptEqbmhQ5tkAhkkGY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HUmvziSH10+NSBA4K69lu3ErQwLVzy6PHh7gve51sqI=; b=Kjlz7ueSo2PFI2J3dQpHzXOsnN/CjZf13LrjxpjoPNGcHi4/I4pofBmq08ZUjVi3UB ePr0DNPCJc0C8gMWr9wInLXWhkOoUJUhb0cuqH9HawIPNg+ZzkBQ/Sive1QOCy0IQt+Z v9k6woXefT6cA5Gov3STK63gEUJ/BminVSsaY/ulT8YI7N0CRnPH4mpgO1XTtGxhnQY3 gncALPiLXKPfXKcqITo1bf9hU3XowOfGG4io/Rg6FXL710CSswHMMCSsvRDKTqNFrE4/ ywYKLmv6+HftBIYzJdq63QBawXT9H63R9xWkmHo5/6y1MCh2YX82YBObXZcaVrwu8wP5 xpeQ== X-Gm-Message-State: AOAM533HU98Y78s4atcobPJbab+sAozhjp9HSHh1shnUJskI7p+qyJeC uLykyjUBqu3YqngaF2MQ48u2jVtV+/0mHQ== X-Google-Smtp-Source: ABdhPJz1YI5f39B5TnYL42MQk3dvCLx0fH0U7Fm8x0IxkFBWPHTCBaadK74ISWyU5bRVeAGyEGZ65w== X-Received: by 2002:a65:6086:: with SMTP id t6mr11077390pgu.146.1601839412622; Sun, 04 Oct 2020 12:23:32 -0700 (PDT) Received: from localhost.swdvt.lab.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id 138sm9824234pfu.180.2020.10.04.12.23.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 04 Oct 2020 12:23:31 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, kuba@kernel.org, gospo@broadcom.com, Edwin Peer Subject: [PATCH net-next 08/11] bnxt_en: Implement RX ring reset in response to buffer errors. Date: Sun, 4 Oct 2020 15:22:58 -0400 Message-Id: <1601839381-10446-9-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601839381-10446-1-git-send-email-michael.chan@broadcom.com> References: <1601839381-10446-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On some older chips, it is necessary to do a reset when we get buffer errors associated with an RX ring. These buffer errors may become frequent if the RX ring underruns under heavy traffic. The current code does a global reset of all reasources when this happens. This works but creates a big disruption of all rings when one RX ring is having problem. This patch implements a localized RX ring reset of just the RX ring having the issue. All other rings including all TX rings will not be affected by this single RX ring reset. Only the older chips prior to the P5 class supports this reset. Because it is not a global reset, packets may still be arriving while we are calling firmware to reset that ring. We need to be sure that we don't post any buffers during this time while the ring is undergoing reset. After firmware completes successfully, the ring will be in the reset state with no buffers and we can start filling it with new buffers and posting them. Reviewed-by: Pavan Chebbi Signed-off-by: Edwin Peer Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 79 +++++++++++++++++++++-- 1 file changed, 75 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index ef0267060a46..31877fb4ddd9 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -1172,7 +1172,10 @@ static void bnxt_sched_reset(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) { if (!rxr->bnapi->in_reset) { rxr->bnapi->in_reset = true; - set_bit(BNXT_RESET_TASK_SP_EVENT, &bp->sp_event); + if (bp->flags & BNXT_FLAG_CHIP_P5) + set_bit(BNXT_RESET_TASK_SP_EVENT, &bp->sp_event); + else + set_bit(BNXT_RST_RING_SP_EVENT, &bp->sp_event); bnxt_queue_sp_work(bp); } rxr->rx_next_cons = 0xffff; @@ -1773,8 +1776,8 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, if (rx_err & RX_CMPL_ERRORS_BUFFER_ERROR_MASK) { bnapi->cp_ring.sw_stats.rx.rx_buf_errors++; if (!(bp->flags & BNXT_FLAG_CHIP_P5)) { - netdev_warn(bp->dev, "RX buffer error %x\n", - rx_err); + netdev_warn_once(bp->dev, "RX buffer error %x\n", + rx_err); bnxt_sched_reset(bp, rxr); } } @@ -2250,7 +2253,7 @@ static void __bnxt_poll_work_done(struct bnxt *bp, struct bnxt_napi *bnapi) bnapi->tx_pkts = 0; } - if (bnapi->events & BNXT_RX_EVENT) { + if ((bnapi->events & BNXT_RX_EVENT) && !(bnapi->in_reset)) { struct bnxt_rx_ring_info *rxr = bnapi->rx_ring; if (bnapi->events & BNXT_AGG_EVENT) @@ -10510,6 +10513,23 @@ static void bnxt_dbg_dump_states(struct bnxt *bp) } } +static int bnxt_hwrm_rx_ring_reset(struct bnxt *bp, int ring_nr) +{ + struct bnxt_rx_ring_info *rxr = &bp->rx_ring[ring_nr]; + struct hwrm_ring_reset_input req = {0}; + struct bnxt_napi *bnapi = rxr->bnapi; + struct bnxt_cp_ring_info *cpr; + u16 cp_ring_id; + + cpr = &bnapi->cp_ring; + cp_ring_id = cpr->cp_ring_struct.fw_ring_id; + bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_RING_RESET, cp_ring_id, -1); + req.ring_type = RING_RESET_REQ_RING_TYPE_RX_RING_GRP; + req.ring_id = cpu_to_le16(bp->grp_info[bnapi->index].fw_grp_id); + return hwrm_send_message_silent(bp, &req, sizeof(req), + HWRM_CMD_TIMEOUT); +} + static void bnxt_reset_task(struct bnxt *bp, bool silent) { if (!silent) @@ -10645,6 +10665,54 @@ static void bnxt_reset(struct bnxt *bp, bool silent) bnxt_rtnl_unlock_sp(bp); } +/* Only called from bnxt_sp_task() */ +static void bnxt_rx_ring_reset(struct bnxt *bp) +{ + int i; + + bnxt_rtnl_lock_sp(bp); + if (!test_bit(BNXT_STATE_OPEN, &bp->state)) { + bnxt_rtnl_unlock_sp(bp); + return; + } + /* Disable and flush TPA before resetting the RX ring */ + if (bp->flags & BNXT_FLAG_TPA) + bnxt_set_tpa(bp, false); + for (i = 0; i < bp->rx_nr_rings; i++) { + struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i]; + struct bnxt_cp_ring_info *cpr; + int rc; + + if (!rxr->bnapi->in_reset) + continue; + + rc = bnxt_hwrm_rx_ring_reset(bp, i); + if (rc) { + if (rc == -EINVAL || rc == -EOPNOTSUPP) + netdev_info_once(bp->dev, "RX ring reset not supported by firmware, falling back to global reset\n"); + else + netdev_warn(bp->dev, "RX ring reset failed, rc = %d, falling back to global reset\n", + rc); + bnxt_reset_task(bp, false); + break; + } + bnxt_free_one_rx_ring_skbs(bp, i); + rxr->rx_prod = 0; + rxr->rx_agg_prod = 0; + rxr->rx_sw_agg_prod = 0; + rxr->rx_next_cons = 0; + rxr->bnapi->in_reset = false; + bnxt_alloc_one_rx_ring(bp, i); + cpr = &rxr->bnapi->cp_ring; + if (bp->flags & BNXT_FLAG_AGG_RINGS) + bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); + } + if (bp->flags & BNXT_FLAG_TPA) + bnxt_set_tpa(bp, true); + bnxt_rtnl_unlock_sp(bp); +} + static void bnxt_fw_reset_close(struct bnxt *bp) { bnxt_ulp_stop(bp); @@ -10933,6 +11001,9 @@ static void bnxt_sp_task(struct work_struct *work) if (test_and_clear_bit(BNXT_RESET_TASK_SILENT_SP_EVENT, &bp->sp_event)) bnxt_reset(bp, true); + if (test_and_clear_bit(BNXT_RST_RING_SP_EVENT, &bp->sp_event)) + bnxt_rx_ring_reset(bp); + if (test_and_clear_bit(BNXT_FW_RESET_NOTIFY_SP_EVENT, &bp->sp_event)) bnxt_devlink_health_report(bp, BNXT_FW_RESET_NOTIFY_SP_EVENT);