From patchwork Fri Mar 13 09:42:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 222566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4593C10DCE for ; Fri, 13 Mar 2020 09:43:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AF04820749 for ; Fri, 13 Mar 2020 09:43:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BHFVBYcb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726554AbgCMJnM (ORCPT ); Fri, 13 Mar 2020 05:43:12 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:32954 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726216AbgCMJnL (ORCPT ); Fri, 13 Mar 2020 05:43:11 -0400 Received: by mail-pg1-f195.google.com with SMTP id m5so4692869pgg.0 for ; Fri, 13 Mar 2020 02:43:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/6xVhu5Ov+HLrAKUtgd9aJ/JgDuuCC11WFFCuDiDbhI=; b=BHFVBYcbsiGoZkkzatIOhei44zaItw7faNYaQ+z+JNclIu/UhStREdRoSHerbgbPXw TbL4XRHUWw7FxBYRdf2vh+73kU2wcLh7vNRp443CA0mYdb5PweGwAO5xpXgimQQdChB1 ZUJwF1ksOCaEPAy+UVpxd6vrj19CTPnYr4GXystLiy4vhswtBo8eANbhTjiXbRnECkfW HhjhDCDDYG87zUnWUizxxQO5WEClvSlAwVGXBX8CoN9xiTpuPO1FZSZSE8L2/AECM33w LxHybdYYH2olQnUPwW2MeU8mqEwh6Fl9grh3rFCsiJQMxNQLxbJ7WKku6zdTaDjwFixK 2WFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/6xVhu5Ov+HLrAKUtgd9aJ/JgDuuCC11WFFCuDiDbhI=; b=TKbqDy5YuYJzymRlVMRiniY6AAk9Ri5BRi/rIkIouHTaZFJW1hzDK83LcYPeg2xJGt v86eVeabYb/OrDlQNjXQKCg7HEKpQqYd7clZxo96KKF9HHXABTfhyvE4MJYsS0IcquDz dkt/Q0M8oTqekbHqwQEFjJRwPj3ImPNZ1Sfau0FTv5WO6BtnZ2DoHko0oXmkSKNll6KS f6+PozxZzz7qTMZXMX6s4NwjMy9b8gF287u2Os9LZ4DnneO/+q83mWjs/MqR/gKe4ggX NNlJ5zWW0fCZUvYnuLDNcaEvCpzNlghpbM9rO4Mp1D4L+PaH9S8qyBsPlr3K5hkvbeuc Dxxw== X-Gm-Message-State: ANhLgQ25Q5zldoV+ww3cobdJhYFnjCoA/aCj7GGUDfczmfk63qQBOEk/ E/Pt3Uty2HsVoADCLhSUt6VdSYtJf3U= X-Google-Smtp-Source: ADFU+vv4rwCX+DUjTbiDPLn9DwkhTf44pvTLxoUpqpkGqDSIZvhi5TIg9TxTurb/xSiHjsL/fsgmhA== X-Received: by 2002:a63:3103:: with SMTP id x3mr12068732pgx.209.1584092589618; Fri, 13 Mar 2020 02:43:09 -0700 (PDT) Received: from machine421.marvell.com ([115.113.156.2]) by smtp.googlemail.com with ESMTPSA id v67sm13896386pfc.120.2020.03.13.02.43.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 Mar 2020 02:43:08 -0700 (PDT) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, leon@kernel.org, Geetha sowjanya , Sunil Goutham Subject: [PATCH v2 net-next 2/7] octeontx2-pf: Handle VF function level reset Date: Fri, 13 Mar 2020 15:12:41 +0530 Message-Id: <1584092566-4793-3-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> References: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Geetha sowjanya When FLR is initiated for a VF (PCI function level reset), the parent PF gets a interrupt. PF then sends a message to admin function (AF), which then cleanups all resources attached to that VF. Also handled IRQs triggered when master enable bit is cleared or set for VFs. This handler just clears the transaction pending ie TRPEND bit. Signed-off-by: Geetha sowjanya Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 7 + .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 234 ++++++++++++++++++++- 2 files changed, 240 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 74439e1..c0a9693 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -191,6 +191,11 @@ struct otx2_hw { u64 cgx_tx_stats[CGX_TX_STATS_COUNT]; }; +struct flr_work { + struct work_struct work; + struct otx2_nic *pf; +}; + struct refill_work { struct delayed_work pool_refill_work; struct otx2_nic *pf; @@ -226,6 +231,8 @@ struct otx2_nic { u64 reset_count; struct work_struct reset_task; + struct workqueue_struct *flr_wq; + struct flr_work *flr_wrk; struct refill_work *refill_wrk; /* Ethtool stuff */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 967ef7b..bf6e2529 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -61,6 +61,224 @@ static int otx2_change_mtu(struct net_device *netdev, int new_mtu) return err; } +static void otx2_disable_flr_me_intr(struct otx2_nic *pf) +{ + int irq, vfs = pf->total_vfs; + + /* Disable VFs ME interrupts */ + otx2_write64(pf, RVU_PF_VFME_INT_ENA_W1CX(0), INTR_MASK(vfs)); + irq = pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFME0); + free_irq(irq, pf); + + /* Disable VFs FLR interrupts */ + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1CX(0), INTR_MASK(vfs)); + irq = pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFFLR0); + free_irq(irq, pf); + + if (vfs <= 64) + return; + + otx2_write64(pf, RVU_PF_VFME_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + irq = pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFME1); + free_irq(irq, pf); + + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + irq = pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFFLR1); + free_irq(irq, pf); +} + +static void otx2_flr_wq_destroy(struct otx2_nic *pf) +{ + if (!pf->flr_wq) + return; + flush_workqueue(pf->flr_wq); + destroy_workqueue(pf->flr_wq); + pf->flr_wq = NULL; + devm_kfree(pf->dev, pf->flr_wrk); +} + +static void otx2_flr_handler(struct work_struct *work) +{ + struct flr_work *flrwork = container_of(work, struct flr_work, work); + struct otx2_nic *pf = flrwork->pf; + struct msg_req *req; + int vf, reg = 0; + + vf = flrwork - pf->flr_wrk; + + otx2_mbox_lock(&pf->mbox); + req = otx2_mbox_alloc_msg_vf_flr(&pf->mbox); + if (!req) { + otx2_mbox_unlock(&pf->mbox); + return; + } + req->hdr.pcifunc &= RVU_PFVF_FUNC_MASK; + req->hdr.pcifunc |= (vf + 1) & RVU_PFVF_FUNC_MASK; + + if (!otx2_sync_mbox_msg(&pf->mbox)) { + if (vf >= 64) { + reg = 1; + vf = vf - 64; + } + /* clear transcation pending bit */ + otx2_write64(pf, RVU_PF_VFTRPENDX(reg), BIT_ULL(vf)); + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1SX(reg), BIT_ULL(vf)); + } + + otx2_mbox_unlock(&pf->mbox); +} + +static irqreturn_t otx2_pf_flr_intr_handler(int irq, void *pf_irq) +{ + struct otx2_nic *pf = (struct otx2_nic *)pf_irq; + int reg, dev, vf, start_vf, num_reg = 1; + u64 intr; + + if (pf->total_vfs > 64) + num_reg = 2; + + for (reg = 0; reg < num_reg; reg++) { + intr = otx2_read64(pf, RVU_PF_VFFLR_INTX(reg)); + if (!intr) + continue; + start_vf = 64 * reg; + for (vf = 0; vf < 64; vf++) { + if (!(intr & BIT_ULL(vf))) + continue; + dev = vf + start_vf; + queue_work(pf->flr_wq, &pf->flr_wrk[dev].work); + /* Clear interrupt */ + otx2_write64(pf, RVU_PF_VFFLR_INTX(reg), BIT_ULL(vf)); + /* Disable the interrupt */ + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1CX(reg), + BIT_ULL(vf)); + } + } + return IRQ_HANDLED; +} + +static irqreturn_t otx2_pf_me_intr_handler(int irq, void *pf_irq) +{ + struct otx2_nic *pf = (struct otx2_nic *)pf_irq; + int vf, reg, num_reg = 1; + u64 intr; + + if (pf->total_vfs > 64) + num_reg = 2; + + for (reg = 0; reg < num_reg; reg++) { + intr = otx2_read64(pf, RVU_PF_VFME_INTX(reg)); + if (!intr) + continue; + for (vf = 0; vf < 64; vf++) { + if (!(intr & BIT_ULL(vf))) + continue; + /* clear trpend bit */ + otx2_write64(pf, RVU_PF_VFTRPENDX(reg), BIT_ULL(vf)); + /* clear interrupt */ + otx2_write64(pf, RVU_PF_VFME_INTX(reg), BIT_ULL(vf)); + } + } + return IRQ_HANDLED; +} + +static int otx2_register_flr_me_intr(struct otx2_nic *pf, int numvfs) +{ + struct otx2_hw *hw = &pf->hw; + char *irq_name; + int ret; + + /* Register ME interrupt handler*/ + irq_name = &hw->irq_name[RVU_PF_INT_VEC_VFME0 * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d_ME0", rvu_get_pf(pf->pcifunc)); + ret = request_irq(pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFME0), + otx2_pf_me_intr_handler, 0, irq_name, pf); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for ME0\n"); + } + + /* Register FLR interrupt handler */ + irq_name = &hw->irq_name[RVU_PF_INT_VEC_VFFLR0 * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d_FLR0", rvu_get_pf(pf->pcifunc)); + ret = request_irq(pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_VFFLR0), + otx2_pf_flr_intr_handler, 0, irq_name, pf); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for FLR0\n"); + return ret; + } + + if (numvfs > 64) { + irq_name = &hw->irq_name[RVU_PF_INT_VEC_VFME1 * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d_ME1", + rvu_get_pf(pf->pcifunc)); + ret = request_irq(pci_irq_vector + (pf->pdev, RVU_PF_INT_VEC_VFME1), + otx2_pf_me_intr_handler, 0, irq_name, pf); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for ME1\n"); + } + irq_name = &hw->irq_name[RVU_PF_INT_VEC_VFFLR1 * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d_FLR1", + rvu_get_pf(pf->pcifunc)); + ret = request_irq(pci_irq_vector + (pf->pdev, RVU_PF_INT_VEC_VFFLR1), + otx2_pf_flr_intr_handler, 0, irq_name, pf); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for FLR1\n"); + return ret; + } + } + + /* Enable ME interrupt for all VFs*/ + otx2_write64(pf, RVU_PF_VFME_INTX(0), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_PF_VFME_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + + /* Enable FLR interrupt for all VFs*/ + otx2_write64(pf, RVU_PF_VFFLR_INTX(0), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + + if (numvfs > 64) { + numvfs -= 64; + + otx2_write64(pf, RVU_PF_VFME_INTX(1), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_PF_VFME_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + + otx2_write64(pf, RVU_PF_VFFLR_INTX(1), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_PF_VFFLR_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + } + return 0; +} + +static int otx2_pf_flr_init(struct otx2_nic *pf, int num_vfs) +{ + int vf; + + pf->flr_wq = alloc_workqueue("otx2_pf_flr_wq", + WQ_UNBOUND | WQ_HIGHPRI, 1); + if (!pf->flr_wq) + return -ENOMEM; + + pf->flr_wrk = devm_kcalloc(pf->dev, num_vfs, + sizeof(struct flr_work), GFP_KERNEL); + if (!pf->flr_wrk) { + destroy_workqueue(pf->flr_wq); + return -ENOMEM; + } + + for (vf = 0; vf < num_vfs; vf++) { + pf->flr_wrk[vf].pf = pf; + INIT_WORK(&pf->flr_wrk[vf].work, otx2_flr_handler); + } + + return 0; +} + static void otx2_queue_work(struct mbox *mw, struct workqueue_struct *mbox_wq, int first, int mdevs, u64 intr, int type) { @@ -1716,11 +1934,23 @@ static int otx2_sriov_enable(struct pci_dev *pdev, int numvfs) if (ret) goto free_mbox; - ret = pci_enable_sriov(pdev, numvfs); + ret = otx2_pf_flr_init(pf, numvfs); if (ret) goto free_intr; + ret = otx2_register_flr_me_intr(pf, numvfs); + if (ret) + goto free_flr; + + ret = pci_enable_sriov(pdev, numvfs); + if (ret) + goto free_flr_intr; + return numvfs; +free_flr_intr: + otx2_disable_flr_me_intr(pf); +free_flr: + otx2_flr_wq_destroy(pf); free_intr: otx2_disable_pfvf_mbox_intr(pf, numvfs); free_mbox: @@ -1739,6 +1969,8 @@ static int otx2_sriov_disable(struct pci_dev *pdev) pci_disable_sriov(pdev); + otx2_disable_flr_me_intr(pf); + otx2_flr_wq_destroy(pf); otx2_disable_pfvf_mbox_intr(pf, numvfs); otx2_pfvf_mbox_destroy(pf); From patchwork Fri Mar 13 09:42:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 222565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ACCFC4CECE for ; Fri, 13 Mar 2020 09:43:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 281D32073E for ; Fri, 13 Mar 2020 09:43:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Tj0W39XP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726613AbgCMJnS (ORCPT ); Fri, 13 Mar 2020 05:43:18 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:33171 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726216AbgCMJnR (ORCPT ); Fri, 13 Mar 2020 05:43:17 -0400 Received: by mail-pf1-f195.google.com with SMTP id n7so4910925pfn.0 for ; Fri, 13 Mar 2020 02:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eSXTtkK8YAITEFR45aIeIoIaX6PrkM/ZgURRctX+FYw=; b=Tj0W39XPKeCMAlKE2DJJbNA6pkqg0dbQkt0wHySA7ME03m+1K4FQt2UyXWcCjeF+nt tS7gF/cCRlPQ29mDNvNq+bVazN3Ga/XimVaixHeGQFQ+moVrxcSVxTlFJlXRwnzym9dl 1uaxMC+u+3HAHL8BYk1/YYwl7Ye4f77KyQRQGn1iqws15cuUqcZEzb20HKlHh29boh5o 2jxiD2dp/lBip5d74UK7T32adof2PFPu5/u2hYgmGFs0WMUN/DPaOiYA9cHqaMadACns 0BX0VjdMXk5HNzO6kTTDyIbQ5NJtUM8HSzeeOEYac9JoZH9GEfjOQ0aDOUbjg9VVZxjn UYTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eSXTtkK8YAITEFR45aIeIoIaX6PrkM/ZgURRctX+FYw=; b=KuU+tp1AkfQbv6BpEADR3ly5qP9V6aqGa7e22XfDxC4MW9MkQ1CpjtdbEq0nCzB2CY k6z24Plp/4vp6selaYgeKfZpIej1sCmRHY8BNQ+AB5PRiRR7hJBJPXGks9xV7ea3M0yB oV+F6oZWlINadL1jZQFffaQ66i2WaVI6ixDPU2x+3402Tjc3r+ePEqWYztFInL9moeLP X6B1AA1UYA8uzt3RSdmIHb52LtYoZsVLnh0AlYdssYMC6aGwUnMRJJI/1TYtaydLdu/Q gTKE54omUfTPOUpcVKFDWHuXTpoMXEdb9J9aPVKx6Uy0Va1ZuuphzZEKLsYr+i0PXJgA ISgw== X-Gm-Message-State: ANhLgQ0xOYHJruCl8ESUVhkKt91dUS+sdSqu8tirTFa9p3fpXXSgAfMK DRzPnNSMLcuW7YqOS/FjuzP47CbJcz4= X-Google-Smtp-Source: ADFU+vuqwB2CEXgT0OjI4/i+CT7FsJuFWB80uYXdAHe+DmlJn4lJCbdEcVIH1PqDvtYtZVfcaD8tpw== X-Received: by 2002:aa7:91ce:: with SMTP id z14mr12757621pfa.62.1584092596085; Fri, 13 Mar 2020 02:43:16 -0700 (PDT) Received: from machine421.marvell.com ([115.113.156.2]) by smtp.googlemail.com with ESMTPSA id v67sm13896386pfc.120.2020.03.13.02.43.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 Mar 2020 02:43:15 -0700 (PDT) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, leon@kernel.org, Tomasz Duszynski , Sunil Goutham Subject: [PATCH v2 net-next 4/7] octeontx2-vf: Ethtool support Date: Fri, 13 Mar 2020 15:12:43 +0530 Message-Id: <1584092566-4793-5-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> References: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tomasz Duszynski Added ethtool support for VF devices for - Driver stats, Tx/Rx perqueue stats - Set/show Rx/Tx queue count - Set/show Rx/Tx ring sizes - Set/show IRQ coalescing parameters - RSS configuration etc It's the PF which owns the interface, hence VF cannot display underlying CGX interface stats. Except for this rest ethtool support reuses PF's APIs. Signed-off-by: Tomasz Duszynski Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 + .../ethernet/marvell/octeontx2/nic/otx2_ethtool.c | 133 ++++++++++++++++++++- .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 4 + 3 files changed, 136 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index ca757b2..95b8f1e 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -631,6 +631,7 @@ void otx2_update_lmac_stats(struct otx2_nic *pfvf); int otx2_update_rq_stats(struct otx2_nic *pfvf, int qidx); int otx2_update_sq_stats(struct otx2_nic *pfvf, int qidx); void otx2_set_ethtool_ops(struct net_device *netdev); +void otx2vf_set_ethtool_ops(struct net_device *netdev); int otx2_open(struct net_device *netdev); int otx2_stop(struct net_device *netdev); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c index f450111..1751e2d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c @@ -17,6 +17,7 @@ #include "otx2_common.h" #define DRV_NAME "octeontx2-nicpf" +#define DRV_VF_NAME "octeontx2-nicvf" struct otx2_stat { char name[ETH_GSTRING_LEN]; @@ -63,14 +64,34 @@ static const unsigned int otx2_n_dev_stats = ARRAY_SIZE(otx2_dev_stats); static const unsigned int otx2_n_drv_stats = ARRAY_SIZE(otx2_drv_stats); static const unsigned int otx2_n_queue_stats = ARRAY_SIZE(otx2_queue_stats); +int __weak otx2vf_open(struct net_device *netdev) +{ + return 0; +} + +int __weak otx2vf_stop(struct net_device *netdev) +{ + return 0; +} + static void otx2_dev_open(struct net_device *netdev) { - otx2_open(netdev); + struct otx2_nic *pfvf = netdev_priv(netdev); + + if (pfvf->pcifunc & RVU_PFVF_FUNC_MASK) + otx2vf_open(netdev); + else + otx2_open(netdev); } static void otx2_dev_stop(struct net_device *netdev) { - otx2_stop(netdev); + struct otx2_nic *pfvf = netdev_priv(netdev); + + if (pfvf->pcifunc & RVU_PFVF_FUNC_MASK) + otx2vf_stop(netdev); + else + otx2_stop(netdev); } static void otx2_get_drvinfo(struct net_device *netdev, @@ -259,6 +280,9 @@ static void otx2_get_pauseparam(struct net_device *netdev, struct otx2_nic *pfvf = netdev_priv(netdev); struct cgx_pause_frm_cfg *req, *rsp; + if (is_otx2_lbkvf(pfvf->pdev)) + return; + req = otx2_mbox_alloc_msg_cgx_cfg_pause_frm(&pfvf->mbox); if (!req) return; @@ -279,6 +303,9 @@ static int otx2_set_pauseparam(struct net_device *netdev, if (pause->autoneg) return -EOPNOTSUPP; + if (is_otx2_lbkvf(pfvf->pdev)) + return -EOPNOTSUPP; + if (pause->rx_pause) pfvf->flags |= OTX2_FLAG_RX_PAUSE_ENABLED; else @@ -670,6 +697,9 @@ static u32 otx2_get_link(struct net_device *netdev) { struct otx2_nic *pfvf = netdev_priv(netdev); + /* LBK link is internal and always UP */ + if (is_otx2_lbkvf(pfvf->pdev)) + return 1; return pfvf->linfo.link_up; } @@ -701,3 +731,102 @@ void otx2_set_ethtool_ops(struct net_device *netdev) { netdev->ethtool_ops = &otx2_ethtool_ops; } + +/* VF's ethtool APIs */ +static void otx2vf_get_drvinfo(struct net_device *netdev, + struct ethtool_drvinfo *info) +{ + struct otx2_nic *vf = netdev_priv(netdev); + + strlcpy(info->driver, DRV_VF_NAME, sizeof(info->driver)); + strlcpy(info->bus_info, pci_name(vf->pdev), sizeof(info->bus_info)); +} + +static void otx2vf_get_strings(struct net_device *netdev, u32 sset, u8 *data) +{ + struct otx2_nic *vf = netdev_priv(netdev); + int stats; + + if (sset != ETH_SS_STATS) + return; + + for (stats = 0; stats < otx2_n_dev_stats; stats++) { + memcpy(data, otx2_dev_stats[stats].name, ETH_GSTRING_LEN); + data += ETH_GSTRING_LEN; + } + + for (stats = 0; stats < otx2_n_drv_stats; stats++) { + memcpy(data, otx2_drv_stats[stats].name, ETH_GSTRING_LEN); + data += ETH_GSTRING_LEN; + } + + otx2_get_qset_strings(vf, &data, 0); + + strcpy(data, "reset_count"); + data += ETH_GSTRING_LEN; +} + +static void otx2vf_get_ethtool_stats(struct net_device *netdev, + struct ethtool_stats *stats, u64 *data) +{ + struct otx2_nic *vf = netdev_priv(netdev); + int stat; + + otx2_get_dev_stats(vf); + for (stat = 0; stat < otx2_n_dev_stats; stat++) + *(data++) = ((u64 *)&vf->hw.dev_stats) + [otx2_dev_stats[stat].index]; + + for (stat = 0; stat < otx2_n_drv_stats; stat++) + *(data++) = atomic_read(&((atomic_t *)&vf->hw.drv_stats) + [otx2_drv_stats[stat].index]); + + otx2_get_qset_stats(vf, stats, &data); + *(data++) = vf->reset_count; +} + +static int otx2vf_get_sset_count(struct net_device *netdev, int sset) +{ + struct otx2_nic *vf = netdev_priv(netdev); + int qstats_count; + + if (sset != ETH_SS_STATS) + return -EINVAL; + + qstats_count = otx2_n_queue_stats * + (vf->hw.rx_queues + vf->hw.tx_queues); + + return otx2_n_dev_stats + otx2_n_drv_stats + qstats_count + 1; +} + +static const struct ethtool_ops otx2vf_ethtool_ops = { + .supported_coalesce_params = ETHTOOL_COALESCE_USECS | + ETHTOOL_COALESCE_MAX_FRAMES, + .get_link = otx2_get_link, + .get_drvinfo = otx2vf_get_drvinfo, + .get_strings = otx2vf_get_strings, + .get_ethtool_stats = otx2vf_get_ethtool_stats, + .get_sset_count = otx2vf_get_sset_count, + .set_channels = otx2_set_channels, + .get_channels = otx2_get_channels, + .get_rxnfc = otx2_get_rxnfc, + .set_rxnfc = otx2_set_rxnfc, + .get_rxfh_key_size = otx2_get_rxfh_key_size, + .get_rxfh_indir_size = otx2_get_rxfh_indir_size, + .get_rxfh = otx2_get_rxfh, + .set_rxfh = otx2_set_rxfh, + .get_ringparam = otx2_get_ringparam, + .set_ringparam = otx2_set_ringparam, + .get_coalesce = otx2_get_coalesce, + .set_coalesce = otx2_set_coalesce, + .get_msglevel = otx2_get_msglevel, + .set_msglevel = otx2_set_msglevel, + .get_pauseparam = otx2_get_pauseparam, + .set_pauseparam = otx2_set_pauseparam, +}; + +void otx2vf_set_ethtool_ops(struct net_device *netdev) +{ + netdev->ethtool_ops = &otx2vf_ethtool_ops; +} +EXPORT_SYMBOL(otx2vf_set_ethtool_ops); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index cf366dc..11885dc 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -381,11 +381,13 @@ int otx2vf_open(struct net_device *netdev) return 0; } +EXPORT_SYMBOL(otx2vf_open); int otx2vf_stop(struct net_device *netdev) { return otx2_stop(netdev); } +EXPORT_SYMBOL(otx2vf_stop); static netdev_tx_t otx2vf_xmit(struct sk_buff *skb, struct net_device *netdev) { @@ -592,6 +594,8 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto err_detach_rsrc; } + otx2vf_set_ethtool_ops(netdev); + /* Enable pause frames by default */ vf->flags |= OTX2_FLAG_RX_PAUSE_ENABLED; vf->flags |= OTX2_FLAG_TX_PAUSE_ENABLED; From patchwork Fri Mar 13 09:42:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 222564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDCF3C10DCE for ; Fri, 13 Mar 2020 09:43:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C27462073E for ; Fri, 13 Mar 2020 09:43:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UoI7fQkQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726650AbgCMJn0 (ORCPT ); Fri, 13 Mar 2020 05:43:26 -0400 Received: from mail-pj1-f67.google.com ([209.85.216.67]:39957 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726216AbgCMJnZ (ORCPT ); Fri, 13 Mar 2020 05:43:25 -0400 Received: by mail-pj1-f67.google.com with SMTP id bo3so2621214pjb.5 for ; Fri, 13 Mar 2020 02:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pEuzk5nHPaOuqkHNBKDDxj2niCpDxm6FgFh0QJdW1BE=; b=UoI7fQkQuD345QHVULyLcjAjjK9hVmt+FNpknjwdhIu8Xr2472gAHEHUFFJiLg+y08 Vh0XfFBZaF4awKbrKCydkJtyI1avWBU5ZkBQgwyXUuV6ULB82s3AnNo8mKAp3j+mDCbP RYpXOWnWE+ojL6eKEnCjEevs4UP+ChtyN6aqGcduJZ3OoIX+3t1VYmqNmrYt7rShoxOI AtoLSsko4OioXdY4eps8OHlWdi0kOmV2mjF9RoDA3m9IDI1Xh1cGQEe0ndMg2yFwQ/AD 96r01/qCSAeUjk72c8PsTbPg/ho4a2ojNnX7lo+2HsO7WKTDZDDF53fooOUzr2P5e2/t xP1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pEuzk5nHPaOuqkHNBKDDxj2niCpDxm6FgFh0QJdW1BE=; b=tStRYrthjNeqBStT3XyHH2FFEh9tG8RV6YZ0jL27D/rgCeniC0HBOLNmbilTGIpFtd hmblJ8k4bfwCMGyckxlOVlbFwTa3T74UOSSCFZoM34lk477qGbOl+LjnKWfAu4OFwKiW aLeg4EgWX9HqDRTNcfzJRwNRWGoo0+6IMhVO3sZ6NulSFZ8X7RVSM/mP3WAgibWyg3Y7 +MpIwTiJlE8J+BIq44mtfpjdhlObEvXbqsv/zHOpsU5Q/kaTZYolRpCqYifCOdaT02PJ +6QoefjEAWXoAukMTpN9xYBroFUtpav79KULY8IK/3bJ1FzPRpXpU9U9iefYOpbLHSWR ds6g== X-Gm-Message-State: ANhLgQ0xZN7ifS2ZKhvSsyF7bLRrfvk98tku+iVj/5SPTyj9RUK0Wm0i xkhXQmOKfHi7jGjRitUj2989dgvyW9o= X-Google-Smtp-Source: ADFU+vtDfjD1kakFLgtXPvPZOzo3myxEchsgcZrLjX/4XaiU4Eq6uHXlczHHidZ3HwtgZHTylDZNZA== X-Received: by 2002:a17:90b:14c2:: with SMTP id jz2mr8680255pjb.152.1584092602174; Fri, 13 Mar 2020 02:43:22 -0700 (PDT) Received: from machine421.marvell.com ([115.113.156.2]) by smtp.googlemail.com with ESMTPSA id v67sm13896386pfc.120.2020.03.13.02.43.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 Mar 2020 02:43:21 -0700 (PDT) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, leon@kernel.org, Geetha sowjanya , Sunil Goutham Subject: [PATCH v2 net-next 6/7] octeontx2-pf: Cleanup all receive buffers in SG descriptor Date: Fri, 13 Mar 2020 15:12:45 +0530 Message-Id: <1584092566-4793-7-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> References: <1584092566-4793-1-git-send-email-sunil.kovvuri@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Geetha sowjanya With MTU sized receive buffers it is not expected to have CQE_RX with multiple receive buffer pointers. But since same physcial link is shared by PF and it's VFs, the max receive packet configured at link could be morethan MTU. Hence there is a chance of receiving plts morethan MTU which then gets DMA'ed into multiple buffers and notified in a single CQE_RX. This patch treats such pkts as errors and frees up receive buffers pointers back to hardware. Also on the transmit side this patch sets SMQ MAXLEN to max value to avoid HW length errors for the packets whose size > MTU, eg due to path MTU. Signed-off-by: Geetha sowjanya Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.c | 9 +++--- .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.c | 34 ++++++++++++++++++---- 2 files changed, 34 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 70d97c7..bac1922 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -212,8 +212,6 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) return -ENOMEM; } - /* SMQ config limits maximum pkt size that can be transmitted */ - req->update_smq = true; pfvf->max_frs = mtu + OTX2_ETH_HLEN; req->maxlen = pfvf->max_frs; @@ -472,7 +470,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) /* Set topology e.t.c configuration */ if (lvl == NIX_TXSCH_LVL_SMQ) { req->reg[0] = NIX_AF_SMQX_CFG(schq); - req->regval[0] = ((pfvf->netdev->mtu + OTX2_ETH_HLEN) << 8) | + req->regval[0] = ((OTX2_MAX_MTU + OTX2_ETH_HLEN) << 8) | OTX2_MIN_MTU; req->regval[0] |= (0x20ULL << 51) | (0x80ULL << 39) | @@ -582,17 +580,19 @@ void otx2_sqb_flush(struct otx2_nic *pfvf) { int qidx, sqe_tail, sqe_head; u64 incr, *ptr, val; + int timeout = 1000; ptr = (u64 *)otx2_get_regaddr(pfvf, NIX_LF_SQ_OP_STATUS); for (qidx = 0; qidx < pfvf->hw.tx_queues; qidx++) { incr = (u64)qidx << 32; - while (1) { + while (timeout) { val = otx2_atomic64_add(incr, ptr); sqe_head = (val >> 20) & 0x3F; sqe_tail = (val >> 28) & 0x3F; if (sqe_head == sqe_tail) break; usleep_range(1, 3); + timeout--; } } } @@ -988,6 +988,7 @@ void otx2_aura_pool_free(struct otx2_nic *pfvf) qmem_free(pfvf->dev, pool->fc_addr); } devm_kfree(pfvf->dev, pfvf->qset.pool); + pfvf->qset.pool = NULL; } static int otx2_aura_init(struct otx2_nic *pfvf, int aura_id, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index 1865f16..b4d523a 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -138,6 +138,25 @@ static void otx2_set_rxhash(struct otx2_nic *pfvf, skb_set_hash(skb, hash, hash_type); } +static void otx2_free_rcv_seg(struct otx2_nic *pfvf, struct nix_cqe_rx_s *cqe, + int qidx) +{ + struct nix_rx_sg_s *sg = &cqe->sg; + void *end, *start; + u64 *seg_addr; + int seg; + + start = (void *)sg; + end = start + ((cqe->parse.desc_sizem1 + 1) * 16); + while (start < end) { + sg = (struct nix_rx_sg_s *)start; + seg_addr = &sg->seg_addr; + for (seg = 0; seg < sg->segs; seg++, seg_addr++) + otx2_aura_freeptr(pfvf, qidx, *seg_addr & ~0x07ULL); + start += sizeof(*sg); + } +} + static bool otx2_check_rcv_errors(struct otx2_nic *pfvf, struct nix_cqe_rx_s *cqe, int qidx) { @@ -189,16 +208,17 @@ static bool otx2_check_rcv_errors(struct otx2_nic *pfvf, /* For now ignore all the NPC parser errors and * pass the packets to stack. */ - return false; + if (cqe->sg.segs == 1) + return false; } /* If RXALL is enabled pass on packets to stack. */ - if (cqe->sg.segs && (pfvf->netdev->features & NETIF_F_RXALL)) + if (cqe->sg.segs == 1 && (pfvf->netdev->features & NETIF_F_RXALL)) return false; /* Free buffer back to pool */ if (cqe->sg.segs) - otx2_aura_freeptr(pfvf, qidx, cqe->sg.seg_addr & ~0x07ULL); + otx2_free_rcv_seg(pfvf, cqe, qidx); return true; } @@ -210,7 +230,7 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, struct nix_rx_parse_s *parse = &cqe->parse; struct sk_buff *skb = NULL; - if (unlikely(parse->errlev || parse->errcode)) { + if (unlikely(parse->errlev || parse->errcode || cqe->sg.segs > 1)) { if (otx2_check_rcv_errors(pfvf, cqe, cq->cq_idx)) return; } @@ -789,11 +809,15 @@ void otx2_cleanup_rx_cqes(struct otx2_nic *pfvf, struct otx2_cq_queue *cq) while ((cqe = (struct nix_cqe_rx_s *)otx2_get_next_cqe(cq))) { if (!cqe->sg.subdc) continue; + processed_cqe++; + if (cqe->sg.segs > 1) { + otx2_free_rcv_seg(pfvf, cqe, cq->cq_idx); + continue; + } iova = cqe->sg.seg_addr - OTX2_HEAD_ROOM; pa = otx2_iova_to_phys(pfvf->iommu_domain, iova); otx2_dma_unmap_page(pfvf, iova, pfvf->rbsize, DMA_FROM_DEVICE); put_page(virt_to_page(phys_to_virt(pa))); - processed_cqe++; } /* Free CQEs to HW */