From patchwork Tue May 15 18:20:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Salil Mehta X-Patchwork-Id: 135924 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1375008lji; Tue, 15 May 2018 11:24:37 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrKidA+NGTBHi+6gYxiHZHhJq9jWHpsZTQxK/4/Flbv+qyRFfq9CYiXBN5RcthOHKSwJ3cn X-Received: by 2002:a17:902:903:: with SMTP id 3-v6mr595914plm.377.1526408676966; Tue, 15 May 2018 11:24:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526408676; cv=none; d=google.com; s=arc-20160816; b=QWgnfN49WN421DoiVXWq4Nw9QEHkiHRPf2FQOrbWb0DTblXHLYf3Z6DTfkzFrosKDa xlK4ls1KXFPTbfgd9l7SBZqYJvYD7zRZRasJeRvqNyaDjLNIaBcCGr/TW4dBmD8+2sqU auc3n7mnsRPiK3MgjG1j/1SCtIUsMiv0ga3uwKRJ2dR1fGSRgyBLubvfMHWW7cTLnb5Z Mo1/eX+Nt5eoXzYkJV5Fb7TfcL1ba+h4/exKq8XbBhA5ZEgkuoaX21wAneGWwZXo5xG/ NSOxAJ1TtxmDiIJ0Z21CCtTQAoJAFA/kNyqg188QnMaLNyMKu8KweBFl9wHkebYFMUdC PNZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=ParrnGO5yUoosUpKhJb7iGNVQ8QcxrxS0vgsZOkjUwo=; b=Kgs564Cd4rezh7k7MmCjAN/g1ybWJoVCEM7UI/XYPPkWWeEINk/GqgQysr4NM1uBIh n4LLEgQc+BB/4rbBdT6r7RPqSEX9moZT/+jj1IxUrcA6BOUow2md2ez0vWtTu1dVY2Ps IaIOtgVzw1UhdDYZ9JVTIv8Rkvpgj1NobMf6Ob5dxiPJp91UGSAky5HSeOSHmZLNFXkK Wfcn4db0bvY3lbI9/gqjCbAfCa7WaR5sIzrs9opW8xH8bnCVZpUHXO4zK8L7ZxHWUY2K VFQ2kTZOe0OMhcR47yVxwiZYuw6B0zcQU1lay9OHrvnol++pF1S/8NeyZCabj865aNM7 nTTw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f18-v6si493975pgt.63.2018.05.15.11.24.36; Tue, 15 May 2018 11:24:36 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753179AbeEOSVP (ORCPT + 29 others); Tue, 15 May 2018 14:21:15 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:59834 "EHLO huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1752978AbeEOSVL (ORCPT ); Tue, 15 May 2018 14:21:11 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id E8CE6F9A7C66B; Wed, 16 May 2018 02:21:06 +0800 (CST) Received: from S00293818-DELL1.china.huawei.com (10.202.227.234) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.382.0; Wed, 16 May 2018 02:20:58 +0800 From: Salil Mehta To: CC: , , , , , , , Fuyun Liang Subject: [PATCH net-next 01/10] net: hns3: Fix for deadlock problem occurring when unregistering ae_algo Date: Tue, 15 May 2018 19:20:05 +0100 Message-ID: <20180515182014.42196-2-salil.mehta@huawei.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20180515182014.42196-1-salil.mehta@huawei.com> References: <20180515182014.42196-1-salil.mehta@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.234] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Fuyun Liang When hnae3_unregister_ae_algo is called by PF, pci_disable_sriov is called. And then, hns3_remove is called by VF. We get deadlocked in this case. Since VF pci device is dependent on PF pci device, When PF pci device is removed, VF pci device must be removed. Also, To solve the deadlock problem, VF pci device should be removed before PF pci device is removed. This patch moves pci_enable/disable_sriov from hclge to hns3 to solve the deadlock problem. Also, we do not need to return EPROBE_DEFER in hnae3_register_ae_dev, because SRIOV is no longer enabled in the context calling hnae3_register_ae_dev. Mutex_trylock can be replaced with mutex_lock. Fixes: 424eb834a9be ("net: hns3: Unified HNS3 {VF|PF} Ethernet Driver for hip08 SoC") Signed-off-by: Fuyun Liang Signed-off-by: Peng Li Signed-off-by: Salil Mehta --- drivers/net/ethernet/hisilicon/hns3/hnae3.c | 12 +--- drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 74 +++++++++++++++++++++- .../ethernet/hisilicon/hns3/hns3pf/hclge_main.c | 42 ++---------- 3 files changed, 79 insertions(+), 49 deletions(-) -- 2.7.4 diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.c b/drivers/net/ethernet/hisilicon/hns3/hnae3.c index 02145f2..1686ceb 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hnae3.c +++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.c @@ -196,17 +196,9 @@ int hnae3_register_ae_dev(struct hnae3_ae_dev *ae_dev) const struct pci_device_id *id; struct hnae3_ae_algo *ae_algo; struct hnae3_client *client; - int ret = 0, lock_acquired; + int ret = 0; - /* we can get deadlocked if SRIOV is being enabled in context to probe - * and probe gets called again in same context. This can happen when - * pci_enable_sriov() is called to create VFs from PF probes context. - * Therefore, for simplicity uniformly defering further probing in all - * cases where we detect contention. - */ - lock_acquired = mutex_trylock(&hnae3_common_lock); - if (!lock_acquired) - return -EPROBE_DEFER; + mutex_lock(&hnae3_common_lock); list_add_tail(&ae_dev->node, &hnae3_ae_dev_list); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c index 4031174..af9e90f 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c @@ -1487,6 +1487,68 @@ static const struct net_device_ops hns3_nic_netdev_ops = { .ndo_set_vf_vlan = hns3_ndo_set_vf_vlan, }; +static bool hns3_is_phys_func(struct pci_dev *pdev) +{ + u32 dev_id = pdev->device; + + switch (dev_id) { + case HNAE3_DEV_ID_GE: + case HNAE3_DEV_ID_25GE: + case HNAE3_DEV_ID_25GE_RDMA: + case HNAE3_DEV_ID_25GE_RDMA_MACSEC: + case HNAE3_DEV_ID_50GE_RDMA: + case HNAE3_DEV_ID_50GE_RDMA_MACSEC: + case HNAE3_DEV_ID_100G_RDMA_MACSEC: + return true; + case HNAE3_DEV_ID_100G_VF: + case HNAE3_DEV_ID_100G_RDMA_DCB_PFC_VF: + return false; + default: + dev_warn(&pdev->dev, "un-recognized pci device-id %d", + dev_id); + } + + return false; +} + +static int get_num_req_vfs(struct pci_dev *pdev) +{ + /* a variable vf num will be supported later */ + return pci_sriov_get_totalvfs(pdev); +} + +static void hns3_enable_sriov(struct pci_dev *pdev) +{ + int num_req_vfs = get_num_req_vfs(pdev); + int ret; + + /* Enable SRIOV */ + if (!num_req_vfs) + return; + + dev_info(&pdev->dev, "active VFs(%d) found, enabling SRIOV\n", + num_req_vfs); + + ret = pci_enable_sriov(pdev, num_req_vfs); + if (ret) + dev_err(&pdev->dev, "SRIOV enable failed %d\n", ret); +} + +static void hns3_disable_sriov(struct pci_dev *pdev) +{ + /* If our VFs are assigned we cannot shut down SR-IOV + * without causing issues, so just leave the hardware + * available but disabled + */ + if (pci_vfs_assigned(pdev)) { + dev_warn(&pdev->dev, + "disabling driver while VFs are assigned\n"); + return; + } + + pci_disable_sriov(pdev); +} + /* hns3_probe - Device initialization routine * @pdev: PCI device information struct * @ent: entry in hns3_pci_tbl @@ -1514,7 +1576,14 @@ static int hns3_probe(struct pci_dev *pdev, const struct pci_device_id *ent) ae_dev->dev_type = HNAE3_DEV_KNIC; pci_set_drvdata(pdev, ae_dev); - return hnae3_register_ae_dev(ae_dev); + ret = hnae3_register_ae_dev(ae_dev); + if (ret) + return ret; + + if (hns3_is_phys_func(pdev) && IS_ENABLED(CONFIG_PCI_IOV)) + hns3_enable_sriov(pdev); + + return 0; } /* hns3_remove - Device removal routine @@ -1524,6 +1593,9 @@ static void hns3_remove(struct pci_dev *pdev) { struct hnae3_ae_dev *ae_dev = pci_get_drvdata(pdev); + if (hns3_is_phys_func(pdev) && IS_ENABLED(CONFIG_PCI_IOV)) + hns3_disable_sriov(pdev); + hnae3_unregister_ae_dev(ae_dev); } diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c index 316ec842..343197a 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c @@ -1473,21 +1473,8 @@ static int hclge_alloc_vport(struct hclge_dev *hdev) hdev->vport = vport; hdev->num_alloc_vport = num_vport; -#ifdef CONFIG_PCI_IOV - /* Enable SRIOV */ - if (hdev->num_req_vfs) { - dev_info(&pdev->dev, "active VFs(%d) found, enabling SRIOV\n", - hdev->num_req_vfs); - ret = pci_enable_sriov(hdev->pdev, hdev->num_req_vfs); - if (ret) { - hdev->num_alloc_vfs = 0; - dev_err(&pdev->dev, "SRIOV enable failed %d\n", - ret); - return ret; - } - } - hdev->num_alloc_vfs = hdev->num_req_vfs; -#endif + if (IS_ENABLED(CONFIG_PCI_IOV)) + hdev->num_alloc_vfs = hdev->num_req_vfs; for (i = 0; i < num_vport; i++) { vport->back = hdev; @@ -2946,21 +2933,6 @@ static void hclge_service_task(struct work_struct *work) hclge_service_complete(hdev); } -static void hclge_disable_sriov(struct hclge_dev *hdev) -{ - /* If our VFs are assigned we cannot shut down SR-IOV - * without causing issues, so just leave the hardware - * available but disabled - */ - if (pci_vfs_assigned(hdev->pdev)) { - dev_warn(&hdev->pdev->dev, - "disabling driver while VFs are assigned\n"); - return; - } - - pci_disable_sriov(hdev->pdev); -} - struct hclge_vport *hclge_get_vport(struct hnae3_handle *handle) { /* VF handle has no client */ @@ -5540,7 +5512,7 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) ret = hclge_map_tqp(hdev); if (ret) { dev_err(&pdev->dev, "Map tqp error, ret = %d.\n", ret); - goto err_sriov_disable; + goto err_msi_irq_uninit; } if (hdev->hw.mac.media_type == HNAE3_MEDIA_TYPE_COPPER) { @@ -5548,7 +5520,7 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) if (ret) { dev_err(&hdev->pdev->dev, "mdio config fail ret=%d\n", ret); - goto err_sriov_disable; + goto err_msi_irq_uninit; } } @@ -5612,9 +5584,6 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) err_mdiobus_unreg: if (hdev->hw.mac.phydev) mdiobus_unregister(hdev->hw.mac.mdio_bus); -err_sriov_disable: - if (IS_ENABLED(CONFIG_PCI_IOV)) - hclge_disable_sriov(hdev); err_msi_irq_uninit: hclge_misc_irq_uninit(hdev); err_msi_uninit: @@ -5717,9 +5686,6 @@ static void hclge_uninit_ae_dev(struct hnae3_ae_dev *ae_dev) set_bit(HCLGE_STATE_DOWN, &hdev->state); - if (IS_ENABLED(CONFIG_PCI_IOV)) - hclge_disable_sriov(hdev); - if (hdev->service_timer.function) del_timer_sync(&hdev->service_timer); if (hdev->service_task.func)