From patchwork Thu Mar 22 14:28:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Salil Mehta X-Patchwork-Id: 132289 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp921432ljb; Thu, 22 Mar 2018 07:31:51 -0700 (PDT) X-Google-Smtp-Source: AG47ELuTPOdO8czJ+Ml2Zca5UvvLwiEba52NPpVVIziVIwl6CwzfuxeuOWKE+LxAC9Rt2+63vjxm X-Received: by 2002:a17:902:8c83:: with SMTP id t3-v6mr15644823plo.391.1521729111205; Thu, 22 Mar 2018 07:31:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521729111; cv=none; d=google.com; s=arc-20160816; b=xnCGZo7ZwsCwe+wryqN318cRRuMDUr6KABiuqvynYiXa6OAxwvLGFmVTxE49NtQ5WN 0QBgHgaKCQrYEBq7BD/Wvb48BPiyM9LNMlLM0IHZ/s3e7lo20igu1W+HE582H1ViMIyA Iqi1esGF19SCCX7a1I7z/xJnc74g0YzanhM6YHGbB/uki8/Zbm8mPRCTs15diWiA/ywD UG4uaGIEqqRQwcYHlYe+qP6/h5vNZ7uITokYS//FFNN/ogqkqLNxYxrTkkQZ0WppbC/1 pk80ntXfUZ6deny38JxaQHnuE/i6EH1gNNiPGQl8bAbxYF2vprXEDUQlkmg9sMvgqlT9 d3+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=P1qAa47DWNC1khHEXP6fiziUjpStC9CKqnRemGSKSIQ=; b=Jl3hxwcK0SV6rwUSf+1AmizIXe794JBUtA3St3D9elKDAGM7MQvFuuTNvjQ+Prn8Md 3gEN21ptii5k7ECIl6V9k3SjAtBilLgU0NcdIDVzJhT2ZnjaQs9hf3lkw/B4XLRL4G6j QnGBJj/aPjMnKw9AMAfDVkhs1iC8g6hIACER1t6G93z0fy70MbrAZFn4IseoscHuy7T7 VYJjPC70mUmXP8O1Cb5pn4XqiLQGbHY252ibKhB/6uE1mu0trIwRDG+gxf1/21iiH/L6 5xU3Lj8k5/GRM2hFGZ2/0LaVvwR7jr7VEWYlgT4l5YQIBIQMDSNCjelWq+ueJiqyf7fx YMJA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g14si4503265pgn.275.2018.03.22.07.31.50; Thu, 22 Mar 2018 07:31:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755369AbeCVObs (ORCPT + 8 others); Thu, 22 Mar 2018 10:31:48 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:6260 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754583AbeCVOaM (ORCPT ); Thu, 22 Mar 2018 10:30:12 -0400 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 3D7431C3D6BD1; Thu, 22 Mar 2018 22:29:57 +0800 (CST) Received: from S00293818-DELL1.china.huawei.com (10.202.226.47) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.361.1; Thu, 22 Mar 2018 22:29:47 +0800 From: Salil Mehta To: CC: , , , , , Subject: [PATCH net-next 2/9] net: hns3: Add VF Reset Service Task to support event handling Date: Thu, 22 Mar 2018 14:28:53 +0000 Message-ID: <20180322142900.22860-3-salil.mehta@huawei.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20180322142900.22860-1-salil.mehta@huawei.com> References: <20180322142900.22860-1-salil.mehta@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.226.47] X-CFilter-Loop: Reflected Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org VF reset would involve handling of different reset related events from the stack, physical function, mailbox etc. Reset service task would be used in servicing such reset event requests and later handling the hardware completions waits and initiating the stack resets. Signed-off-by: Salil Mehta --- .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c | 31 ++++++++++++++++++++++ .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h | 4 +++ 2 files changed, 35 insertions(+) -- 2.7.4 diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index 6c3881d..cdb6e7a 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -867,6 +867,15 @@ static void hclgevf_get_misc_vector(struct hclgevf_dev *hdev) hdev->num_msi_used += 1; } +void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev) +{ + if (!test_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state) && + !test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) { + set_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state); + schedule_work(&hdev->rst_service_task); + } +} + static void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev) { if (!test_and_set_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state)) @@ -889,6 +898,24 @@ static void hclgevf_service_timer(struct timer_list *t) hclgevf_task_schedule(hdev); } +static void hclgevf_reset_service_task(struct work_struct *work) +{ + struct hclgevf_dev *hdev = + container_of(work, struct hclgevf_dev, rst_service_task); + + if (test_and_set_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) + return; + + clear_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state); + + /* body of the reset service task will constitute of hclge device + * reset state handling. This code shall be added subsequently in + * next patches. + */ + + clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state); +} + static void hclgevf_mailbox_service_task(struct work_struct *work) { struct hclgevf_dev *hdev; @@ -1097,6 +1124,8 @@ static void hclgevf_state_init(struct hclgevf_dev *hdev) INIT_WORK(&hdev->service_task, hclgevf_service_task); clear_bit(HCLGEVF_STATE_SERVICE_SCHED, &hdev->state); + INIT_WORK(&hdev->rst_service_task, hclgevf_reset_service_task); + mutex_init(&hdev->mbx_resp.mbx_mutex); /* bring the device down */ @@ -1113,6 +1142,8 @@ static void hclgevf_state_uninit(struct hclgevf_dev *hdev) cancel_work_sync(&hdev->service_task); if (hdev->mbx_service_task.func) cancel_work_sync(&hdev->mbx_service_task); + if (hdev->rst_service_task.func) + cancel_work_sync(&hdev->rst_service_task); mutex_destroy(&hdev->mbx_resp.mbx_mutex); } diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h index 0eaea06..8b5fa67 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h @@ -52,6 +52,8 @@ enum hclgevf_states { HCLGEVF_STATE_DISABLED, /* task states */ HCLGEVF_STATE_SERVICE_SCHED, + HCLGEVF_STATE_RST_SERVICE_SCHED, + HCLGEVF_STATE_RST_HANDLING, HCLGEVF_STATE_MBX_SERVICE_SCHED, HCLGEVF_STATE_MBX_HANDLING, }; @@ -146,6 +148,7 @@ struct hclgevf_dev { struct timer_list service_timer; struct work_struct service_task; + struct work_struct rst_service_task; struct work_struct mbx_service_task; struct hclgevf_tqp *htqp; @@ -165,4 +168,5 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev); void hclgevf_update_link_status(struct hclgevf_dev *hdev, int link_state); void hclgevf_update_speed_duplex(struct hclgevf_dev *hdev, u32 speed, u8 duplex); +void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev); #endif From patchwork Thu Mar 22 14:28:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Salil Mehta X-Patchwork-Id: 132291 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp922435ljb; Thu, 22 Mar 2018 07:32:36 -0700 (PDT) X-Google-Smtp-Source: AG47ELvN94SLH5kYQx/k/AhAFom4adWWqcvYLhxY8nzKq03y4aoBrzPEnHaGtritk7KmciDIVl62 X-Received: by 10.101.92.138 with SMTP id a10mr18127201pgt.129.1521729156206; Thu, 22 Mar 2018 07:32:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521729156; cv=none; d=google.com; s=arc-20160816; b=kZmhyF5GFtvzbMppTZcbbke+bDoznQd9VEV6MGJqeOFALN621HTOHMIbXfl/aJka+j vaBWZ4DiJpo95TjxB/hUaSxuw681O5wbwb3KHwTefd9ZSj/b2/BH9xTpYuoqg+kdENY5 pKNxx2Hb0wn2k+v5ShddJ14DafK0WUexTRRl3deF1zvYDkdc0M/qOX3HNBbIktNl/ElB Fr1yVcgXl1WBFy6yT8xVRLX7wiAjpIBUedVHysmP+kHii6i56aIZPNVgb5m/04WYiPKk Xv2AijzzY6i04ai7Ywmvy1JSSwyFLhq0cNwW7GEoRLqgCFB1M+13l0hYxcIjLj3k1Vcg T8XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=aM1caXCOXLPgSmFyc4oCmeubvZp+tSP81UX4IgtPB1U=; b=XJz6JIdWyG/s26XgpOUyykFhi0iOcO+EnDBGs6ckPNzS0Pg4h+hyI7nKtLjA9PXpSF u5Ve/8T5tbj6n/es/AzhOZ5MvK1gzciHW692smnG6I3PTuNLZJSmkR9bvL7vRKAmP3+v 6CX+Kt5dj8moGcUXV7VtC8MNMT3MIWJ5HoI6Lpd1YFaimMfPd6fYyCgnvoCoV/hH6144 Fq3Nr8yx4vTCwdDqb2pYEoECZbNpBZPP2BmlmOGBWYA1AxFq3kevOVIGLulIbjl2jmAs zcIm7YBEmtkReaH+5gc+B1UF+7tiMNFcuiMsl9GKQ9wwi/g8Ljyg85QHO8sRKm1JC+Uw VXAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u27si4936726pfi.399.2018.03.22.07.32.35; Thu, 22 Mar 2018 07:32:36 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755174AbeCVOcc (ORCPT + 8 others); Thu, 22 Mar 2018 10:32:32 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:45606 "EHLO huawei.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1754640AbeCVOaH (ORCPT ); Thu, 22 Mar 2018 10:30:07 -0400 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 91952279A11EF; Thu, 22 Mar 2018 22:30:02 +0800 (CST) Received: from S00293818-DELL1.china.huawei.com (10.202.226.47) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.361.1; Thu, 22 Mar 2018 22:29:57 +0800 From: Salil Mehta To: CC: , , , , , Subject: [PATCH net-next 6/9] net: hns3: Add support to re-initialize the hclge device Date: Thu, 22 Mar 2018 14:28:57 +0000 Message-ID: <20180322142900.22860-7-salil.mehta@huawei.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20180322142900.22860-1-salil.mehta@huawei.com> References: <20180322142900.22860-1-salil.mehta@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.226.47] X-CFilter-Loop: Reflected Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org After the hardware reset we should re-fetch the configuration from PF like queue info and tc info. This might have impact on allocations made like that of TQPs. Hence, we should release all such allocations and re-allocate fresh according to new fetched configuration after reset. Signed-off-by: Salil Mehta --- .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c | 106 ++++++++++++++++++--- .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h | 14 +++ 2 files changed, 106 insertions(+), 14 deletions(-) -- 2.7.4 diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index bd45b11..6dd7561 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -10,6 +10,8 @@ #define HCLGEVF_NAME "hclgevf" +static int hclgevf_init_hdev(struct hclgevf_dev *hdev); +static void hclgevf_uninit_hdev(struct hclgevf_dev *hdev); static struct hnae3_ae_algo ae_algovf; static const struct pci_device_id ae_algovf_pci_tbl[] = { @@ -209,6 +211,12 @@ static int hclgevf_alloc_tqps(struct hclgevf_dev *hdev) struct hclgevf_tqp *tqp; int i; + /* if this is on going reset then we need to re-allocate the TPQs + * since we cannot assume we would get same number of TPQs back from PF + */ + if (hclgevf_dev_ongoing_reset(hdev)) + devm_kfree(&hdev->pdev->dev, hdev->htqp); + hdev->htqp = devm_kcalloc(&hdev->pdev->dev, hdev->num_tqps, sizeof(struct hclgevf_tqp), GFP_KERNEL); if (!hdev->htqp) @@ -252,6 +260,12 @@ static int hclgevf_knic_setup(struct hclgevf_dev *hdev) new_tqps = kinfo->rss_size * kinfo->num_tc; kinfo->num_tqps = min(new_tqps, hdev->num_tqps); + /* if this is on going reset then we need to re-allocate the hnae queues + * as well since number of TPQs from PF might have changed. + */ + if (hclgevf_dev_ongoing_reset(hdev)) + devm_kfree(&hdev->pdev->dev, kinfo->tqp); + kinfo->tqp = devm_kcalloc(&hdev->pdev->dev, kinfo->num_tqps, sizeof(struct hnae3_queue *), GFP_KERNEL); if (!kinfo->tqp) @@ -878,10 +892,18 @@ static int hclgevf_reset_wait(struct hclgevf_dev *hdev) static int hclgevf_reset_stack(struct hclgevf_dev *hdev) { + int ret; + /* uninitialize the nic client */ hclgevf_notify_client(hdev, HNAE3_UNINIT_CLIENT); - /* re-initialize the hclge device - add code here */ + /* re-initialize the hclge device */ + ret = hclgevf_init_hdev(hdev); + if (ret) { + dev_err(&hdev->pdev->dev, + "hclge device re-init failed, VF is disabled!\n"); + return ret; + } /* bring up the nic client again */ hclgevf_notify_client(hdev, HNAE3_INIT_CLIENT); @@ -1179,6 +1201,22 @@ static int hclgevf_configure(struct hclgevf_dev *hdev) return hclgevf_get_tc_info(hdev); } +static int hclgevf_alloc_hdev(struct hnae3_ae_dev *ae_dev) +{ + struct pci_dev *pdev = ae_dev->pdev; + struct hclgevf_dev *hdev = ae_dev->priv; + + hdev = devm_kzalloc(&pdev->dev, sizeof(*hdev), GFP_KERNEL); + if (!hdev) + return -ENOMEM; + + hdev->pdev = pdev; + hdev->ae_dev = ae_dev; + ae_dev->priv = hdev; + + return 0; +} + static int hclgevf_init_roce_base_info(struct hclgevf_dev *hdev) { struct hnae3_handle *roce = &hdev->roce; @@ -1284,6 +1322,10 @@ static void hclgevf_ae_stop(struct hnae3_handle *handle) static void hclgevf_state_init(struct hclgevf_dev *hdev) { + /* if this is on going reset then skip this initialization */ + if (hclgevf_dev_ongoing_reset(hdev)) + return; + /* setup tasks for the MBX */ INIT_WORK(&hdev->mbx_service_task, hclgevf_mailbox_service_task); clear_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state); @@ -1325,6 +1367,10 @@ static int hclgevf_init_msi(struct hclgevf_dev *hdev) int vectors; int i; + /* if this is on going reset then skip this initialization */ + if (hclgevf_dev_ongoing_reset(hdev)) + return 0; + hdev->num_msi = HCLGEVF_MAX_VF_VECTOR_NUM; vectors = pci_alloc_irq_vectors(pdev, 1, hdev->num_msi, @@ -1375,6 +1421,10 @@ static int hclgevf_misc_irq_init(struct hclgevf_dev *hdev) { int ret = 0; + /* if this is on going reset then skip this initialization */ + if (hclgevf_dev_ongoing_reset(hdev)) + return 0; + hclgevf_get_misc_vector(hdev); ret = request_irq(hdev->misc_vector.vector_irq, hclgevf_misc_irq_handle, @@ -1485,6 +1535,14 @@ static int hclgevf_pci_init(struct hclgevf_dev *hdev) struct hclgevf_hw *hw; int ret; + /* check if we need to skip initialization of pci. This will happen if + * device is undergoing VF reset. Otherwise, we would need to + * re-initialize pci interface again i.e. when device is not going + * through *any* reset or actually undergoing full reset. + */ + if (hclgevf_dev_ongoing_reset(hdev)) + return 0; + ret = pci_enable_device(pdev); if (ret) { dev_err(&pdev->dev, "failed to enable PCI device\n"); @@ -1536,19 +1594,16 @@ static void hclgevf_pci_uninit(struct hclgevf_dev *hdev) pci_set_drvdata(pdev, NULL); } -static int hclgevf_init_ae_dev(struct hnae3_ae_dev *ae_dev) +static int hclgevf_init_hdev(struct hclgevf_dev *hdev) { - struct pci_dev *pdev = ae_dev->pdev; - struct hclgevf_dev *hdev; + struct pci_dev *pdev = hdev->pdev; int ret; - hdev = devm_kzalloc(&pdev->dev, sizeof(*hdev), GFP_KERNEL); - if (!hdev) - return -ENOMEM; - - hdev->pdev = pdev; - hdev->ae_dev = ae_dev; - ae_dev->priv = hdev; + /* check if device is on-going full reset(i.e. pcie as well) */ + if (hclgevf_dev_ongoing_full_reset(hdev)) { + dev_warn(&pdev->dev, "device is going full reset\n"); + hclgevf_uninit_hdev(hdev); + } ret = hclgevf_pci_init(hdev); if (ret) { @@ -1633,15 +1688,38 @@ static int hclgevf_init_ae_dev(struct hnae3_ae_dev *ae_dev) return ret; } -static void hclgevf_uninit_ae_dev(struct hnae3_ae_dev *ae_dev) +static void hclgevf_uninit_hdev(struct hclgevf_dev *hdev) { - struct hclgevf_dev *hdev = ae_dev->priv; - hclgevf_cmd_uninit(hdev); hclgevf_misc_irq_uninit(hdev); hclgevf_state_uninit(hdev); hclgevf_uninit_msi(hdev); hclgevf_pci_uninit(hdev); +} + +static int hclgevf_init_ae_dev(struct hnae3_ae_dev *ae_dev) +{ + struct pci_dev *pdev = ae_dev->pdev; + int ret; + + ret = hclgevf_alloc_hdev(ae_dev); + if (ret) { + dev_err(&pdev->dev, "hclge device allocation failed\n"); + return ret; + } + + ret = hclgevf_init_hdev(ae_dev->priv); + if (ret) + dev_err(&pdev->dev, "hclge device initialization failed\n"); + + return ret; +} + +static void hclgevf_uninit_ae_dev(struct hnae3_ae_dev *ae_dev) +{ + struct hclgevf_dev *hdev = ae_dev->priv; + + hclgevf_uninit_hdev(hdev); ae_dev->priv = NULL; } diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h index afdb15d..8cdc602 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h @@ -169,6 +169,20 @@ struct hclgevf_dev { u32 flag; }; +static inline bool hclgevf_dev_ongoing_reset(struct hclgevf_dev *hdev) +{ + return (hdev && + (test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) && + (hdev->nic.reset_level == HNAE3_VF_RESET)); +} + +static inline bool hclgevf_dev_ongoing_full_reset(struct hclgevf_dev *hdev) +{ + return (hdev && + (test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) && + (hdev->nic.reset_level == HNAE3_VF_FULL_RESET)); +} + int hclgevf_send_mbx_msg(struct hclgevf_dev *hdev, u16 code, u16 subcode, const u8 *msg_data, u8 msg_len, bool need_resp, u8 *resp_data, u16 resp_len); From patchwork Thu Mar 22 14:28:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Salil Mehta X-Patchwork-Id: 132287 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp919862ljb; Thu, 22 Mar 2018 07:30:37 -0700 (PDT) X-Google-Smtp-Source: AG47ELs1CdSF8VBjCIJ7tudBwXBXv+ssTExuOG2EHXSzrDK9PPgsnSxJ/lS/ZFgXOnwtM3trO+fv X-Received: by 2002:a17:902:b288:: with SMTP id u8-v6mr24943790plr.339.1521729036842; Thu, 22 Mar 2018 07:30:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521729036; cv=none; d=google.com; s=arc-20160816; b=nipmh+uIYZHt6LcXLbrWZPy1gcCiHfD40isBeONpv33HASB2uEwzDUAEd1Yebb7cV1 mNWsxzsitITuL/4SThz5VrtwVJNpKsk+dSjZqbHSBvrWeXqdERx+6gOqI1CZ9YNGrKPD HPpt3NzVXUrWjG40EF1dUlVH1rl2Y2iRbzbJL1R/YiZOfGSDSuoQVkLs+l92aYDWxPzY nv6dkRif8/MNAxqu1hCIjjGiTBXzN9PHEHzpjD0awh76PbqUdbmkpIsSuJV+OR0hw0rb 5KObh9mf/RfeV1r7buLMSOkOICHOQnK9LP2xvJ8q4F8gUJTpS+Lrcccx09kPqzw7CpEI D1jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=E7JPYXajRanf0kyHfmXVtJevz4pJUCprS4f81GcmyOw=; b=JF7E++UyqKhIUy5stk6t5nZLUBAQYO/IIYmvPPO2RG0YA86TOXiEGvWWnNE1LdeqGg bbC2dTEW373ObKiMtL0wQUnX9yBlHJi5Dpo7lebFc+OxAgwX7KEXU5qUphj5bpxl1j1t /St0WGKDNz0LNIY3jwcnr4VVS8y6rD8fQMXo3Bz1Tz4pbeOu07FF+q4qBuhSPJpneJDi NfQfMrrrRqkmXLDsSKaXEvLAflQvkgUbBwYZQjCfZjAAbQoL0lsuY7mhWXVvCzZf+bNH Kn/RLQCBsekvlMJYWlEPcUgB3Fa6GP+9LLTsSF9l5M5Fd85xs+gYlEwzDzTA+c57wnbc PZQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x190si4488285pgd.52.2018.03.22.07.30.36; Thu, 22 Mar 2018 07:30:36 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756090AbeCVOae (ORCPT + 8 others); Thu, 22 Mar 2018 10:30:34 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:6262 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751753AbeCVOa2 (ORCPT ); Thu, 22 Mar 2018 10:30:28 -0400 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 5422FE0AD4868; Thu, 22 Mar 2018 22:30:07 +0800 (CST) Received: from S00293818-DELL1.china.huawei.com (10.202.226.47) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.361.1; Thu, 22 Mar 2018 22:30:00 +0800 From: Salil Mehta To: CC: , , , , , Subject: [PATCH net-next 7/9] net: hns3: Changes to support ARQ(Asynchronous Receive Queue) Date: Thu, 22 Mar 2018 14:28:58 +0000 Message-ID: <20180322142900.22860-8-salil.mehta@huawei.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20180322142900.22860-1-salil.mehta@huawei.com> References: <20180322142900.22860-1-salil.mehta@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.226.47] X-CFilter-Loop: Reflected Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Current mailbox CRQ could consists of both synchronous and async responses from the PF. Synchronous responses are time critical and should be handed over to the waiting tasks/context as quickly as possible otherwise timeout occurs. Above problem gets accentuated if CRQ consists of even single async message. Hence, it is important to have quick handling of synchronous messages and maybe deferred handling of async messages This patch introduces separate ARQ(async receive queues) for the async messages. These messages are processed later with repsect to mailbox task while synchronous messages still gets processed in context to mailbox interrupt. ARQ is important as VF reset introduces some new async messages like MBX_ASSERTING_RESET which adds up to the presssure on the responses for synchronousmessages and they timeout even more quickly. Signed-off-by: Salil Mehta --- drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h | 15 ++++ .../ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c | 6 ++ .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c | 16 +++-- .../ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h | 5 ++ .../ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c | 83 +++++++++++++++++++--- 5 files changed, 111 insertions(+), 14 deletions(-) -- 2.7.4 diff --git a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h index e6e1d22..f3e90c2 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h +++ b/drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h @@ -85,6 +85,21 @@ struct hclge_mbx_pf_to_vf_cmd { u16 msg[8]; }; +/* used by VF to store the received Async responses from PF */ +struct hclgevf_mbx_arq_ring { +#define HCLGE_MBX_MAX_ARQ_MSG_SIZE 8 +#define HCLGE_MBX_MAX_ARQ_MSG_NUM 1024 + struct hclgevf_dev *hdev; + u32 head; + u32 tail; + u32 count; + u16 msg_q[HCLGE_MBX_MAX_ARQ_MSG_NUM][HCLGE_MBX_MAX_ARQ_MSG_SIZE]; +}; + #define hclge_mbx_ring_ptr_move_crq(crq) \ (crq->next_to_use = (crq->next_to_use + 1) % crq->desc_num) +#define hclge_mbx_tail_ptr_move_arq(arq) \ + (arq.tail = (arq.tail + 1) % HCLGE_MBX_MAX_ARQ_MSG_SIZE) +#define hclge_mbx_head_ptr_move_arq(arq) \ + (arq.head = (arq.head + 1) % HCLGE_MBX_MAX_ARQ_MSG_SIZE) #endif diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c index 85985e7..1bbfe13 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c @@ -315,6 +315,12 @@ int hclgevf_cmd_init(struct hclgevf_dev *hdev) goto err_csq; } + /* initialize the pointers of async rx queue of mailbox */ + hdev->arq.hdev = hdev; + hdev->arq.head = 0; + hdev->arq.tail = 0; + hdev->arq.count = 0; + /* get firmware version */ ret = hclgevf_cmd_query_firmware_version(&hdev->hw, &version); if (ret) { diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c index 6dd7561..2b84264 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c @@ -1010,10 +1010,13 @@ void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev) } } -static void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev) +void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev) { - if (!test_and_set_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state)) + if (!test_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state) && + !test_bit(HCLGEVF_STATE_MBX_HANDLING, &hdev->state)) { + set_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state); schedule_work(&hdev->mbx_service_task); + } } static void hclgevf_task_schedule(struct hclgevf_dev *hdev) @@ -1025,6 +1028,10 @@ static void hclgevf_task_schedule(struct hclgevf_dev *hdev) static void hclgevf_deferred_task_schedule(struct hclgevf_dev *hdev) { + /* if we have any pending mailbox event then schedule the mbx task */ + if (hdev->mbx_event_pending) + hclgevf_mbx_task_schedule(hdev); + if (test_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state)) hclgevf_reset_task_schedule(hdev); } @@ -1118,7 +1125,7 @@ static void hclgevf_mailbox_service_task(struct work_struct *work) clear_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state); - hclgevf_mbx_handler(hdev); + hclgevf_mbx_async_handler(hdev); clear_bit(HCLGEVF_STATE_MBX_HANDLING, &hdev->state); } @@ -1178,8 +1185,7 @@ static irqreturn_t hclgevf_misc_irq_handle(int irq, void *data) if (!hclgevf_check_event_cause(hdev, &clearval)) goto skip_sched; - /* schedule the VF mailbox service task, if not already scheduled */ - hclgevf_mbx_task_schedule(hdev); + hclgevf_mbx_handler(hdev); hclgevf_clear_event_cause(hdev, clearval); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h index 8cdc602..a477a7c 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h @@ -152,7 +152,9 @@ struct hclgevf_dev { int *vector_irq; bool accept_mta_mc; /* whether to accept mta filter multicast */ + bool mbx_event_pending; struct hclgevf_mbx_resp_status mbx_resp; /* mailbox response */ + struct hclgevf_mbx_arq_ring arq; /* mailbox async rx queue */ struct timer_list service_timer; struct work_struct service_task; @@ -187,8 +189,11 @@ int hclgevf_send_mbx_msg(struct hclgevf_dev *hdev, u16 code, u16 subcode, const u8 *msg_data, u8 msg_len, bool need_resp, u8 *resp_data, u16 resp_len); void hclgevf_mbx_handler(struct hclgevf_dev *hdev); +void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev); + void hclgevf_update_link_status(struct hclgevf_dev *hdev, int link_state); void hclgevf_update_speed_duplex(struct hclgevf_dev *hdev, u32 speed, u8 duplex); void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev); +void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev); #endif diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c index a63ed3a..7687911 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c @@ -132,9 +132,8 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev) struct hclge_mbx_pf_to_vf_cmd *req; struct hclgevf_cmq_ring *crq; struct hclgevf_desc *desc; - u16 link_status, flag; - u32 speed; - u8 duplex; + u16 *msg_q; + u16 flag; u8 *temp; int i; @@ -146,6 +145,12 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev) desc = &crq->desc[crq->next_to_use]; req = (struct hclge_mbx_pf_to_vf_cmd *)desc->data; + /* synchronous messages are time critical and need preferential + * treatment. Therefore, we need to acknowledge all the sync + * responses as quickly as possible so that waiting tasks do not + * timeout and simultaneously queue the async messages for later + * prcessing in context of mailbox task i.e. the slow path. + */ switch (req->msg[0]) { case HCLGE_MBX_PF_VF_RESP: if (resp->received_resp) @@ -165,13 +170,30 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev) } break; case HCLGE_MBX_LINK_STAT_CHANGE: - link_status = le16_to_cpu(req->msg[1]); - memcpy(&speed, &req->msg[2], sizeof(speed)); - duplex = (u8)le16_to_cpu(req->msg[4]); + /* set this mbx event as pending. This is required as we + * might loose interrupt event when mbx task is busy + * handling. This shall be cleared when mbx task just + * enters handling state. + */ + hdev->mbx_event_pending = true; - /* update upper layer with new link link status */ - hclgevf_update_link_status(hdev, link_status); - hclgevf_update_speed_duplex(hdev, speed, duplex); + /* we will drop the async msg if we find ARQ as full + * and continue with next message + */ + if (hdev->arq.count >= HCLGE_MBX_MAX_ARQ_MSG_NUM) { + dev_warn(&hdev->pdev->dev, + "Async Q full, dropping msg(%d)\n", + req->msg[1]); + break; + } + + /* tail the async message in arq */ + msg_q = hdev->arq.msg_q[hdev->arq.tail]; + memcpy(&msg_q[0], req->msg, HCLGE_MBX_MAX_ARQ_MSG_SIZE); + hclge_mbx_tail_ptr_move_arq(hdev->arq); + hdev->arq.count++; + + hclgevf_mbx_task_schedule(hdev); break; default: @@ -189,3 +211,46 @@ void hclgevf_mbx_handler(struct hclgevf_dev *hdev) hclgevf_write_dev(&hdev->hw, HCLGEVF_NIC_CRQ_HEAD_REG, crq->next_to_use); } + +void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev) +{ + u16 link_status; + u16 *msg_q; + u8 duplex; + u32 speed; + u32 tail; + + /* we can safely clear it now as we are at start of the async message + * processing + */ + hdev->mbx_event_pending = false; + + tail = hdev->arq.tail; + + /* process all the async queue messages */ + while (tail != hdev->arq.head) { + msg_q = hdev->arq.msg_q[hdev->arq.head]; + + switch (msg_q[0]) { + case HCLGE_MBX_LINK_STAT_CHANGE: + link_status = le16_to_cpu(msg_q[1]); + memcpy(&speed, &msg_q[2], sizeof(speed)); + duplex = (u8)le16_to_cpu(msg_q[4]); + + /* update upper layer with new link link status */ + hclgevf_update_link_status(hdev, link_status); + hclgevf_update_speed_duplex(hdev, speed, duplex); + + break; + default: + dev_err(&hdev->pdev->dev, + "fetched unsupported(%d) message from arq\n", + msg_q[0]); + break; + } + + hclge_mbx_head_ptr_move_arq(hdev->arq); + hdev->arq.count--; + msg_q = hdev->arq.msg_q[hdev->arq.head]; + } +}