From patchwork Tue Jun 15 04:01:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 228C4C48BE5 for ; Tue, 15 Jun 2021 04:01:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0528D6008E for ; Tue, 15 Jun 2021 04:01:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230516AbhFOEDn (ORCPT ); Tue, 15 Jun 2021 00:03:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:37444 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbhFOEDe (ORCPT ); Tue, 15 Jun 2021 00:03:34 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8504461417; Tue, 15 Jun 2021 04:01:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729689; bh=a95i3ThGabJ9REwwCJQX1YLHCjeqExl0qDh644MXJBc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IEs96y3/AvKI02h1ZcBrfdAXmMB5++7odLnyBq60uur34oxM+a1vqEaEuKMRwCzTm 7log/8TH6vfxGhIH9HQ4VNzoVb+Z8n67w81+iGLwEqdBelKZaZFOEjED0O50YCzREA v/pq3WKPRdpBJ4MXa1JoY/VF3KDO/Xu+agKaxfWX4JLWo8mSKS1f6DFfcMrTLq7C5Y XT8fQP/sB23i+oCnjU0yOPHE3xUhxKkbbKF2rW3J9nPNklMXY0N/N4/Cu4NhXv6EBg WcmLfMtHOH0UMImmdTpcSs5cVDUqbI0Y6AQ7rO2AOm9y7F8OuV4WKqWsgQ3i343CYT Io27YyNQ/S/Mw== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Mark Bloch , Saeed Mahameed Subject: [net-next 03/15] net/mlx5: Change ownership model for lag Date: Mon, 14 Jun 2021 21:01:11 -0700 Message-Id: <20210615040123.287101-4-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Mark Bloch Lag is used to combine two PCI functions of the same HCA into a single logical unit. This is a core functionality and as such should be managed by the core driver. Currently this isn't the case. While we store the lag software structure inside the lower device, its lifetime (creation / destruction) is dictated by the mlx5e part. Change the ownership model so lag is tied to the lifetime of the lower level driver instead to the mlx5e part. Signed-off-by: Mark Bloch Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 4 +- .../net/ethernet/mellanox/mlx5/core/en_rep.c | 4 +- drivers/net/ethernet/mellanox/mlx5/core/lag.c | 221 +++++++++++------- drivers/net/ethernet/mellanox/mlx5/core/lag.h | 3 +- .../net/ethernet/mellanox/mlx5/core/lag_mp.c | 2 +- .../net/ethernet/mellanox/mlx5/core/main.c | 2 + .../ethernet/mellanox/mlx5/core/mlx5_core.h | 6 +- 7 files changed, 154 insertions(+), 88 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 59ee28156603..930b225dfe77 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -5114,7 +5114,7 @@ static void mlx5e_nic_enable(struct mlx5e_priv *priv) mlx5e_set_netdev_mtu_boundaries(priv); mlx5e_set_dev_port_mtu(priv); - mlx5_lag_add(mdev, netdev); + mlx5_lag_add_netdev(mdev, netdev); mlx5e_enable_async_events(priv); mlx5e_enable_blocking_events(priv); @@ -5162,7 +5162,7 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv) priv->en_trap = NULL; } mlx5e_disable_async_events(priv); - mlx5_lag_remove(mdev); + mlx5_lag_remove_netdev(mdev, priv->netdev); mlx5_vxlan_reset_to_default(mdev->vxlan); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c index 8290e0086178..2d2cc5f3b03f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c @@ -976,7 +976,7 @@ static void mlx5e_uplink_rep_enable(struct mlx5e_priv *priv) if (MLX5_CAP_GEN(mdev, uplink_follow)) mlx5_modify_vport_admin_state(mdev, MLX5_VPORT_STATE_OP_MOD_UPLINK, 0, 0, MLX5_VPORT_ADMIN_STATE_AUTO); - mlx5_lag_add(mdev, netdev); + mlx5_lag_add_netdev(mdev, netdev); priv->events_nb.notifier_call = uplink_rep_async_event; mlx5_notifier_register(mdev, &priv->events_nb); mlx5e_dcbnl_initialize(priv); @@ -1009,7 +1009,7 @@ static void mlx5e_uplink_rep_disable(struct mlx5e_priv *priv) mlx5e_dcbnl_delete_app(priv); mlx5_notifier_unregister(mdev, &priv->events_nb); mlx5e_rep_tc_disable(priv); - mlx5_lag_remove(mdev); + mlx5_lag_remove_netdev(mdev, priv->netdev); } static MLX5E_DEFINE_STATS_GRP(sw_rep, 0); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag.c index 4a4e9b228ba0..5c043c5cc403 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag.c @@ -93,6 +93,64 @@ int mlx5_cmd_destroy_vport_lag(struct mlx5_core_dev *dev) } EXPORT_SYMBOL(mlx5_cmd_destroy_vport_lag); +static int mlx5_lag_netdev_event(struct notifier_block *this, + unsigned long event, void *ptr); +static void mlx5_do_bond_work(struct work_struct *work); + +static void mlx5_ldev_free(struct kref *ref) +{ + struct mlx5_lag *ldev = container_of(ref, struct mlx5_lag, ref); + + if (ldev->nb.notifier_call) + unregister_netdevice_notifier_net(&init_net, &ldev->nb); + mlx5_lag_mp_cleanup(ldev); + cancel_delayed_work_sync(&ldev->bond_work); + destroy_workqueue(ldev->wq); + kfree(ldev); +} + +static void mlx5_ldev_put(struct mlx5_lag *ldev) +{ + kref_put(&ldev->ref, mlx5_ldev_free); +} + +static void mlx5_ldev_get(struct mlx5_lag *ldev) +{ + kref_get(&ldev->ref); +} + +static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev) +{ + struct mlx5_lag *ldev; + int err; + + ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); + if (!ldev) + return NULL; + + ldev->wq = create_singlethread_workqueue("mlx5_lag"); + if (!ldev->wq) { + kfree(ldev); + return NULL; + } + + kref_init(&ldev->ref); + INIT_DELAYED_WORK(&ldev->bond_work, mlx5_do_bond_work); + + ldev->nb.notifier_call = mlx5_lag_netdev_event; + if (register_netdevice_notifier_net(&init_net, &ldev->nb)) { + ldev->nb.notifier_call = NULL; + mlx5_core_err(dev, "Failed to register LAG netdev notifier\n"); + } + + err = mlx5_lag_mp_init(ldev); + if (err) + mlx5_core_err(dev, "Failed to init multipath lag err=%d\n", + err); + + return ldev; +} + int mlx5_lag_dev_get_netdev_idx(struct mlx5_lag *ldev, struct net_device *ndev) { @@ -511,55 +569,52 @@ static int mlx5_lag_netdev_event(struct notifier_block *this, return NOTIFY_DONE; } -static struct mlx5_lag *mlx5_lag_dev_alloc(void) +static void mlx5_ldev_add_netdev(struct mlx5_lag *ldev, + struct mlx5_core_dev *dev, + struct net_device *netdev) { - struct mlx5_lag *ldev; - - ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); - if (!ldev) - return NULL; - - ldev->wq = create_singlethread_workqueue("mlx5_lag"); - if (!ldev->wq) { - kfree(ldev); - return NULL; - } + unsigned int fn = PCI_FUNC(dev->pdev->devfn); - INIT_DELAYED_WORK(&ldev->bond_work, mlx5_do_bond_work); + if (fn >= MLX5_MAX_PORTS) + return; - return ldev; + spin_lock(&lag_lock); + ldev->pf[fn].netdev = netdev; + ldev->tracker.netdev_state[fn].link_up = 0; + ldev->tracker.netdev_state[fn].tx_enabled = 0; + spin_unlock(&lag_lock); } -static void mlx5_lag_dev_free(struct mlx5_lag *ldev) +static void mlx5_ldev_remove_netdev(struct mlx5_lag *ldev, + struct net_device *netdev) { - destroy_workqueue(ldev->wq); - kfree(ldev); + int i; + + spin_lock(&lag_lock); + for (i = 0; i < MLX5_MAX_PORTS; i++) { + if (ldev->pf[i].netdev == netdev) { + ldev->pf[i].netdev = NULL; + break; + } + } + spin_unlock(&lag_lock); } -static int mlx5_lag_dev_add_pf(struct mlx5_lag *ldev, - struct mlx5_core_dev *dev, - struct net_device *netdev) +static void mlx5_ldev_add_mdev(struct mlx5_lag *ldev, + struct mlx5_core_dev *dev) { unsigned int fn = PCI_FUNC(dev->pdev->devfn); if (fn >= MLX5_MAX_PORTS) - return -EPERM; - - spin_lock(&lag_lock); - ldev->pf[fn].dev = dev; - ldev->pf[fn].netdev = netdev; - ldev->tracker.netdev_state[fn].link_up = 0; - ldev->tracker.netdev_state[fn].tx_enabled = 0; + return; + ldev->pf[fn].dev = dev; dev->priv.lag = ldev; - - spin_unlock(&lag_lock); - - return fn; } -static void mlx5_lag_dev_remove_pf(struct mlx5_lag *ldev, - struct mlx5_core_dev *dev) +/* Must be called with intf_mutex held */ +static void mlx5_ldev_remove_mdev(struct mlx5_lag *ldev, + struct mlx5_core_dev *dev) { int i; @@ -570,19 +625,15 @@ static void mlx5_lag_dev_remove_pf(struct mlx5_lag *ldev, if (i == MLX5_MAX_PORTS) return; - spin_lock(&lag_lock); - memset(&ldev->pf[i], 0, sizeof(*ldev->pf)); - + ldev->pf[i].dev = NULL; dev->priv.lag = NULL; - spin_unlock(&lag_lock); } /* Must be called with intf_mutex held */ -void mlx5_lag_add(struct mlx5_core_dev *dev, struct net_device *netdev) +static void __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev) { struct mlx5_lag *ldev = NULL; struct mlx5_core_dev *tmp_dev; - int i, err; if (!MLX5_CAP_GEN(dev, vport_group_manager) || !MLX5_CAP_GEN(dev, lag_master) || @@ -594,67 +645,77 @@ void mlx5_lag_add(struct mlx5_core_dev *dev, struct net_device *netdev) ldev = tmp_dev->priv.lag; if (!ldev) { - ldev = mlx5_lag_dev_alloc(); + ldev = mlx5_lag_dev_alloc(dev); if (!ldev) { mlx5_core_err(dev, "Failed to alloc lag dev\n"); return; } + } else { + mlx5_ldev_get(ldev); } - if (mlx5_lag_dev_add_pf(ldev, dev, netdev) < 0) - return; + mlx5_ldev_add_mdev(ldev, dev); - for (i = 0; i < MLX5_MAX_PORTS; i++) - if (!ldev->pf[i].dev) - break; + return; +} - if (i >= MLX5_MAX_PORTS) - ldev->flags |= MLX5_LAG_FLAG_READY; +void mlx5_lag_remove_mdev(struct mlx5_core_dev *dev) +{ + struct mlx5_lag *ldev; - if (!ldev->nb.notifier_call) { - ldev->nb.notifier_call = mlx5_lag_netdev_event; - if (register_netdevice_notifier_net(&init_net, &ldev->nb)) { - ldev->nb.notifier_call = NULL; - mlx5_core_err(dev, "Failed to register LAG netdev notifier\n"); - } - } + ldev = mlx5_lag_dev(dev); + if (!ldev) + return; - err = mlx5_lag_mp_init(ldev); - if (err) - mlx5_core_err(dev, "Failed to init multipath lag err=%d\n", - err); + mlx5_dev_list_lock(); + mlx5_ldev_remove_mdev(ldev, dev); + mlx5_dev_list_unlock(); + mlx5_ldev_put(ldev); +} + +void mlx5_lag_add_mdev(struct mlx5_core_dev *dev) +{ + mlx5_dev_list_lock(); + __mlx5_lag_dev_add_mdev(dev); + mlx5_dev_list_unlock(); } /* Must be called with intf_mutex held */ -void mlx5_lag_remove(struct mlx5_core_dev *dev) +void mlx5_lag_remove_netdev(struct mlx5_core_dev *dev, + struct net_device *netdev) { struct mlx5_lag *ldev; - int i; - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); if (!ldev) return; if (__mlx5_lag_is_active(ldev)) mlx5_disable_lag(ldev); - mlx5_lag_dev_remove_pf(ldev, dev); - + mlx5_ldev_remove_netdev(ldev, netdev); ldev->flags &= ~MLX5_LAG_FLAG_READY; +} + +/* Must be called with intf_mutex held */ +void mlx5_lag_add_netdev(struct mlx5_core_dev *dev, + struct net_device *netdev) +{ + struct mlx5_lag *ldev; + int i; + + ldev = mlx5_lag_dev(dev); + if (!ldev) + return; + + mlx5_ldev_add_netdev(ldev, dev, netdev); for (i = 0; i < MLX5_MAX_PORTS; i++) - if (ldev->pf[i].dev) + if (!ldev->pf[i].dev) break; - if (i == MLX5_MAX_PORTS) { - if (ldev->nb.notifier_call) { - unregister_netdevice_notifier_net(&init_net, &ldev->nb); - ldev->nb.notifier_call = NULL; - } - mlx5_lag_mp_cleanup(ldev); - cancel_delayed_work_sync(&ldev->bond_work); - mlx5_lag_dev_free(ldev); - } + if (i >= MLX5_MAX_PORTS) + ldev->flags |= MLX5_LAG_FLAG_READY; } bool mlx5_lag_is_roce(struct mlx5_core_dev *dev) @@ -663,7 +724,7 @@ bool mlx5_lag_is_roce(struct mlx5_core_dev *dev) bool res; spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); res = ldev && __mlx5_lag_is_roce(ldev); spin_unlock(&lag_lock); @@ -677,7 +738,7 @@ bool mlx5_lag_is_active(struct mlx5_core_dev *dev) bool res; spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); res = ldev && __mlx5_lag_is_active(ldev); spin_unlock(&lag_lock); @@ -691,7 +752,7 @@ bool mlx5_lag_is_sriov(struct mlx5_core_dev *dev) bool res; spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); res = ldev && __mlx5_lag_is_sriov(ldev); spin_unlock(&lag_lock); @@ -704,7 +765,7 @@ void mlx5_lag_update(struct mlx5_core_dev *dev) struct mlx5_lag *ldev; mlx5_dev_list_lock(); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); if (!ldev) goto unlock; @@ -720,7 +781,7 @@ struct net_device *mlx5_lag_get_roce_netdev(struct mlx5_core_dev *dev) struct mlx5_lag *ldev; spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); if (!(ldev && __mlx5_lag_is_roce(ldev))) goto unlock; @@ -749,7 +810,7 @@ u8 mlx5_lag_get_slave_port(struct mlx5_core_dev *dev, u8 port = 0; spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); if (!(ldev && __mlx5_lag_is_roce(ldev))) goto unlock; @@ -785,7 +846,7 @@ int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev, memset(values, 0, sizeof(*values) * num_counters); spin_lock(&lag_lock); - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); if (ldev && __mlx5_lag_is_active(ldev)) { num_ports = MLX5_MAX_PORTS; mdev[MLX5_LAG_P1] = ldev->pf[MLX5_LAG_P1].dev; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag.h b/drivers/net/ethernet/mellanox/mlx5/core/lag.h index 8d8cf2d0bc6d..191392c37558 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag.h @@ -40,6 +40,7 @@ struct lag_tracker { struct mlx5_lag { u8 flags; u8 v2p_map[MLX5_MAX_PORTS]; + struct kref ref; struct lag_func pf[MLX5_MAX_PORTS]; struct lag_tracker tracker; struct workqueue_struct *wq; @@ -49,7 +50,7 @@ struct mlx5_lag { }; static inline struct mlx5_lag * -mlx5_lag_dev_get(struct mlx5_core_dev *dev) +mlx5_lag_dev(struct mlx5_core_dev *dev) { return dev->priv.lag; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c b/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c index fd6196b5e163..c4bf8b679541 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c @@ -28,7 +28,7 @@ bool mlx5_lag_is_multipath(struct mlx5_core_dev *dev) struct mlx5_lag *ldev; bool res; - ldev = mlx5_lag_dev_get(dev); + ldev = mlx5_lag_dev(dev); res = ldev && __mlx5_lag_is_multipath(ldev); return res; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index a1d67bd7fb43..310518fabf77 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1185,6 +1185,7 @@ static int mlx5_load(struct mlx5_core_dev *dev) } mlx5_sf_dev_table_create(dev); + mlx5_lag_add_mdev(dev); return 0; @@ -1219,6 +1220,7 @@ static int mlx5_load(struct mlx5_core_dev *dev) static void mlx5_unload(struct mlx5_core_dev *dev) { + mlx5_lag_remove_mdev(dev); mlx5_sf_dev_table_destroy(dev); mlx5_sriov_detach(dev); mlx5_ec_cleanup(dev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index a22b706eebd3..dd95aa6eb2f8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -164,8 +164,10 @@ int mlx5_query_mcam_reg(struct mlx5_core_dev *dev, u32 *mcap, u8 feature_group, int mlx5_query_qcam_reg(struct mlx5_core_dev *mdev, u32 *qcam, u8 feature_group, u8 access_reg_group); -void mlx5_lag_add(struct mlx5_core_dev *dev, struct net_device *netdev); -void mlx5_lag_remove(struct mlx5_core_dev *dev); +void mlx5_lag_add_netdev(struct mlx5_core_dev *dev, struct net_device *netdev); +void mlx5_lag_remove_netdev(struct mlx5_core_dev *dev, struct net_device *netdev); +void mlx5_lag_add_mdev(struct mlx5_core_dev *dev); +void mlx5_lag_remove_mdev(struct mlx5_core_dev *dev); int mlx5_irq_table_init(struct mlx5_core_dev *dev); void mlx5_irq_table_cleanup(struct mlx5_core_dev *dev); From patchwork Tue Jun 15 04:01:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9592C48BE8 for ; Tue, 15 Jun 2021 04:01:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8BA646141F for ; Tue, 15 Jun 2021 04:01:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230283AbhFOEDk (ORCPT ); Tue, 15 Jun 2021 00:03:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:37452 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229494AbhFOEDe (ORCPT ); Tue, 15 Jun 2021 00:03:34 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 098106141D; Tue, 15 Jun 2021 04:01:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729690; bh=zUrxgzG3wB9wVPjgkQXhvajzUqe5iIOKWiu5i74jTx0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N6T3vDVs/RW5fQhEH1IXSH46a9G+OPGw0PX5OR5rPQ389PWqkQGqkWzjQns3ph7iP 6KoDct/4CDWwnmpO2eIR1roPO1nG2fA1M3N6RPo7QTtgYA+vzk6s6AZODEpO+DuJSu EK1u7CDpf5awDEXVNBSXgB//TQyRfU+NLaOh8UYWj+z+mRbzL8DTouywVjDItL637g ktC1dfYlYK+1HKIUiNH5ZU0V6aHOlAyKnYFbFlEApufQdQZzBWiVd02WmcUWCgskGv LwNG5GMpjTEKUQAwfqMak24aEX3NIYGul1yLCj7t6ac4waVh3sD/XW6dYFvsNwD9Xa MzuXOeuMmR9iw== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Saeed Mahameed Subject: [net-next 04/15] net/mlx5: Delay IRQ destruction till all users are gone Date: Mon, 14 Jun 2021 21:01:12 -0700 Message-Id: <20210615040123.287101-5-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Leon Romanovsky Shared IRQ are consumed by multiple EQ users and in order to properly initialize and later release such IRQs, we add kref counting of IRQ structure. Signed-off-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 55 ++++++++++++------- 1 file changed, 36 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index c3373fb1cd7f..0e65ac3301c5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -16,6 +16,8 @@ struct mlx5_irq { struct atomic_notifier_head nh; cpumask_var_t mask; char name[MLX5_MAX_IRQ_NAME]; + struct kref kref; + int irqn; }; struct mlx5_irq_table { @@ -146,13 +148,35 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int function_id, return ret; } +static void irq_release(struct kref *kref) +{ + struct mlx5_irq *irq = container_of(kref, struct mlx5_irq, kref); + + free_irq(irq->irqn, &irq->nh); +} + +static void irq_put(struct mlx5_irq *irq) +{ + kref_put(&irq->kref, irq_release); +} + int mlx5_irq_attach_nb(struct mlx5_irq_table *irq_table, int vecidx, struct notifier_block *nb) { struct mlx5_irq *irq; + int err; irq = &irq_table->irq[vecidx]; - return atomic_notifier_chain_register(&irq->nh, nb); + err = kref_get_unless_zero(&irq->kref); + if (WARN_ON_ONCE(!err)) + /* Something very bad happens here, we are enabling EQ + * on non-existing IRQ. + */ + return -ENOENT; + err = atomic_notifier_chain_register(&irq->nh, nb); + if (err) + irq_put(irq); + return err; } int mlx5_irq_detach_nb(struct mlx5_irq_table *irq_table, int vecidx, @@ -161,6 +185,7 @@ int mlx5_irq_detach_nb(struct mlx5_irq_table *irq_table, int vecidx, struct mlx5_irq *irq; irq = &irq_table->irq[vecidx]; + irq_put(irq); return atomic_notifier_chain_unregister(&irq->nh, nb); } @@ -189,28 +214,26 @@ static int request_irqs(struct mlx5_core_dev *dev, int nvec) for (i = 0; i < nvec; i++) { struct mlx5_irq *irq = mlx5_irq_get(dev, i); - int irqn = pci_irq_vector(dev->pdev, i); + irq->irqn = pci_irq_vector(dev->pdev, i); irq_set_name(name, i); ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh); snprintf(irq->name, MLX5_MAX_IRQ_NAME, "%s@pci:%s", name, pci_name(dev->pdev)); - err = request_irq(irqn, mlx5_irq_int_handler, 0, irq->name, + err = request_irq(irq->irqn, mlx5_irq_int_handler, 0, irq->name, &irq->nh); if (err) { mlx5_core_err(dev, "Failed to request irq\n"); goto err_request_irq; } + kref_init(&irq->kref); } return 0; err_request_irq: - while (i--) { - struct mlx5_irq *irq = mlx5_irq_get(dev, i); - int irqn = pci_irq_vector(dev->pdev, i); + while (i--) + irq_put(mlx5_irq_get(dev, i)); - free_irq(irqn, &irq->nh); - } return err; } @@ -264,10 +287,8 @@ static int set_comp_irq_affinity_hint(struct mlx5_core_dev *mdev, int i) { int vecidx = MLX5_IRQ_VEC_COMP_BASE + i; struct mlx5_irq *irq; - int irqn; irq = mlx5_irq_get(mdev, vecidx); - irqn = pci_irq_vector(mdev->pdev, vecidx); if (!zalloc_cpumask_var(&irq->mask, GFP_KERNEL)) { mlx5_core_warn(mdev, "zalloc_cpumask_var failed"); return -ENOMEM; @@ -276,9 +297,9 @@ static int set_comp_irq_affinity_hint(struct mlx5_core_dev *mdev, int i) cpumask_set_cpu(cpumask_local_spread(i, mdev->priv.numa_node), irq->mask); if (IS_ENABLED(CONFIG_SMP) && - irq_set_affinity_hint(irqn, irq->mask)) + irq_set_affinity_hint(irq->irqn, irq->mask)) mlx5_core_warn(mdev, "irq_set_affinity_hint failed, irq 0x%.4x", - irqn); + irq->irqn); return 0; } @@ -287,11 +308,9 @@ static void clear_comp_irq_affinity_hint(struct mlx5_core_dev *mdev, int i) { int vecidx = MLX5_IRQ_VEC_COMP_BASE + i; struct mlx5_irq *irq; - int irqn; irq = mlx5_irq_get(mdev, vecidx); - irqn = pci_irq_vector(mdev->pdev, vecidx); - irq_set_affinity_hint(irqn, NULL); + irq_set_affinity_hint(irq->irqn, NULL); free_cpumask_var(irq->mask); } @@ -344,8 +363,7 @@ static void unrequest_irqs(struct mlx5_core_dev *dev) int i; for (i = 0; i < table->nvec; i++) - free_irq(pci_irq_vector(dev->pdev, i), - &mlx5_irq_get(dev, i)->nh); + irq_put(mlx5_irq_get(dev, i)); } int mlx5_irq_table_create(struct mlx5_core_dev *dev) @@ -422,8 +440,7 @@ void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) irq_clear_rmap(dev); clear_comp_irqs_affinity_hints(dev); for (i = 0; i < table->nvec; i++) - free_irq(pci_irq_vector(dev->pdev, i), - &mlx5_irq_get(dev, i)->nh); + irq_release(&mlx5_irq_get(dev, i)->kref); pci_free_irq_vectors(dev->pdev); kfree(table->irq); } From patchwork Tue Jun 15 04:01:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF00C49361 for ; Tue, 15 Jun 2021 04:01:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D7329600D4 for ; Tue, 15 Jun 2021 04:01:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230504AbhFOEDq (ORCPT ); Tue, 15 Jun 2021 00:03:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:37452 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbhFOEDe (ORCPT ); Tue, 15 Jun 2021 00:03:34 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id D2EE061421; Tue, 15 Jun 2021 04:01:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729691; bh=Y1ADodKao4F4CW8Iv6/wX6cYD9CRKhYE1iSkIlFpO1U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K2C9PzMJHdnheg905SiP4PM/oakie69MFD0QQ4nMLO7kagSt1x7l4FBSr8CrdfpTK Xav90bBWGUjBeUgFxVs4m9o+16M7TRhEhL4Oyp58lI/kynwZj99W0+90ttA7yZ9XRc kLZLBbaJs6+WTLxPJydKWYRSc7SqvcSl8juWTRnB02rp5Vqe8FceCdrJGJZ9YWeJpu sWsHGG9OCqfMY7MJbbbvOF0b6HZ1jkTHQuhaAD7ngAE5E7/4u7lAVSAxOTn8YzpfbA AsVSI656mWNC0RDVwb+PE7mwAl+AmIQET53ilpgJwkKHMPT7ysX8bFsblV9A+RDUBn 0l4mcOTT86XAA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Shay Drory , Saeed Mahameed Subject: [net-next 06/15] net/mlx5: Provide cpumask at EQ creation phase Date: Mon, 14 Jun 2021 21:01:14 -0700 Message-Id: <20210615040123.287101-7-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Leon Romanovsky The users of EQ are running their code on different CPUs and with various affinity patterns. Move the cpumask setting close to their actual usage. Signed-off-by: Leon Romanovsky Reviewed-by: Shay Drory Signed-off-by: Saeed Mahameed --- drivers/infiniband/hw/mlx5/odp.c | 5 + drivers/net/ethernet/mellanox/mlx5/core/eq.c | 27 +++-- .../ethernet/mellanox/mlx5/core/mlx5_irq.h | 3 +- .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 103 ++++-------------- include/linux/mlx5/eq.h | 1 + 5 files changed, 49 insertions(+), 90 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c index 782b2af8f211..8f88b044ccbc 100644 --- a/drivers/infiniband/hw/mlx5/odp.c +++ b/drivers/infiniband/hw/mlx5/odp.c @@ -1564,7 +1564,12 @@ int mlx5r_odp_create_eq(struct mlx5_ib_dev *dev, struct mlx5_ib_pf_eq *eq) .nent = MLX5_IB_NUM_PF_EQE, }; param.mask[0] = 1ull << MLX5_EVENT_TYPE_PAGE_FAULT; + if (!zalloc_cpumask_var(¶m.affinity, GFP_KERNEL)) { + err = -ENOMEM; + goto err_wq; + } eq->core = mlx5_eq_create_generic(dev->mdev, ¶m); + free_cpumask_var(param.affinity); if (IS_ERR(eq->core)) { err = PTR_ERR(eq->core); goto err_wq; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index 7e7bbed3763d..5a88887c1a58 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -310,7 +310,7 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq, mlx5_init_fbc(eq->frag_buf.frags, log_eq_stride, log_eq_size, &eq->fbc); init_eq_buf(eq); - eq->irq = mlx5_irq_request(dev, vecidx); + eq->irq = mlx5_irq_request(dev, vecidx, param->affinity); if (IS_ERR(eq->irq)) { err = PTR_ERR(eq->irq); goto err_buf; @@ -621,8 +621,11 @@ setup_async_eq(struct mlx5_core_dev *dev, struct mlx5_eq_async *eq, eq->irq_nb.notifier_call = mlx5_eq_async_int; spin_lock_init(&eq->lock); + if (!zalloc_cpumask_var(¶m->affinity, GFP_KERNEL)) + return -ENOMEM; err = create_async_eq(dev, &eq->core, param); + free_cpumask_var(param->affinity); if (err) { mlx5_core_warn(dev, "failed to create %s EQ %d\n", name, err); return err; @@ -740,6 +743,9 @@ mlx5_eq_create_generic(struct mlx5_core_dev *dev, struct mlx5_eq *eq = kvzalloc(sizeof(*eq), GFP_KERNEL); int err; + if (!param->affinity) + return ERR_PTR(-EINVAL); + if (!eq) return ERR_PTR(-ENOMEM); @@ -850,16 +856,21 @@ static int create_comp_eqs(struct mlx5_core_dev *dev) .irq_index = vecidx, .nent = nent, }; - err = create_map_eq(dev, &eq->core, ¶m); - if (err) { - kfree(eq); - goto clean; + + if (!zalloc_cpumask_var(¶m.affinity, GFP_KERNEL)) { + err = -ENOMEM; + goto clean_eq; } + cpumask_set_cpu(cpumask_local_spread(i, dev->priv.numa_node), + param.affinity); + err = create_map_eq(dev, &eq->core, ¶m); + free_cpumask_var(param.affinity); + if (err) + goto clean_eq; err = mlx5_eq_enable(dev, &eq->core, &eq->irq_nb); if (err) { destroy_unmap_eq(dev, &eq->core); - kfree(eq); - goto clean; + goto clean_eq; } mlx5_core_dbg(dev, "allocated completion EQN %d\n", eq->core.eqn); @@ -868,6 +879,8 @@ static int create_comp_eqs(struct mlx5_core_dev *dev) } return 0; +clean_eq: + kfree(eq); clean: destroy_comp_eqs(dev); return err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h index dd138b38bf36..81bfb5f0d332 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h @@ -20,7 +20,8 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int devfn, int msix_vec_count); int mlx5_get_default_msix_vec_count(struct mlx5_core_dev *dev, int num_vfs); -struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx); +struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx, + struct cpumask *affinity); void mlx5_irq_release(struct mlx5_irq *irq); int mlx5_irq_attach_nb(struct mlx5_irq *irq, struct notifier_block *nb); int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index ecace7ca4a01..81b06b5693cd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -17,6 +17,7 @@ struct mlx5_irq { struct atomic_notifier_head nh; cpumask_var_t mask; char name[MLX5_MAX_IRQ_NAME]; + spinlock_t lock; /* protects affinity assignment */ struct kref kref; int irqn; }; @@ -153,6 +154,8 @@ static void irq_release(struct kref *kref) { struct mlx5_irq *irq = container_of(kref, struct mlx5_irq, kref); + irq_set_affinity_hint(irq->irqn, NULL); + free_cpumask_var(irq->mask); free_irq(irq->irqn, &irq->nh); } @@ -189,7 +192,8 @@ void mlx5_irq_release(struct mlx5_irq *irq) irq_put(irq); } -struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx) +struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx, + struct cpumask *affinity) { struct mlx5_irq_table *table = mlx5_irq_table_get(dev); struct mlx5_irq *irq = &table->irq[vecidx]; @@ -199,6 +203,16 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx) if (!err) return ERR_PTR(-ENOENT); + spin_lock(&irq->lock); + if (!cpumask_empty(irq->mask)) { + /* already configured */ + spin_unlock(&irq->lock); + return irq; + } + + cpumask_copy(irq->mask, affinity); + irq_set_affinity_hint(irq->irqn, irq->mask); + spin_unlock(&irq->lock); return irq; } @@ -239,6 +253,12 @@ static int request_irqs(struct mlx5_core_dev *dev, int nvec) mlx5_core_err(dev, "Failed to request irq\n"); goto err_request_irq; } + if (!zalloc_cpumask_var(&irq->mask, GFP_KERNEL)) { + mlx5_core_warn(dev, "zalloc_cpumask_var failed\n"); + err = -ENOMEM; + goto err_request_irq; + } + spin_lock_init(&irq->lock); kref_init(&irq->kref); } return 0; @@ -294,69 +314,6 @@ static int irq_set_rmap(struct mlx5_core_dev *mdev) return err; } -/* Completion IRQ vectors */ - -static int set_comp_irq_affinity_hint(struct mlx5_core_dev *mdev, int i) -{ - int vecidx = MLX5_IRQ_VEC_COMP_BASE + i; - struct mlx5_irq *irq; - - irq = mlx5_irq_get(mdev, vecidx); - if (!zalloc_cpumask_var(&irq->mask, GFP_KERNEL)) { - mlx5_core_warn(mdev, "zalloc_cpumask_var failed"); - return -ENOMEM; - } - - cpumask_set_cpu(cpumask_local_spread(i, mdev->priv.numa_node), - irq->mask); - if (IS_ENABLED(CONFIG_SMP) && - irq_set_affinity_hint(irq->irqn, irq->mask)) - mlx5_core_warn(mdev, "irq_set_affinity_hint failed, irq 0x%.4x", - irq->irqn); - - return 0; -} - -static void clear_comp_irq_affinity_hint(struct mlx5_core_dev *mdev, int i) -{ - int vecidx = MLX5_IRQ_VEC_COMP_BASE + i; - struct mlx5_irq *irq; - - irq = mlx5_irq_get(mdev, vecidx); - irq_set_affinity_hint(irq->irqn, NULL); - free_cpumask_var(irq->mask); -} - -static int set_comp_irq_affinity_hints(struct mlx5_core_dev *mdev) -{ - int nvec = mlx5_irq_get_num_comp(mdev->priv.irq_table); - int err; - int i; - - for (i = 0; i < nvec; i++) { - err = set_comp_irq_affinity_hint(mdev, i); - if (err) - goto err_out; - } - - return 0; - -err_out: - for (i--; i >= 0; i--) - clear_comp_irq_affinity_hint(mdev, i); - - return err; -} - -static void clear_comp_irqs_affinity_hints(struct mlx5_core_dev *mdev) -{ - int nvec = mlx5_irq_get_num_comp(mdev->priv.irq_table); - int i; - - for (i = 0; i < nvec; i++) - clear_comp_irq_affinity_hint(mdev, i); -} - struct cpumask * mlx5_irq_get_affinity_mask(struct mlx5_irq_table *irq_table, int vecidx) { @@ -370,15 +327,6 @@ struct cpu_rmap *mlx5_irq_get_rmap(struct mlx5_irq_table *irq_table) } #endif -static void unrequest_irqs(struct mlx5_core_dev *dev) -{ - struct mlx5_irq_table *table = dev->priv.irq_table; - int i; - - for (i = 0; i < table->nvec; i++) - irq_put(mlx5_irq_get(dev, i)); -} - int mlx5_irq_table_create(struct mlx5_core_dev *dev) { struct mlx5_priv *priv = &dev->priv; @@ -419,16 +367,8 @@ int mlx5_irq_table_create(struct mlx5_core_dev *dev) if (err) goto err_request_irqs; - err = set_comp_irq_affinity_hints(dev); - if (err) { - mlx5_core_err(dev, "Failed to alloc affinity hint cpumask\n"); - goto err_set_affinity; - } - return 0; -err_set_affinity: - unrequest_irqs(dev); err_request_irqs: irq_clear_rmap(dev); err_set_rmap: @@ -451,7 +391,6 @@ void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) * which should be called after alloc_irq but before request_irq. */ irq_clear_rmap(dev); - clear_comp_irqs_affinity_hints(dev); for (i = 0; i < table->nvec; i++) irq_release(&mlx5_irq_get(dev, i)->kref); pci_free_irq_vectors(dev->pdev); diff --git a/include/linux/mlx5/eq.h b/include/linux/mlx5/eq.h index e49d8c0d4f26..cea6ecb4b73e 100644 --- a/include/linux/mlx5/eq.h +++ b/include/linux/mlx5/eq.h @@ -16,6 +16,7 @@ struct mlx5_eq_param { u8 irq_index; int nent; u64 mask[4]; + cpumask_var_t affinity; }; struct mlx5_eq * From patchwork Tue Jun 15 04:01:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F50FC48BE8 for ; Tue, 15 Jun 2021 04:01:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 089676023E for ; Tue, 15 Jun 2021 04:01:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231146AbhFOEDt (ORCPT ); Tue, 15 Jun 2021 00:03:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:37462 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229749AbhFOEDf (ORCPT ); Tue, 15 Jun 2021 00:03:35 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id A63F861416; Tue, 15 Jun 2021 04:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729691; bh=co+bBBntoA/WzLJOwEbpvbVwu5qdqmj36w7SQmp7cdc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nyjxc+AAwb00Lk3i47oKZMo0xJDyydMLZrzVjG2s4KJC9TuXz9gZw5E/rqBzL+t9g MOS6sb5kP1e+PZ+zX3YD80qzSRwUq6pUUNCw1G/1Eu0rPq5Yr5duGFmtuu7+CoT+aT lZjj+VkN5JY6m28A8pAv2ZMxs5AkosOHn3DHhtvliQRvj4tJefUjmfTcXEEtpICQJi 1CLls3GStyonM+WYExIBZJP8dfqPZN7PEBVgFH4bzVvC4IbN6PIu7p6JxYAOUP/PI3 y5RY8baGr9bqu9y5Ny40IGE/owoCudWvV4Q58agVQngiszni2w5+5DuWo7iQO07VmA hKyF5hU0+w2Ew== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Shay Drory , Saeed Mahameed Subject: [net-next 08/15] net/mlx5: Removing rmap per IRQ Date: Mon, 14 Jun 2021 21:01:16 -0700 Message-Id: <20210615040123.287101-9-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Shay Drory In next patches, IRQs will be requested according to demand, instead of statically on driver boot. Also, currently, rmap is managed by the IRQ layer. rmap management will move out from the IRQ layer in future patches. Therefore, we want to remove the IRQ from the rmap, when IRQ is destroyed, instead of removing all the IRQs from the rmap when irq_table is destroyed. Signed-off-by: Shay Drory Reviewed-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index 81b06b5693cd..6a5a6ec0ddbf 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -154,8 +154,14 @@ static void irq_release(struct kref *kref) { struct mlx5_irq *irq = container_of(kref, struct mlx5_irq, kref); + /* free_irq requires that affinity and rmap will be cleared + * before calling it. This is why there is asymmetry with set_rmap + * which should be called after alloc_irq but before request_irq. + */ irq_set_affinity_hint(irq->irqn, NULL); free_cpumask_var(irq->mask); + /* this line is releasing this irq from the rmap */ + irq_set_affinity_notifier(irq->irqn, NULL); free_irq(irq->irqn, &irq->nh); } @@ -378,6 +384,11 @@ int mlx5_irq_table_create(struct mlx5_core_dev *dev) return err; } +static void irq_table_clear_rmap(struct mlx5_irq_table *table) +{ + cpu_rmap_put(table->rmap); +} + void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) { struct mlx5_irq_table *table = dev->priv.irq_table; @@ -386,11 +397,7 @@ void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) if (mlx5_core_is_sf(dev)) return; - /* free_irq requires that affinity and rmap will be cleared - * before calling it. This is why there is asymmetry with set_rmap - * which should be called after alloc_irq but before request_irq. - */ - irq_clear_rmap(dev); + irq_table_clear_rmap(table); for (i = 0; i < table->nvec; i++) irq_release(&mlx5_irq_get(dev, i)->kref); pci_free_irq_vectors(dev->pdev); From patchwork Tue Jun 15 04:01:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96A6EC48BE8 for ; Tue, 15 Jun 2021 04:01:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7961F6008E for ; Tue, 15 Jun 2021 04:01:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231261AbhFOEDx (ORCPT ); Tue, 15 Jun 2021 00:03:53 -0400 Received: from mail.kernel.org ([198.145.29.99]:37444 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229946AbhFOEDg (ORCPT ); Tue, 15 Jun 2021 00:03:36 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8494E6141D; Tue, 15 Jun 2021 04:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729692; bh=2EXBU977pIy0pSoyT78mm6Z8HJvDZt1pCIHR3an2VBc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cRo2q+/SmkrvAPpJJYpI8ZzNGwoDaN8FzyLUSAl8q2dynXzUIxwhkFSM8+Wd+6LQQ wPXdSzc9jDpNlfyZtBBqzInCXYe08Qey+h/K2wZTxH8WI4Qe76zocAlR+ZtiwzHWiT JeQhGXTQyoffsxkvlUYZ87tV0kIZ5S19P7pAUekVl8wyVgNdTPVV+Wsbn1otCA4eRq CnmIVijy2mzzo+VImA33DEMJlNNONdT/gntEu7evMT47zntXfjk9RJUjIZX+F9LR6n yhd1ng/U7Hpaqoby4juVY7En2bc3VlEX7zQqEInRjCx9unU2rzQbAPg7BNBzToH0Tb qabKF1xprXpkg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Shay Drory , Saeed Mahameed Subject: [net-next 10/15] net/mlx5: Moving rmap logic to EQs Date: Mon, 14 Jun 2021 21:01:18 -0700 Message-Id: <20210615040123.287101-11-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Shay Drory IRQs are being simplified in order to ease their sharing and any feature specific object will be moved to upper layer. Hence we move rmap object into eq_table. Signed-off-by: Shay Drory Reviewed-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 67 ++++++++++++++- .../ethernet/mellanox/mlx5/core/mlx5_irq.h | 1 - .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 84 +++---------------- 3 files changed, 78 insertions(+), 74 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index ef0fe499eaed..898ae3d47f20 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -58,6 +58,9 @@ struct mlx5_eq_table { struct mutex lock; /* sync async eqs creations */ int num_comp_eqs; struct mlx5_irq_table *irq_table; +#ifdef CONFIG_RFS_ACCEL + struct cpu_rmap *rmap; +#endif }; #define MLX5_ASYNC_EVENT_MASK ((1ull << MLX5_EVENT_TYPE_PATH_MIG) | \ @@ -899,7 +902,7 @@ EXPORT_SYMBOL(mlx5_comp_irq_get_affinity_mask); #ifdef CONFIG_RFS_ACCEL struct cpu_rmap *mlx5_eq_table_get_rmap(struct mlx5_core_dev *dev) { - return mlx5_irq_get_rmap(dev->priv.eq_table->irq_table); + return dev->priv.eq_table->rmap; } #endif @@ -916,12 +919,57 @@ struct mlx5_eq_comp *mlx5_eqn2comp_eq(struct mlx5_core_dev *dev, int eqn) return ERR_PTR(-ENOENT); } +static void clear_rmap(struct mlx5_core_dev *dev) +{ +#ifdef CONFIG_RFS_ACCEL + struct mlx5_eq_table *eq_table = dev->priv.eq_table; + + free_irq_cpu_rmap(eq_table->rmap); +#endif +} + +static int set_rmap(struct mlx5_core_dev *mdev) +{ + int err = 0; +#ifdef CONFIG_RFS_ACCEL + struct mlx5_eq_table *eq_table = mdev->priv.eq_table; + int vecidx; + + eq_table->rmap = alloc_irq_cpu_rmap(eq_table->num_comp_eqs); + if (!eq_table->rmap) { + err = -ENOMEM; + mlx5_core_err(mdev, "Failed to allocate cpu_rmap. err %d", err); + goto err_out; + } + + vecidx = MLX5_IRQ_VEC_COMP_BASE; + for (; vecidx < eq_table->num_comp_eqs + MLX5_IRQ_VEC_COMP_BASE; + vecidx++) { + err = irq_cpu_rmap_add(eq_table->rmap, + pci_irq_vector(mdev->pdev, vecidx)); + if (err) { + mlx5_core_err(mdev, "irq_cpu_rmap_add failed. err %d", + err); + goto err_irq_cpu_rmap_add; + } + } + return 0; + +err_irq_cpu_rmap_add: + clear_rmap(mdev); +err_out: +#endif + return err; +} + /* This function should only be called after mlx5_cmd_force_teardown_hca */ void mlx5_core_eq_free_irqs(struct mlx5_core_dev *dev) { struct mlx5_eq_table *table = dev->priv.eq_table; mutex_lock(&table->lock); /* sync with create/destroy_async_eq */ + if (!mlx5_core_is_sf(dev)) + clear_rmap(dev); mlx5_irq_table_destroy(dev); mutex_unlock(&table->lock); } @@ -951,6 +999,18 @@ int mlx5_eq_table_create(struct mlx5_core_dev *dev) goto err_async_eqs; } + if (!mlx5_core_is_sf(dev)) { + /* rmap is a mapping between irq number and queue number. + * each irq can be assign only to a single rmap. + * since SFs share IRQs, rmap mapping cannot function correctly + * for irqs that are shared for different core/netdev RX rings. + * Hence we don't allow netdev rmap for SFs + */ + err = set_rmap(dev); + if (err) + goto err_rmap; + } + err = create_comp_eqs(dev); if (err) { mlx5_core_err(dev, "Failed to create completion EQs\n"); @@ -959,6 +1019,9 @@ int mlx5_eq_table_create(struct mlx5_core_dev *dev) return 0; err_comp_eqs: + if (!mlx5_core_is_sf(dev)) + clear_rmap(dev); +err_rmap: destroy_async_eqs(dev); err_async_eqs: return err; @@ -966,6 +1029,8 @@ int mlx5_eq_table_create(struct mlx5_core_dev *dev) void mlx5_eq_table_destroy(struct mlx5_core_dev *dev) { + if (!mlx5_core_is_sf(dev)) + clear_rmap(dev); destroy_comp_eqs(dev); destroy_async_eqs(dev); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h index 81bfb5f0d332..d4be79884cb4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h @@ -12,7 +12,6 @@ int mlx5_irq_table_init(struct mlx5_core_dev *dev); void mlx5_irq_table_cleanup(struct mlx5_core_dev *dev); int mlx5_irq_table_create(struct mlx5_core_dev *dev); void mlx5_irq_table_destroy(struct mlx5_core_dev *dev); -struct cpu_rmap *mlx5_irq_get_rmap(struct mlx5_irq_table *table); int mlx5_irq_get_num_comp(struct mlx5_irq_table *table); struct mlx5_irq_table *mlx5_irq_table_get(struct mlx5_core_dev *dev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index 7d6ca2581532..149d6db9ee0e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -24,9 +24,6 @@ struct mlx5_irq { struct mlx5_irq_table { struct mlx5_irq *irq; int nvec; -#ifdef CONFIG_RFS_ACCEL - struct cpu_rmap *rmap; -#endif }; int mlx5_irq_table_init(struct mlx5_core_dev *dev) @@ -159,8 +156,6 @@ static void irq_release(struct kref *kref) */ irq_set_affinity_hint(irq->irqn, NULL); free_cpumask_var(irq->mask); - /* this line is releasing this irq from the rmap */ - irq_set_affinity_notifier(irq->irqn, NULL); free_irq(irq->irqn, &irq->nh); } @@ -210,10 +205,11 @@ static void irq_set_name(char *name, int vecidx) static int irq_request(struct mlx5_core_dev *dev, int i) { - struct mlx5_irq *irq = mlx5_irq_get(dev, i); char name[MLX5_MAX_IRQ_NAME]; + struct mlx5_irq *irq; int err; + irq = mlx5_irq_get(dev, i); irq->irqn = pci_irq_vector(dev->pdev, i); irq_set_name(name, i); ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh); @@ -223,15 +219,22 @@ static int irq_request(struct mlx5_core_dev *dev, int i) &irq->nh); if (err) { mlx5_core_err(dev, "Failed to request irq. err = %d\n", err); - return err; + goto err_req_irq; } if (!zalloc_cpumask_var(&irq->mask, GFP_KERNEL)) { mlx5_core_warn(dev, "zalloc_cpumask_var failed\n"); - free_irq(irq->irqn, &irq->nh); - return -ENOMEM; + err = -ENOMEM; + goto err_cpumask; } kref_init(&irq->kref); return 0; + +err_cpumask: + free_irq(irq->irqn, &irq->nh); +err_req_irq: + if (i != 0) + irq_set_affinity_notifier(irq->irqn, NULL); + return err; } /** @@ -271,63 +274,12 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx, return irq; } -static void irq_clear_rmap(struct mlx5_core_dev *dev) -{ -#ifdef CONFIG_RFS_ACCEL - struct mlx5_irq_table *irq_table = dev->priv.irq_table; - - free_irq_cpu_rmap(irq_table->rmap); -#endif -} - -static int irq_set_rmap(struct mlx5_core_dev *mdev) -{ - int err = 0; -#ifdef CONFIG_RFS_ACCEL - struct mlx5_irq_table *irq_table = mdev->priv.irq_table; - int num_affinity_vec; - int vecidx; - - num_affinity_vec = mlx5_irq_get_num_comp(irq_table); - irq_table->rmap = alloc_irq_cpu_rmap(num_affinity_vec); - if (!irq_table->rmap) { - err = -ENOMEM; - mlx5_core_err(mdev, "Failed to allocate cpu_rmap. err %d", err); - goto err_out; - } - - vecidx = MLX5_IRQ_VEC_COMP_BASE; - for (; vecidx < irq_table->nvec; vecidx++) { - err = irq_cpu_rmap_add(irq_table->rmap, - pci_irq_vector(mdev->pdev, vecidx)); - if (err) { - mlx5_core_err(mdev, "irq_cpu_rmap_add failed. err %d", - err); - goto err_irq_cpu_rmap_add; - } - } - return 0; - -err_irq_cpu_rmap_add: - irq_clear_rmap(mdev); -err_out: -#endif - return err; -} - struct cpumask * mlx5_irq_get_affinity_mask(struct mlx5_irq_table *irq_table, int vecidx) { return irq_table->irq[vecidx].mask; } -#ifdef CONFIG_RFS_ACCEL -struct cpu_rmap *mlx5_irq_get_rmap(struct mlx5_irq_table *irq_table) -{ - return irq_table->rmap; -} -#endif - int mlx5_irq_table_create(struct mlx5_core_dev *dev) { struct mlx5_priv *priv = &dev->priv; @@ -360,24 +312,13 @@ int mlx5_irq_table_create(struct mlx5_core_dev *dev) table->nvec = nvec; - err = irq_set_rmap(dev); - if (err) - goto err_set_rmap; - return 0; -err_set_rmap: - pci_free_irq_vectors(dev->pdev); err_free_irq: kfree(table->irq); return err; } -static void irq_table_clear_rmap(struct mlx5_irq_table *table) -{ - cpu_rmap_put(table->rmap); -} - void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) { struct mlx5_irq_table *table = dev->priv.irq_table; @@ -385,7 +326,6 @@ void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) if (mlx5_core_is_sf(dev)) return; - irq_table_clear_rmap(table); pci_free_irq_vectors(dev->pdev); kfree(table->irq); } From patchwork Tue Jun 15 04:01:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E89D6C48BE5 for ; Tue, 15 Jun 2021 04:01:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D15826008E for ; Tue, 15 Jun 2021 04:01:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231193AbhFOED5 (ORCPT ); Tue, 15 Jun 2021 00:03:57 -0400 Received: from mail.kernel.org ([198.145.29.99]:37462 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229972AbhFOEDg (ORCPT ); Tue, 15 Jun 2021 00:03:36 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 06E84600D4; Tue, 15 Jun 2021 04:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729693; bh=SlCHA+dsv2AUlstqEZWMCCTRiPi5ZBZnTVoGFCNb+z4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kUnzGzvgHBqPFzYFXzRHNPomSmG44BnJtXgWJTlj4I/6kxrqP89Kr1ZYHvyvxk6hi vC0pDEJ2dVQegfaOYo4YhwKXNXYHLh1aF0rHqQlhFZRYnn/dhQRzcltAH4ynuSt9ZW YONXxQ1YTspVWdWinblBaQ7CWY2nd7Xa43CSL+H8UA/lUAaTNerbx2UhwE8pqaqp6j vT9IhYtk2u2Sl4I2GECUJerV01uQBSIRX/f0QfIad62pMnqeqHF9WkaZrBOfOkY91r LwQIOvHKvyk1QdqLo9qsANEVXw4ej03wOaYtTIKBGTWoNMo2h1FUZpf+rA53SZyaZ/ CV5norEv2riWA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Shay Drory , Saeed Mahameed Subject: [net-next 11/15] net/mlx5: Change IRQ storage logic from static to dynamic Date: Mon, 14 Jun 2021 21:01:19 -0700 Message-Id: <20210615040123.287101-12-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Shay Drory Store newly created IRQs in the xarray DB instead of a static array, so we will be able to store only IRQs which are being used. Signed-off-by: Shay Drory Reviewed-by: Leon Romanovsky Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 12 ++- .../ethernet/mellanox/mlx5/core/mlx5_irq.h | 3 +- .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 79 +++++++++++-------- 3 files changed, 58 insertions(+), 36 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index 898ae3d47f20..96649dbcef39 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -892,10 +892,16 @@ EXPORT_SYMBOL(mlx5_comp_vectors_count); struct cpumask * mlx5_comp_irq_get_affinity_mask(struct mlx5_core_dev *dev, int vector) { - int vecidx = vector + MLX5_IRQ_VEC_COMP_BASE; + struct mlx5_eq_table *table = dev->priv.eq_table; + struct mlx5_eq_comp *eq, *n; + int i = 0; + + list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) { + if (i++ == vector) + break; + } - return mlx5_irq_get_affinity_mask(dev->priv.eq_table->irq_table, - vecidx); + return mlx5_irq_get_affinity_mask(eq->core.irq); } EXPORT_SYMBOL(mlx5_comp_irq_get_affinity_mask); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h index d4be79884cb4..63b33cd37f7c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h @@ -24,7 +24,6 @@ struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx, void mlx5_irq_release(struct mlx5_irq *irq); int mlx5_irq_attach_nb(struct mlx5_irq *irq, struct notifier_block *nb); int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb); -struct cpumask * -mlx5_irq_get_affinity_mask(struct mlx5_irq_table *irq_table, int vecidx); +struct cpumask *mlx5_irq_get_affinity_mask(struct mlx5_irq *irq); #endif /* __MLX5_IRQ_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index 149d6db9ee0e..a6acc78bd1a3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -14,15 +14,17 @@ #define MLX5_MAX_IRQ_NAME (32) struct mlx5_irq { + u32 index; struct atomic_notifier_head nh; cpumask_var_t mask; char name[MLX5_MAX_IRQ_NAME]; struct kref kref; int irqn; + struct mlx5_irq_table *table; }; struct mlx5_irq_table { - struct mlx5_irq *irq; + struct xarray irqs; int nvec; }; @@ -54,13 +56,6 @@ int mlx5_irq_get_num_comp(struct mlx5_irq_table *table) return table->nvec - MLX5_IRQ_VEC_COMP_BASE; } -static struct mlx5_irq *mlx5_irq_get(struct mlx5_core_dev *dev, int vecidx) -{ - struct mlx5_irq_table *irq_table = mlx5_irq_table_get(dev); - - return &irq_table->irq[vecidx]; -} - /** * mlx5_get_default_msix_vec_count - Get the default number of MSI-X vectors * to be ssigned to each VF. @@ -149,7 +144,9 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int function_id, static void irq_release(struct kref *kref) { struct mlx5_irq *irq = container_of(kref, struct mlx5_irq, kref); + struct mlx5_irq_table *table = irq->table; + xa_erase(&table->irqs, irq->index); /* free_irq requires that affinity and rmap will be cleared * before calling it. This is why there is asymmetry with set_rmap * which should be called after alloc_irq but before request_irq. @@ -157,6 +154,7 @@ static void irq_release(struct kref *kref) irq_set_affinity_hint(irq->irqn, NULL); free_cpumask_var(irq->mask); free_irq(irq->irqn, &irq->nh); + kfree(irq); } static void irq_put(struct mlx5_irq *irq) @@ -203,13 +201,17 @@ static void irq_set_name(char *name, int vecidx) vecidx - MLX5_IRQ_VEC_COMP_BASE); } -static int irq_request(struct mlx5_core_dev *dev, int i) +static struct mlx5_irq *irq_request(struct mlx5_core_dev *dev, int i) { + struct mlx5_irq_table *table = mlx5_irq_table_get(dev); char name[MLX5_MAX_IRQ_NAME]; + struct xa_limit xa_num_irqs; struct mlx5_irq *irq; int err; - irq = mlx5_irq_get(dev, i); + irq = kzalloc(sizeof(*irq), GFP_KERNEL); + if (!irq) + return ERR_PTR(-ENOMEM); irq->irqn = pci_irq_vector(dev->pdev, i); irq_set_name(name, i); ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh); @@ -226,15 +228,25 @@ static int irq_request(struct mlx5_core_dev *dev, int i) err = -ENOMEM; goto err_cpumask; } + xa_num_irqs.min = 0; + xa_num_irqs.max = table->nvec; + err = xa_alloc(&table->irqs, &irq->index, irq, xa_num_irqs, + GFP_KERNEL); + if (err) { + mlx5_core_err(dev, "Failed to alloc xa entry for irq(%u). err = %d\n", + irq->index, err); + goto err_xa; + } + irq->table = table; kref_init(&irq->kref); - return 0; - + return irq; +err_xa: + free_cpumask_var(irq->mask); err_cpumask: free_irq(irq->irqn, &irq->nh); err_req_irq: - if (i != 0) - irq_set_affinity_notifier(irq->irqn, NULL); - return err; + kfree(irq); + return ERR_PTR(err); } /** @@ -259,25 +271,25 @@ void mlx5_irq_release(struct mlx5_irq *irq) struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, int vecidx, struct cpumask *affinity) { - struct mlx5_irq_table *table = mlx5_irq_table_get(dev); - struct mlx5_irq *irq = &table->irq[vecidx]; - int ret; + struct mlx5_irq_table *irq_table = mlx5_irq_table_get(dev); + struct mlx5_irq *irq; - ret = kref_get_unless_zero(&irq->kref); - if (ret) + irq = xa_load(&irq_table->irqs, vecidx); + if (irq) { + kref_get(&irq->kref); + return irq; + } + irq = irq_request(dev, vecidx); + if (IS_ERR(irq)) return irq; - ret = irq_request(dev, vecidx); - if (ret) - return ERR_PTR(ret); cpumask_copy(irq->mask, affinity); irq_set_affinity_hint(irq->irqn, irq->mask); return irq; } -struct cpumask * -mlx5_irq_get_affinity_mask(struct mlx5_irq_table *irq_table, int vecidx) +struct cpumask *mlx5_irq_get_affinity_mask(struct mlx5_irq *irq) { - return irq_table->irq[vecidx].mask; + return irq->mask; } int mlx5_irq_table_create(struct mlx5_core_dev *dev) @@ -299,9 +311,7 @@ int mlx5_irq_table_create(struct mlx5_core_dev *dev) if (nvec <= MLX5_IRQ_VEC_COMP_BASE) return -ENOMEM; - table->irq = kcalloc(nvec, sizeof(*table->irq), GFP_KERNEL); - if (!table->irq) - return -ENOMEM; + xa_init_flags(&table->irqs, XA_FLAGS_ALLOC); nvec = pci_alloc_irq_vectors(dev->pdev, MLX5_IRQ_VEC_COMP_BASE + 1, nvec, PCI_IRQ_MSIX); @@ -315,19 +325,26 @@ int mlx5_irq_table_create(struct mlx5_core_dev *dev) return 0; err_free_irq: - kfree(table->irq); + xa_destroy(&table->irqs); return err; } void mlx5_irq_table_destroy(struct mlx5_core_dev *dev) { struct mlx5_irq_table *table = dev->priv.irq_table; + struct mlx5_irq *irq; + unsigned long index; if (mlx5_core_is_sf(dev)) return; + /* There are cases where IRQs still will be in used when we reaching + * to here. Hence, making sure all the irqs are realeased. + */ + xa_for_each(&table->irqs, index, irq) + irq_release(&irq->kref); pci_free_irq_vectors(dev->pdev); - kfree(table->irq); + xa_destroy(&table->irqs); } struct mlx5_irq_table *mlx5_irq_table_get(struct mlx5_core_dev *dev) From patchwork Tue Jun 15 04:01:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 460939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3B1CC49361 for ; Tue, 15 Jun 2021 04:02:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91DDD6008E for ; Tue, 15 Jun 2021 04:02:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231202AbhFOEEE (ORCPT ); Tue, 15 Jun 2021 00:04:04 -0400 Received: from mail.kernel.org ([198.145.29.99]:37542 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229992AbhFOEDh (ORCPT ); Tue, 15 Jun 2021 00:03:37 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id F13F8613CC; Tue, 15 Jun 2021 04:01:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623729694; bh=IU6PuN01K7aoGfDTMhdQ2N5edD6IwpPaCr/2J8e0KQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J0dSzwx5MmdvjhnS6iozwbx4oN6iT0ycNxsgTTI51cRVjh/INvNahUn50b12EmuWl PYxHEkb6oDTsMor9vAmLUChXx0bKJI1I4BR1VtU3CWaFlY2GrshcKWkuW/RtgzQk0G tVk1XOWoWUuRs+5UQlrFsR6XHCMkyvhugC/eIiUek0FKFPpiGhaot50YJzW8+R54Y7 XKb+3qJPfrUvUMmbaDUjCEIkhY26adc+Y4/rxiyzOQxYS4Spjv+nlrrABQhYHfKk6h AOnqIHjO2Kz5TjyjetLMxtZ+ffPUJLq9iejWt4PfikFHSj4RMUGkDvLrGHCvIufzuo C8v0PFt+95aRw== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Leon Romanovsky , Shay Drory , Maor Gottlieb , Saeed Mahameed Subject: [net-next 13/15] net/mlx5: Enlarge interrupt field in CREATE_EQ Date: Mon, 14 Jun 2021 21:01:21 -0700 Message-Id: <20210615040123.287101-14-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210615040123.287101-1-saeed@kernel.org> References: <20210615040123.287101-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Shay Drory FW is now supporting more than 256 MSI-X per PF (up to 2K). Hence, enlarge interrupt field in CREATE_EQ to make use of the new MSI-X's. Signed-off-by: Shay Drory Reviewed-by: Maor Gottlieb Signed-off-by: Saeed Mahameed --- include/linux/mlx5/mlx5_ifc.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 057db0eaf195..2d1ed78289ff 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -3806,8 +3806,8 @@ struct mlx5_ifc_eqc_bits { u8 reserved_at_80[0x20]; - u8 reserved_at_a0[0x18]; - u8 intr[0x8]; + u8 reserved_at_a0[0x14]; + u8 intr[0xc]; u8 reserved_at_c0[0x3]; u8 log_page_size[0x5];