From patchwork Mon Aug 16 23:22:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 497856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EFE1C4338F for ; Mon, 16 Aug 2021 23:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83C2B60EBD for ; Mon, 16 Aug 2021 23:22:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235108AbhHPXXX (ORCPT ); Mon, 16 Aug 2021 19:23:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:51404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234432AbhHPXXH (ORCPT ); Mon, 16 Aug 2021 19:23:07 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1103860F38; Mon, 16 Aug 2021 23:22:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1629156155; bh=fWp3DU7/Q2wSKNInrN9A5rkm+BLu4O8GyOG54rLEMFI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TLA3kxLVNComULd7edNAuCyhJN5t3IRmY6xZwojE8vNfoXHM0hoNGn/TuSdhYuVH7 rZZ1ARQ4byy5S0rlYzHX/QMSfPsghO1aG3H9WjB1HlIGlZQZLSUgKXMIN1t7Idf7Ze /Hc7LauAIEyCTXO1Z8BjCyJTTCq/6YeQmCAVeG2H9ZBsq3TaPLTjrjVV/coo8fzhn9 atcG9FOKge5ybBaMHkM2Yw8jnEbNQmqzTc8Bzis/bvsZWgjE+yNKF5QsBZIGI7NwKY zCuEbZPf4HPB+i2GtYop5i1l/BrvSfCs4vysVvUa602u5MBmFPXUw8pFfn6Vxk/7dI tFtWUOt+i5qKg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Tariq Toukan , Leon Romanovsky , Maxim Mikityanskiy , Saeed Mahameed Subject: [net-next V2 11/17] net/mlx5e: Support MQPRIO channel mode Date: Mon, 16 Aug 2021 16:22:13 -0700 Message-Id: <20210816232219.557083-12-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210816232219.557083-1-saeed@kernel.org> References: <20210816232219.557083-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Add support for MQPRIO channel mode, in which a partition to TCs is defined over the channels. We allow partitions with contiguous queue indices, with no holes within. We do not allow modification to the num of channels while this MQPRIO mode is active. Signed-off-by: Tariq Toukan Reviewed-by: Maxim Mikityanskiy Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 99 +++++++++++++++++-- 3 files changed, 102 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 3dbcb2cf2ff8..669a75f3537a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -72,6 +72,7 @@ struct page_pool; #define MLX5E_SW2HW_MTU(params, swmtu) ((swmtu) + ((params)->hard_mtu)) #define MLX5E_MAX_NUM_TC 8 +#define MLX5E_MAX_NUM_MQPRIO_CH_TC TC_QOPT_MAX_QUEUE #define MLX5_RX_HEADROOM NET_SKB_PAD #define MLX5_SKB_FRAG_SZ(len) (SKB_DATA_ALIGN(len) + \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index 839a753fda32..5696d3f1baaf 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -467,6 +467,16 @@ int mlx5e_ethtool_set_channels(struct mlx5e_priv *priv, goto out; } + /* Don't allow changing the number of channels if MQPRIO mode channel offload is active, + * because it defines a partition over the channels queues. + */ + if (cur_params->mqprio.mode == TC_MQPRIO_MODE_CHANNEL) { + err = -EINVAL; + netdev_err(priv->netdev, "%s: MQPRIO mode channel offload is active, cannot change the number of channels\n", + __func__); + goto out; + } + new_params = *cur_params; new_params.num_channels = count; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index f5c89a00214d..26d2f78c7706 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2263,7 +2263,8 @@ void mlx5e_set_netdev_mtu_boundaries(struct mlx5e_priv *priv) ETH_MAX_MTU); } -static int mlx5e_netdev_set_tcs(struct net_device *netdev, u16 nch, u8 ntc) +static int mlx5e_netdev_set_tcs(struct net_device *netdev, u16 nch, u8 ntc, + struct tc_mqprio_qopt_offload *mqprio) { int tc, err; @@ -2278,11 +2279,16 @@ static int mlx5e_netdev_set_tcs(struct net_device *netdev, u16 nch, u8 ntc) return err; } - /* Map netdev TCs to offset 0 - * We have our own UP to TXQ mapping for QoS - */ - for (tc = 0; tc < ntc; tc++) - netdev_set_tc_queue(netdev, tc, nch, 0); + for (tc = 0; tc < ntc; tc++) { + u16 count, offset; + + /* For DCB mode, map netdev TCs to offset 0 + * We have our own UP to TXQ mapping for QoS + */ + count = mqprio ? mqprio->qopt.count[tc] : nch; + offset = mqprio ? mqprio->qopt.offset[tc] : 0; + netdev_set_tc_queue(netdev, tc, count, offset); + } return 0; } @@ -2321,7 +2327,7 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv) ntc = mlx5e_get_dcb_num_tc(&priv->channels.params); num_rxqs = nch * priv->profile->rq_groups; - err = mlx5e_netdev_set_tcs(netdev, nch, ntc); + err = mlx5e_netdev_set_tcs(netdev, nch, ntc, NULL); if (err) goto err_out; err = mlx5e_update_tx_netdev_queues(priv); @@ -2344,7 +2350,7 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv) WARN_ON_ONCE(netif_set_real_num_tx_queues(netdev, old_num_txqs)); err_tcs: - mlx5e_netdev_set_tcs(netdev, old_num_txqs / old_ntc, old_ntc); + mlx5e_netdev_set_tcs(netdev, old_num_txqs / old_ntc, old_ntc, NULL); err_out: return err; } @@ -2879,6 +2885,81 @@ static int mlx5e_setup_tc_mqprio_dcb(struct mlx5e_priv *priv, return err; } +static int mlx5e_mqprio_channel_validate(struct mlx5e_priv *priv, + struct tc_mqprio_qopt_offload *mqprio) +{ + struct net_device *netdev = priv->netdev; + int agg_count = 0; + int i; + + if (mqprio->qopt.offset[0] != 0 || mqprio->qopt.num_tc < 1 || + mqprio->qopt.num_tc > MLX5E_MAX_NUM_MQPRIO_CH_TC) + return -EINVAL; + + for (i = 0; i < mqprio->qopt.num_tc; i++) { + if (!mqprio->qopt.count[i]) { + netdev_err(netdev, "Zero size for queue-group (%d) is not supported\n", i); + return -EINVAL; + } + if (mqprio->min_rate[i]) { + netdev_err(netdev, "Min tx rate is not supported\n"); + return -EINVAL; + } + if (mqprio->max_rate[i]) { + netdev_err(netdev, "Max tx rate is not supported\n"); + return -EINVAL; + } + + if (mqprio->qopt.offset[i] != agg_count) { + netdev_err(netdev, "Discontinuous queues config is not supported\n"); + return -EINVAL; + } + agg_count += mqprio->qopt.count[i]; + } + + if (priv->channels.params.num_channels < agg_count) { + netdev_err(netdev, "Num of queues (%d) exceeds available (%d)\n", + agg_count, priv->channels.params.num_channels); + return -EINVAL; + } + + return 0; +} + +static int mlx5e_mqprio_channel_set_tcs_ctx(struct mlx5e_priv *priv, void *ctx) +{ + struct tc_mqprio_qopt_offload *mqprio = (struct tc_mqprio_qopt_offload *)ctx; + struct net_device *netdev = priv->netdev; + u8 num_tc; + + if (priv->channels.params.mqprio.mode != TC_MQPRIO_MODE_CHANNEL) + return -EINVAL; + + num_tc = priv->channels.params.mqprio.num_tc; + mlx5e_netdev_set_tcs(netdev, 0, num_tc, mqprio); + + return 0; +} + +static int mlx5e_setup_tc_mqprio_channel(struct mlx5e_priv *priv, + struct tc_mqprio_qopt_offload *mqprio) +{ + struct mlx5e_params new_params; + int err; + + err = mlx5e_mqprio_channel_validate(priv, mqprio); + if (err) + return err; + + new_params = priv->channels.params; + new_params.mqprio.mode = TC_MQPRIO_MODE_CHANNEL; + new_params.mqprio.num_tc = mqprio->qopt.num_tc; + err = mlx5e_safe_switch_params(priv, &new_params, + mlx5e_mqprio_channel_set_tcs_ctx, mqprio, true); + + return err; +} + static int mlx5e_setup_tc_mqprio(struct mlx5e_priv *priv, struct tc_mqprio_qopt_offload *mqprio) { @@ -2891,6 +2972,8 @@ static int mlx5e_setup_tc_mqprio(struct mlx5e_priv *priv, switch (mqprio->mode) { case TC_MQPRIO_MODE_DCB: return mlx5e_setup_tc_mqprio_dcb(priv, &mqprio->qopt); + case TC_MQPRIO_MODE_CHANNEL: + return mlx5e_setup_tc_mqprio_channel(priv, mqprio); default: return -EOPNOTSUPP; }