From patchwork Sat Apr 24 08:01:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 427159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60C3EC433ED for ; Sat, 24 Apr 2021 08:02:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C33160724 for ; Sat, 24 Apr 2021 08:02:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233188AbhDXICm (ORCPT ); Sat, 24 Apr 2021 04:02:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:47256 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229850AbhDXIB7 (ORCPT ); Sat, 24 Apr 2021 04:01:59 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8FFB0611AE; Sat, 24 Apr 2021 08:01:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1619251281; bh=WMO9rukmXytsB95118+FhiKVMMZHTD4DIps0VSNJodE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AbxJdVoao76D+uUQFfR7xaN/3qwuGRojWsWk4R3IbRkHEHrrfv0Nk1MqmR92Ue2uw aTsHEKZxmV/YzvfeJeFkzLyCsK7GPX2Ug0PMmt1uyKGCR3lfmCxc3q9JJTz/df4mBl nIO1kizsbBLEkFMm9itNsjSQUA7Tr2kWabQiL3riplEvH9uufGol1qw47qmCyTP06d EUkNdNbSrJ4LPozibi30L7CS61IUldiW7N/J08MbidbWpmikcwTagWFvT1JRfRSiz+ 9RsG+y6mb7AzGn9fZJYetts1y1h8H7p06KCcxgpjGrEql1qj1Au0dimk5g9ub7Au0A ROFuhyCGdqLQw== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Parav Pandit , Roi Dayan , Vu Pham , Saeed Mahameed Subject: [net-next V2 01/11] net/mlx5: E-Switch, Return eswitch max ports when eswitch is supported Date: Sat, 24 Apr 2021 01:01:05 -0700 Message-Id: <20210424080115.97273-2-saeed@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210424080115.97273-1-saeed@kernel.org> References: <20210424080115.97273-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Parav Pandit mlx5_eswitch_get_total_vports() doesn't honor MLX5_ESWICH Kconfig flag. When MLX5_ESWITCH is disabled, FS layer continues to initialize eswitch specific ACL namespaces. Instead, start honoring MLX5_ESWITCH flag and perform vport specific initialization only when vport count is non zero. Signed-off-by: Parav Pandit Reviewed-by: Roi Dayan Reviewed-by: Vu Pham Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 13 +++++++++++++ drivers/net/ethernet/mellanox/mlx5/core/vport.c | 14 -------------- include/linux/mlx5/eswitch.h | 11 +++++++++-- 3 files changed, 22 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 1bb229ecd43b..c3a58224ae12 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -2205,3 +2205,16 @@ void mlx5_esw_unlock(struct mlx5_eswitch *esw) { up_write(&esw->mode_lock); } + +/** + * mlx5_eswitch_get_total_vports - Get total vports of the eswitch + * + * @dev: Pointer to core device + * + * mlx5_eswitch_get_total_vports returns total number of eswitch vports. + */ +u16 mlx5_eswitch_get_total_vports(const struct mlx5_core_dev *dev) +{ + return MLX5_SPECIAL_VPORTS(dev) + mlx5_core_max_vfs(dev) + mlx5_sf_max_functions(dev); +} +EXPORT_SYMBOL_GPL(mlx5_eswitch_get_total_vports); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c index e05c5c0f3ae1..457ad42eaa2a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c @@ -1151,20 +1151,6 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev) } EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid); -/** - * mlx5_eswitch_get_total_vports - Get total vports of the eswitch - * - * @dev: Pointer to core device - * - * mlx5_eswitch_get_total_vports returns total number of vports for - * the eswitch. - */ -u16 mlx5_eswitch_get_total_vports(const struct mlx5_core_dev *dev) -{ - return MLX5_SPECIAL_VPORTS(dev) + mlx5_core_max_vfs(dev) + mlx5_sf_max_functions(dev); -} -EXPORT_SYMBOL_GPL(mlx5_eswitch_get_total_vports); - int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out) { u16 opmod = (MLX5_CAP_GENERAL << 1) | (HCA_CAP_OPMOD_GET_MAX & 0x01); diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h index 9cf1da2883c6..17109b65c1ac 100644 --- a/include/linux/mlx5/eswitch.h +++ b/include/linux/mlx5/eswitch.h @@ -65,8 +65,6 @@ struct mlx5_flow_handle * mlx5_eswitch_add_send_to_vport_rule(struct mlx5_eswitch *on_esw, struct mlx5_eswitch_rep *rep, u32 sqn); -u16 mlx5_eswitch_get_total_vports(const struct mlx5_core_dev *dev); - #ifdef CONFIG_MLX5_ESWITCH enum devlink_eswitch_encap_mode mlx5_eswitch_get_encap_mode(const struct mlx5_core_dev *dev); @@ -126,6 +124,8 @@ u32 mlx5_eswitch_get_vport_metadata_for_set(struct mlx5_eswitch *esw, #define ESW_TUN_SLOW_TABLE_GOTO_VPORT_MARK ESW_TUN_OPTS_MASK u8 mlx5_eswitch_mode(struct mlx5_core_dev *dev); +u16 mlx5_eswitch_get_total_vports(const struct mlx5_core_dev *dev); + #else /* CONFIG_MLX5_ESWITCH */ static inline u8 mlx5_eswitch_mode(struct mlx5_core_dev *dev) @@ -162,10 +162,17 @@ mlx5_eswitch_get_vport_metadata_mask(void) { return 0; } + +static inline u16 mlx5_eswitch_get_total_vports(const struct mlx5_core_dev *dev) +{ + return 0; +} + #endif /* CONFIG_MLX5_ESWITCH */ static inline bool is_mdev_switchdev_mode(struct mlx5_core_dev *dev) { return mlx5_eswitch_mode(dev) == MLX5_ESWITCH_OFFLOADS; } + #endif From patchwork Sat Apr 24 08:01:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 427157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BD50C43460 for ; Sat, 24 Apr 2021 08:02:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B3B0613CD for ; Sat, 24 Apr 2021 08:02:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233922AbhDXIDX (ORCPT ); Sat, 24 Apr 2021 04:03:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:47340 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbhDXICU (ORCPT ); Sat, 24 Apr 2021 04:02:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 866EA61490; Sat, 24 Apr 2021 08:01:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1619251283; bh=Q80QWz9WuMuq647rBwI9VdSnMLeouMcuMpMAiZBhdaA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XApnIh+u4CZ88TJsMRMZRd8YlPDfJ73EWUQQUP23J6z0NcHF6VgKWUaG0oYEwZshC 7uopvACX8nq/xSH5CuLXO4yh2QAMxghGnSdLgX5QnElCMaf2t9FIShTkTj0VORTqqJ tryWSt2vM6Ar5GG6+QSo0hW4hk4BW53/+Rqa4VKpOA/qjMN4+pCOkgIsQT5Agc1UNI KTULRglCXVVaAd28ZI2LybAWoecMF2NAKV9q1S7FCPlxeVY/iEZzGOeqCvF5Pmnqan 073XZQZPYf//s+vCCbENHL5Gs0SUg4+VEnFMRmGEbYiW1cc1i0Pj827iGLSxIqDviD hLWhkFI173PJg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Parav Pandit , Vu Pham , Saeed Mahameed Subject: [net-next V2 05/11] net/mlx5: SF, Rely on hw table for SF devlink port allocation Date: Sat, 24 Apr 2021 01:01:09 -0700 Message-Id: <20210424080115.97273-6-saeed@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210424080115.97273-1-saeed@kernel.org> References: <20210424080115.97273-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Parav Pandit Supporting SF allocation is currently checked at two places: (a) SF devlink port allocator and (b) SF HW table handler. Both layers are using HCA CAP to identify it using helper routine mlx5_sf_supported() and mlx5_sf_max_functions(). Instead, rely on the HW table handler to check if SF is supported or not. Signed-off-by: Parav Pandit Reviewed-by: Vu Pham Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c | 9 ++------- drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c | 7 ++++++- drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h | 1 + 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c index 52226d9b9a6d..5fa261334cd0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c @@ -437,9 +437,6 @@ static int mlx5_sf_vhca_event(struct notifier_block *nb, unsigned long opcode, v static void mlx5_sf_table_enable(struct mlx5_sf_table *table) { - if (!mlx5_sf_max_functions(table->dev)) - return; - init_completion(&table->disable_complete); refcount_set(&table->refcount, 1); } @@ -462,9 +459,6 @@ static void mlx5_sf_deactivate_all(struct mlx5_sf_table *table) static void mlx5_sf_table_disable(struct mlx5_sf_table *table) { - if (!mlx5_sf_max_functions(table->dev)) - return; - if (!refcount_read(&table->refcount)) return; @@ -498,7 +492,8 @@ static int mlx5_sf_esw_event(struct notifier_block *nb, unsigned long event, voi static bool mlx5_sf_table_supported(const struct mlx5_core_dev *dev) { - return dev->priv.eswitch && MLX5_ESWITCH_MANAGER(dev) && mlx5_sf_supported(dev); + return dev->priv.eswitch && MLX5_ESWITCH_MANAGER(dev) && + mlx5_sf_hw_table_supported(dev); } int mlx5_sf_table_init(struct mlx5_core_dev *dev) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index ec53c11c8344..9140c81aa03a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -41,7 +41,7 @@ int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) int err; int i; - if (!table->max_local_functions) + if (!table || !table->max_local_functions) return -EOPNOTSUPP; mutex_lock(&table->table_lock); @@ -230,3 +230,8 @@ void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev) /* Dealloc SFs whose firmware event has been missed. */ mlx5_sf_hw_dealloc_all(table); } + +bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev) +{ + return !!dev->priv.sf_hw_table; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h index cb02a51d0986..b36be5ecb496 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h @@ -17,5 +17,6 @@ u16 mlx5_sf_sw_to_hw_id(const struct mlx5_core_dev *dev, u16 sw_id); int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum); void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id); void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id); +bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev); #endif From patchwork Sat Apr 24 08:01:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 427156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 454A8C433ED for ; Sat, 24 Apr 2021 08:02:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 24CFF6023E for ; Sat, 24 Apr 2021 08:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234710AbhDXIDd (ORCPT ); Sat, 24 Apr 2021 04:03:33 -0400 Received: from mail.kernel.org ([198.145.29.99]:47368 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232724AbhDXICU (ORCPT ); Sat, 24 Apr 2021 04:02:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3EF6F61580; Sat, 24 Apr 2021 08:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1619251285; bh=cuQJvEvTXx0g3zCbrzBlAmD39ycUPBkW1quY6VTmGfs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AZAi0a5sg8SMJyFZZQdrWnq9C/jPn4F5/OOdt+bm5H5MnIbxUjUoKP+VCX6K/kIw/ Gc6sfRL2+F07nc7t1cg6h2ZGJd5otnq5di2ze5sUywLxghGr8yg8X2T5WmEDGZkPdO G5emLDYZTQyWQWRyMXgJghGVoN1Lzaug0Q+NBR0wI/wa3rF1VVTagjyCVBBM+Aw3R0 XOOAk0ppJW/qH1/xR2YHROmJxUgllAVABhLfrfJj+UKGaNNUdMt0WpIHbcl2komYZz nMPw/QrYMAxCX29kAb5xv2u2gCuGRGKxsYTj1f7unYc6Pk9o6lfdMDok0MVMfJhy/F LMr0vjnJvtE+Q== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Parav Pandit , Vu Pham , Saeed Mahameed Subject: [net-next V2 09/11] net/mlx5: SF, Use helpers for allocation and free Date: Sat, 24 Apr 2021 01:01:13 -0700 Message-Id: <20210424080115.97273-10-saeed@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210424080115.97273-1-saeed@kernel.org> References: <20210424080115.97273-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Parav Pandit Use helper routines for SF id and SF table allocation and free so that subsequent patch can reuse it for multiple SF function id range. Signed-off-by: Parav Pandit Reviewed-by: Vu Pham Signed-off-by: Saeed Mahameed --- .../ethernet/mellanox/mlx5/core/sf/hw_table.c | 97 ++++++++++++------- 1 file changed, 60 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index c3126031c2bf..172bfad372ec 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -38,37 +38,46 @@ static u16 mlx5_sf_hw_to_sw_id(const struct mlx5_core_dev *dev, u16 hw_id) return hw_id - table->start_fn_id; } -int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) +static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 usr_sfnum) { - struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; - int sw_id = -ENOSPC; - u16 hw_fn_id; - int err; int i; - if (!table || !table->max_local_functions) - return -EOPNOTSUPP; - - mutex_lock(&table->table_lock); /* Check if sf with same sfnum already exists or not. */ for (i = 0; i < table->max_local_functions; i++) { - if (table->sfs[i].allocated && table->sfs[i].usr_sfnum == usr_sfnum) { - err = -EEXIST; - goto exist_err; - } + if (table->sfs[i].allocated && table->sfs[i].usr_sfnum == usr_sfnum) + return -EEXIST; } - /* Find the free entry and allocate the entry from the array */ for (i = 0; i < table->max_local_functions; i++) { if (!table->sfs[i].allocated) { table->sfs[i].usr_sfnum = usr_sfnum; table->sfs[i].allocated = true; - sw_id = i; - break; + return i; } } - if (sw_id == -ENOSPC) { - err = -ENOSPC; + return -ENOSPC; +} + +static void mlx5_sf_hw_table_id_free(struct mlx5_sf_hw_table *table, int id) +{ + table->sfs[id].allocated = false; + table->sfs[id].pending_delete = false; +} + +int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) +{ + struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; + u16 hw_fn_id; + int sw_id; + int err; + + if (!table) + return -EOPNOTSUPP; + + mutex_lock(&table->table_lock); + sw_id = mlx5_sf_hw_table_id_alloc(table, usr_sfnum); + if (sw_id < 0) { + err = sw_id; goto exist_err; } @@ -87,21 +96,20 @@ int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) vhca_err: mlx5_cmd_dealloc_sf(dev, hw_fn_id); err: - table->sfs[i].allocated = false; + mlx5_sf_hw_table_id_free(table, sw_id); exist_err: mutex_unlock(&table->table_lock); return err; } -static void _mlx5_sf_hw_id_free(struct mlx5_core_dev *dev, u16 id) +static void _mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id) { struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; u16 hw_fn_id; hw_fn_id = mlx5_sf_sw_to_hw_id(dev, id); mlx5_cmd_dealloc_sf(dev, hw_fn_id); - table->sfs[id].allocated = false; - table->sfs[id].pending_delete = false; + mlx5_sf_hw_table_id_free(table, id); } void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id) @@ -109,7 +117,7 @@ void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id) struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; mutex_lock(&table->table_lock); - _mlx5_sf_hw_id_free(dev, id); + _mlx5_sf_hw_table_sf_free(dev, id); mutex_unlock(&table->table_lock); } @@ -143,40 +151,55 @@ static void mlx5_sf_hw_dealloc_all(struct mlx5_sf_hw_table *table) for (i = 0; i < table->max_local_functions; i++) { if (table->sfs[i].allocated) - _mlx5_sf_hw_id_free(table->dev, i); + _mlx5_sf_hw_table_sf_free(table->dev, i); } } +static int mlx5_sf_hw_table_alloc(struct mlx5_sf_hw_table *table, u16 max_fn, u16 base_id) +{ + struct mlx5_sf_hw *sfs; + + sfs = kcalloc(max_fn, sizeof(*sfs), GFP_KERNEL); + if (!sfs) + return -ENOMEM; + + table->sfs = sfs; + table->max_local_functions = max_fn; + table->start_fn_id = base_id; + return 0; +} + int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) { struct mlx5_sf_hw_table *table; - struct mlx5_sf_hw *sfs; - int max_functions; + u16 base_id; + u16 max_fn; + int err; if (!mlx5_sf_supported(dev) || !mlx5_vhca_event_supported(dev)) return 0; - max_functions = mlx5_sf_max_functions(dev); + max_fn = mlx5_sf_max_functions(dev); table = kzalloc(sizeof(*table), GFP_KERNEL); if (!table) return -ENOMEM; - sfs = kcalloc(max_functions, sizeof(*sfs), GFP_KERNEL); - if (!sfs) - goto table_err; - mutex_init(&table->table_lock); table->dev = dev; - table->sfs = sfs; - table->max_local_functions = max_functions; - table->start_fn_id = mlx5_sf_start_function_id(dev); dev->priv.sf_hw_table = table; - mlx5_core_dbg(dev, "SF HW table: max sfs = %d\n", max_functions); + + base_id = mlx5_sf_start_function_id(dev); + err = mlx5_sf_hw_table_alloc(table, max_fn, base_id); + if (err) + goto table_err; + + mlx5_core_dbg(dev, "SF HW table: max sfs = %d\n", max_fn); return 0; table_err: + mutex_destroy(&table->table_lock); kfree(table); - return -ENOMEM; + return err; } void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev) @@ -209,7 +232,7 @@ static int mlx5_sf_hw_vhca_event(struct notifier_block *nb, unsigned long opcode * Hence recycle the sf hardware id for reuse. */ if (sf_hw->allocated && sf_hw->pending_delete) - _mlx5_sf_hw_id_free(table->dev, sw_id); + _mlx5_sf_hw_table_sf_free(table->dev, sw_id); mutex_unlock(&table->table_lock); return 0; } From patchwork Sat Apr 24 08:01:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 427155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFD78C433ED for ; Sat, 24 Apr 2021 08:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 94F9460724 for ; Sat, 24 Apr 2021 08:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236929AbhDXIDl (ORCPT ); Sat, 24 Apr 2021 04:03:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:47370 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232738AbhDXICU (ORCPT ); Sat, 24 Apr 2021 04:02:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id A79D661582; Sat, 24 Apr 2021 08:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1619251285; bh=gnA8sdwoOQUIOCTdTQFPzm/PM1ZOmnRm0rK5ja/VNbs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X33L0HgVKV8RCrEu4JMV70vME5Kq/JdqFwntfXfO7jYmNmZgtL5queUnLee19pftU 3lYDvsWxwecm1AWyGUT834hP/lZ4sOMuln3ZUVnwzLtIY7kNU80Zgko/5vzWOmVGJm N5j26dJC6fVjapuBxUa2ucjWDSkUdcFkGj6ja82WR7vToUnty+15KFrhXrFOCvBdIz lGvUTtPub0DVWdveh7vPqSS4tdqJ5m78WuunHv37n5zc5JkrYvSRFTeVh6+2MwTjj5 FT6LioL5Tkxb596aqhn2c50tMYgGKWFIXWVhtPTDj5+s1GP0bNoW936mRh9LvNC+RY rR6fZNR9vUCjw== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Parav Pandit , Vu Pham , Saeed Mahameed Subject: [net-next V2 10/11] net/mlx5: SF, Split mlx5_sf_hw_table into two parts Date: Sat, 24 Apr 2021 01:01:14 -0700 Message-Id: <20210424080115.97273-11-saeed@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210424080115.97273-1-saeed@kernel.org> References: <20210424080115.97273-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Parav Pandit Device has SF ids in two different contiguous ranges. One for the local controller and second for the external controller's PF. Each such range has its own maximum number of functions and base id. To allocate SF from either of the range, prepare code to split into range specific fields into its own structure. Signed-off-by: Parav Pandit Reviewed-by: Vu Pham Signed-off-by: Saeed Mahameed --- .../ethernet/mellanox/mlx5/core/sf/hw_table.c | 87 ++++++++++++------- 1 file changed, 58 insertions(+), 29 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index 172bfad372ec..b5eab48bbe08 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -15,43 +15,55 @@ struct mlx5_sf_hw { u8 pending_delete: 1; }; -struct mlx5_sf_hw_table { - struct mlx5_core_dev *dev; +struct mlx5_sf_hwc_table { struct mlx5_sf_hw *sfs; - int max_local_functions; + int max_fn; u16 start_fn_id; +}; + +enum mlx5_sf_hwc_index { + MLX5_SF_HWC_LOCAL, + MLX5_SF_HWC_MAX, +}; + +struct mlx5_sf_hw_table { + struct mlx5_core_dev *dev; struct mutex table_lock; /* Serializes sf deletion and vhca state change handler. */ struct notifier_block vhca_nb; + struct mlx5_sf_hwc_table hwc[MLX5_SF_HWC_MAX]; }; u16 mlx5_sf_sw_to_hw_id(const struct mlx5_core_dev *dev, u16 sw_id) { - struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; + struct mlx5_sf_hwc_table *hwc = &dev->priv.sf_hw_table->hwc[MLX5_SF_HWC_LOCAL]; - return table->start_fn_id + sw_id; + return hwc->start_fn_id + sw_id; } static u16 mlx5_sf_hw_to_sw_id(const struct mlx5_core_dev *dev, u16 hw_id) { - struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; + struct mlx5_sf_hwc_table *hwc = &dev->priv.sf_hw_table->hwc[MLX5_SF_HWC_LOCAL]; - return hw_id - table->start_fn_id; + return hw_id - hwc->start_fn_id; } static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 usr_sfnum) { + struct mlx5_sf_hwc_table *hwc; int i; + hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; + /* Check if sf with same sfnum already exists or not. */ - for (i = 0; i < table->max_local_functions; i++) { - if (table->sfs[i].allocated && table->sfs[i].usr_sfnum == usr_sfnum) + for (i = 0; i < hwc->max_fn; i++) { + if (hwc->sfs[i].allocated && hwc->sfs[i].usr_sfnum == usr_sfnum) return -EEXIST; } /* Find the free entry and allocate the entry from the array */ - for (i = 0; i < table->max_local_functions; i++) { - if (!table->sfs[i].allocated) { - table->sfs[i].usr_sfnum = usr_sfnum; - table->sfs[i].allocated = true; + for (i = 0; i < hwc->max_fn; i++) { + if (!hwc->sfs[i].allocated) { + hwc->sfs[i].usr_sfnum = usr_sfnum; + hwc->sfs[i].allocated = true; return i; } } @@ -60,8 +72,10 @@ static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 usr_sfn static void mlx5_sf_hw_table_id_free(struct mlx5_sf_hw_table *table, int id) { - table->sfs[id].allocated = false; - table->sfs[id].pending_delete = false; + struct mlx5_sf_hwc_table *hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; + + hwc->sfs[id].allocated = false; + hwc->sfs[id].pending_delete = false; } int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) @@ -125,11 +139,13 @@ void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id) { struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; u32 out[MLX5_ST_SZ_DW(query_vhca_state_out)] = {}; + struct mlx5_sf_hwc_table *hwc; u16 hw_fn_id; u8 state; int err; hw_fn_id = mlx5_sf_sw_to_hw_id(dev, id); + hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; mutex_lock(&table->table_lock); err = mlx5_cmd_query_vhca_state(dev, hw_fn_id, out, sizeof(out)); if (err) @@ -137,25 +153,31 @@ void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id) state = MLX5_GET(query_vhca_state_out, out, vhca_state_context.vhca_state); if (state == MLX5_VHCA_STATE_ALLOCATED) { mlx5_cmd_dealloc_sf(dev, hw_fn_id); - table->sfs[id].allocated = false; + hwc->sfs[id].allocated = false; } else { - table->sfs[id].pending_delete = true; + hwc->sfs[id].pending_delete = true; } err: mutex_unlock(&table->table_lock); } -static void mlx5_sf_hw_dealloc_all(struct mlx5_sf_hw_table *table) +static void mlx5_sf_hw_table_hwc_dealloc_all(struct mlx5_core_dev *dev, + struct mlx5_sf_hwc_table *hwc) { int i; - for (i = 0; i < table->max_local_functions; i++) { - if (table->sfs[i].allocated) - _mlx5_sf_hw_table_sf_free(table->dev, i); + for (i = 0; i < hwc->max_fn; i++) { + if (hwc->sfs[i].allocated) + _mlx5_sf_hw_table_sf_free(dev, i); } } -static int mlx5_sf_hw_table_alloc(struct mlx5_sf_hw_table *table, u16 max_fn, u16 base_id) +static void mlx5_sf_hw_table_dealloc_all(struct mlx5_sf_hw_table *table) +{ + mlx5_sf_hw_table_hwc_dealloc_all(table->dev, &table->hwc[MLX5_SF_HWC_LOCAL]); +} + +static int mlx5_sf_hw_table_hwc_init(struct mlx5_sf_hwc_table *hwc, u16 max_fn, u16 base_id) { struct mlx5_sf_hw *sfs; @@ -163,12 +185,17 @@ static int mlx5_sf_hw_table_alloc(struct mlx5_sf_hw_table *table, u16 max_fn, u1 if (!sfs) return -ENOMEM; - table->sfs = sfs; - table->max_local_functions = max_fn; - table->start_fn_id = base_id; + hwc->sfs = sfs; + hwc->max_fn = max_fn; + hwc->start_fn_id = base_id; return 0; } +static void mlx5_sf_hw_table_hwc_cleanup(struct mlx5_sf_hwc_table *hwc) +{ + kfree(hwc->sfs); +} + int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) { struct mlx5_sf_hw_table *table; @@ -189,7 +216,7 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) dev->priv.sf_hw_table = table; base_id = mlx5_sf_start_function_id(dev); - err = mlx5_sf_hw_table_alloc(table, max_fn, base_id); + err = mlx5_sf_hw_table_hwc_init(&table->hwc[MLX5_SF_HWC_LOCAL], max_fn, base_id); if (err) goto table_err; @@ -210,7 +237,7 @@ void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev) return; mutex_destroy(&table->table_lock); - kfree(table->sfs); + mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]); kfree(table); } @@ -218,14 +245,16 @@ static int mlx5_sf_hw_vhca_event(struct notifier_block *nb, unsigned long opcode { struct mlx5_sf_hw_table *table = container_of(nb, struct mlx5_sf_hw_table, vhca_nb); const struct mlx5_vhca_state_event *event = data; + struct mlx5_sf_hwc_table *hwc; struct mlx5_sf_hw *sf_hw; u16 sw_id; if (event->new_vhca_state != MLX5_VHCA_STATE_ALLOCATED) return 0; + hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; sw_id = mlx5_sf_hw_to_sw_id(table->dev, event->function_id); - sf_hw = &table->sfs[sw_id]; + sf_hw = &hwc->sfs[sw_id]; mutex_lock(&table->table_lock); /* SF driver notified through firmware that SF is finally detached. @@ -257,7 +286,7 @@ void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev) mlx5_vhca_event_notifier_unregister(dev, &table->vhca_nb); /* Dealloc SFs whose firmware event has been missed. */ - mlx5_sf_hw_dealloc_all(table); + mlx5_sf_hw_table_dealloc_all(table); } bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev) From patchwork Sat Apr 24 08:01:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 427154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39255C433ED for ; Sat, 24 Apr 2021 08:03:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1402660724 for ; Sat, 24 Apr 2021 08:03:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237123AbhDXIDt (ORCPT ); Sat, 24 Apr 2021 04:03:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:47458 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232775AbhDXICU (ORCPT ); Sat, 24 Apr 2021 04:02:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D49C61584; Sat, 24 Apr 2021 08:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1619251286; bh=+OCLveJFgFLRu/0LTZNA0ujJRXZ/ehylZ8O7uhzqSGk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=brqLWGfvH2CQQlrgvHyMMD4gzptOy54c2el7bFQapuUbxQEfNc1nFW9RpGquoWLVG HzIjpiU7os10/GDgjq5m5e4dyMVnyryT3yfre8IYLrkctlRyBR8ZGClS79qbf1nJcR sCkuaSkzK+RGVSUEC3P4r63gq/u5zUlaWxCcJAL4kYLwGwmpcnsOXkMsN2rTXTJORG 7dvCoFT4CYtVr7ksWZZSw/eN+f80xepSfxphUMuGrx+JN20sztGt7ZMfK/ibrEZW7n XGoiXi3ew+JadFActt0Rg+nkb/lXJ1uYB/DO6Ws+lcrC9VpMiMgKcMfZteLfwrZMio uu8b+CMF5oapA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Parav Pandit , Saeed Mahameed Subject: [net-next V2 11/11] net/mlx5: SF, Extend SF table for additional SF id range Date: Sat, 24 Apr 2021 01:01:15 -0700 Message-Id: <20210424080115.97273-12-saeed@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210424080115.97273-1-saeed@kernel.org> References: <20210424080115.97273-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Parav Pandit Extended the SF table to cover additioanl SF id range of external controller. A user optionallly provides the external controller number when user wants to create SF on the external controller. An example on eswitch system: $ devlink dev eswitch set pci/0033:01:00.0 mode switchdev $ devlink port show pci/0033:01:00.0/196607: type eth netdev enP51p1s0f0np0 flavour physical port 0 splittable false pci/0033:01:00.0/131072: type eth netdev eth0 flavour pcipf controller 1 pfnum 0 external true splittable false function: hw_addr 00:00:00:00:00:00 $ devlink port add pci/0033:01:00.0 flavour pcisf pfnum 0 sfnum 77 controller 1 pci/0033:01:00.0/163840: type eth netdev eth1 flavour pcisf controller 1 pfnum 0 sfnum 77 external true splittable false function: hw_addr 00:00:00:00:00:00 state inactive opstate detached Signed-off-by: Parav Pandit Signed-off-by: Saeed Mahameed --- .../mellanox/mlx5/core/esw/devlink_port.c | 4 +- .../net/ethernet/mellanox/mlx5/core/eswitch.h | 6 +- .../mellanox/mlx5/core/eswitch_offloads.c | 17 ++- .../ethernet/mellanox/mlx5/core/sf/devlink.c | 29 ++-- .../ethernet/mellanox/mlx5/core/sf/hw_table.c | 129 +++++++++++++----- .../net/ethernet/mellanox/mlx5/core/sf/priv.h | 8 +- 6 files changed, 140 insertions(+), 53 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c index 183f782b940f..1703384eca95 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c @@ -123,7 +123,7 @@ struct devlink_port *mlx5_esw_offloads_devlink_port(struct mlx5_eswitch *esw, u1 } int mlx5_esw_devlink_sf_port_register(struct mlx5_eswitch *esw, struct devlink_port *dl_port, - u16 vport_num, u32 sfnum) + u16 vport_num, u32 controller, u32 sfnum) { struct mlx5_core_dev *dev = esw->dev; struct netdev_phys_item_id ppid = {}; @@ -141,7 +141,7 @@ int mlx5_esw_devlink_sf_port_register(struct mlx5_eswitch *esw, struct devlink_p mlx5_esw_get_port_parent_id(dev, &ppid); memcpy(dl_port->attrs.switch_id.id, &ppid.id[0], ppid.id_len); dl_port->attrs.switch_id.id_len = ppid.id_len; - devlink_port_attrs_pci_sf_set(dl_port, 0, pfnum, sfnum, false); + devlink_port_attrs_pci_sf_set(dl_port, controller, pfnum, sfnum, !!controller); devlink = priv_to_devlink(dev); dl_port_index = mlx5_esw_vport_to_devlink_port_index(dev, vport_num); err = devlink_port_register(devlink, dl_port, dl_port_index); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h index 0812cee8f603..64ccb2bc0b58 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h @@ -662,11 +662,11 @@ void mlx5_esw_offloads_devlink_port_unregister(struct mlx5_eswitch *esw, u16 vpo struct devlink_port *mlx5_esw_offloads_devlink_port(struct mlx5_eswitch *esw, u16 vport_num); int mlx5_esw_devlink_sf_port_register(struct mlx5_eswitch *esw, struct devlink_port *dl_port, - u16 vport_num, u32 sfnum); + u16 vport_num, u32 controller, u32 sfnum); void mlx5_esw_devlink_sf_port_unregister(struct mlx5_eswitch *esw, u16 vport_num); int mlx5_esw_offloads_sf_vport_enable(struct mlx5_eswitch *esw, struct devlink_port *dl_port, - u16 vport_num, u32 sfnum); + u16 vport_num, u32 controller, u32 sfnum); void mlx5_esw_offloads_sf_vport_disable(struct mlx5_eswitch *esw, u16 vport_num); int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *sf_base_id); @@ -695,6 +695,8 @@ void mlx5_esw_unlock(struct mlx5_eswitch *esw); void esw_vport_change_handle_locked(struct mlx5_vport *vport); +bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 controller); + #else /* CONFIG_MLX5_ESWITCH */ /* eswitch API stubs */ static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index a1dd66540ba0..db1e74280e57 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -2747,6 +2747,19 @@ static int mlx5_esw_host_number_init(struct mlx5_eswitch *esw) return 0; } +bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 controller) +{ + /* Local controller is always valid */ + if (controller == 0) + return true; + + if (!mlx5_core_is_ecpf_esw_manager(esw->dev)) + return false; + + /* External host number starts with zero in device */ + return (controller == esw->offloads.host_number + 1); +} + int esw_offloads_enable(struct mlx5_eswitch *esw) { struct mapping_ctx *reg_c0_obj_pool; @@ -3295,7 +3308,7 @@ u32 mlx5_eswitch_get_vport_metadata_for_match(struct mlx5_eswitch *esw, EXPORT_SYMBOL(mlx5_eswitch_get_vport_metadata_for_match); int mlx5_esw_offloads_sf_vport_enable(struct mlx5_eswitch *esw, struct devlink_port *dl_port, - u16 vport_num, u32 sfnum) + u16 vport_num, u32 controller, u32 sfnum) { int err; @@ -3303,7 +3316,7 @@ int mlx5_esw_offloads_sf_vport_enable(struct mlx5_eswitch *esw, struct devlink_p if (err) return err; - err = mlx5_esw_devlink_sf_port_register(esw, dl_port, vport_num, sfnum); + err = mlx5_esw_devlink_sf_port_register(esw, dl_port, vport_num, controller, sfnum); if (err) goto devlink_err; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c index 5fa261334cd0..a8e73c9ed1ea 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c @@ -12,6 +12,7 @@ struct mlx5_sf { struct devlink_port dl_port; unsigned int port_index; + u32 controller; u16 id; u16 hw_fn_id; u16 hw_state; @@ -58,7 +59,8 @@ static void mlx5_sf_id_erase(struct mlx5_sf_table *table, struct mlx5_sf *sf) } static struct mlx5_sf * -mlx5_sf_alloc(struct mlx5_sf_table *table, u32 sfnum, struct netlink_ext_ack *extack) +mlx5_sf_alloc(struct mlx5_sf_table *table, struct mlx5_eswitch *esw, + u32 controller, u32 sfnum, struct netlink_ext_ack *extack) { unsigned int dl_port_index; struct mlx5_sf *sf; @@ -66,7 +68,12 @@ mlx5_sf_alloc(struct mlx5_sf_table *table, u32 sfnum, struct netlink_ext_ack *ex int id_err; int err; - id_err = mlx5_sf_hw_table_sf_alloc(table->dev, sfnum); + if (!mlx5_esw_offloads_controller_valid(esw, controller)) { + NL_SET_ERR_MSG_MOD(extack, "Invalid controller number"); + return ERR_PTR(-EINVAL); + } + + id_err = mlx5_sf_hw_table_sf_alloc(table->dev, controller, sfnum); if (id_err < 0) { err = id_err; goto id_err; @@ -78,11 +85,12 @@ mlx5_sf_alloc(struct mlx5_sf_table *table, u32 sfnum, struct netlink_ext_ack *ex goto alloc_err; } sf->id = id_err; - hw_fn_id = mlx5_sf_sw_to_hw_id(table->dev, sf->id); + hw_fn_id = mlx5_sf_sw_to_hw_id(table->dev, controller, sf->id); dl_port_index = mlx5_esw_vport_to_devlink_port_index(table->dev, hw_fn_id); sf->port_index = dl_port_index; sf->hw_fn_id = hw_fn_id; sf->hw_state = MLX5_VHCA_STATE_ALLOCATED; + sf->controller = controller; err = mlx5_sf_id_insert(table, sf); if (err) @@ -93,7 +101,7 @@ mlx5_sf_alloc(struct mlx5_sf_table *table, u32 sfnum, struct netlink_ext_ack *ex insert_err: kfree(sf); alloc_err: - mlx5_sf_hw_table_sf_free(table->dev, id_err); + mlx5_sf_hw_table_sf_free(table->dev, controller, id_err); id_err: if (err == -EEXIST) NL_SET_ERR_MSG_MOD(extack, "SF already exist. Choose different sfnum"); @@ -103,7 +111,7 @@ mlx5_sf_alloc(struct mlx5_sf_table *table, u32 sfnum, struct netlink_ext_ack *ex static void mlx5_sf_free(struct mlx5_sf_table *table, struct mlx5_sf *sf) { mlx5_sf_id_erase(table, sf); - mlx5_sf_hw_table_sf_free(table->dev, sf->id); + mlx5_sf_hw_table_sf_free(table->dev, sf->controller, sf->id); kfree(sf); } @@ -272,12 +280,12 @@ static int mlx5_sf_add(struct mlx5_core_dev *dev, struct mlx5_sf_table *table, struct mlx5_sf *sf; int err; - sf = mlx5_sf_alloc(table, new_attr->sfnum, extack); + sf = mlx5_sf_alloc(table, esw, new_attr->controller, new_attr->sfnum, extack); if (IS_ERR(sf)) return PTR_ERR(sf); err = mlx5_esw_offloads_sf_vport_enable(esw, &sf->dl_port, sf->hw_fn_id, - new_attr->sfnum); + new_attr->controller, new_attr->sfnum); if (err) goto esw_err; *new_port_index = sf->port_index; @@ -306,7 +314,8 @@ mlx5_sf_new_check_attr(struct mlx5_core_dev *dev, const struct devlink_port_new_ "User must provide unique sfnum. Driver does not support auto assignment"); return -EOPNOTSUPP; } - if (new_attr->controller_valid && new_attr->controller) { + if (new_attr->controller_valid && new_attr->controller && + !mlx5_core_is_ecpf_esw_manager(dev)) { NL_SET_ERR_MSG_MOD(extack, "External controller is unsupported"); return -EOPNOTSUPP; } @@ -352,10 +361,10 @@ static void mlx5_sf_dealloc(struct mlx5_sf_table *table, struct mlx5_sf *sf) * firmware gives confirmation that it is detached by the driver. */ mlx5_cmd_sf_disable_hca(table->dev, sf->hw_fn_id); - mlx5_sf_hw_table_sf_deferred_free(table->dev, sf->id); + mlx5_sf_hw_table_sf_deferred_free(table->dev, sf->controller, sf->id); kfree(sf); } else { - mlx5_sf_hw_table_sf_deferred_free(table->dev, sf->id); + mlx5_sf_hw_table_sf_deferred_free(table->dev, sf->controller, sf->id); kfree(sf); } } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index b5eab48bbe08..ef5f892aafad 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -8,6 +8,7 @@ #include "ecpf.h" #include "vhca_event.h" #include "mlx5_core.h" +#include "eswitch.h" struct mlx5_sf_hw { u32 usr_sfnum; @@ -23,6 +24,7 @@ struct mlx5_sf_hwc_table { enum mlx5_sf_hwc_index { MLX5_SF_HWC_LOCAL, + MLX5_SF_HWC_EXTERNAL, MLX5_SF_HWC_MAX, }; @@ -33,26 +35,50 @@ struct mlx5_sf_hw_table { struct mlx5_sf_hwc_table hwc[MLX5_SF_HWC_MAX]; }; -u16 mlx5_sf_sw_to_hw_id(const struct mlx5_core_dev *dev, u16 sw_id) +static struct mlx5_sf_hwc_table * +mlx5_sf_controller_to_hwc(struct mlx5_core_dev *dev, u32 controller) { - struct mlx5_sf_hwc_table *hwc = &dev->priv.sf_hw_table->hwc[MLX5_SF_HWC_LOCAL]; + int idx = !!controller; - return hwc->start_fn_id + sw_id; + return &dev->priv.sf_hw_table->hwc[idx]; } -static u16 mlx5_sf_hw_to_sw_id(const struct mlx5_core_dev *dev, u16 hw_id) +u16 mlx5_sf_sw_to_hw_id(struct mlx5_core_dev *dev, u32 controller, u16 sw_id) { - struct mlx5_sf_hwc_table *hwc = &dev->priv.sf_hw_table->hwc[MLX5_SF_HWC_LOCAL]; + struct mlx5_sf_hwc_table *hwc; + + hwc = mlx5_sf_controller_to_hwc(dev, controller); + return hwc->start_fn_id + sw_id; +} +static u16 mlx5_sf_hw_to_sw_id(struct mlx5_sf_hwc_table *hwc, u16 hw_id) +{ return hw_id - hwc->start_fn_id; } -static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 usr_sfnum) +static struct mlx5_sf_hwc_table * +mlx5_sf_table_fn_to_hwc(struct mlx5_sf_hw_table *table, u16 fn_id) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(table->hwc); i++) { + if (table->hwc[i].max_fn && + fn_id >= table->hwc[i].start_fn_id && + fn_id < (table->hwc[i].start_fn_id + table->hwc[i].max_fn)) + return &table->hwc[i]; + } + return NULL; +} + +static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 controller, + u32 usr_sfnum) { struct mlx5_sf_hwc_table *hwc; int i; - hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; + hwc = mlx5_sf_controller_to_hwc(table->dev, controller); + if (!hwc->sfs) + return -ENOSPC; /* Check if sf with same sfnum already exists or not. */ for (i = 0; i < hwc->max_fn; i++) { @@ -70,15 +96,16 @@ static int mlx5_sf_hw_table_id_alloc(struct mlx5_sf_hw_table *table, u32 usr_sfn return -ENOSPC; } -static void mlx5_sf_hw_table_id_free(struct mlx5_sf_hw_table *table, int id) +static void mlx5_sf_hw_table_id_free(struct mlx5_sf_hw_table *table, u32 controller, int id) { - struct mlx5_sf_hwc_table *hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; + struct mlx5_sf_hwc_table *hwc; + hwc = mlx5_sf_controller_to_hwc(table->dev, controller); hwc->sfs[id].allocated = false; hwc->sfs[id].pending_delete = false; } -int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) +int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 controller, u32 usr_sfnum) { struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; u16 hw_fn_id; @@ -89,13 +116,13 @@ int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) return -EOPNOTSUPP; mutex_lock(&table->table_lock); - sw_id = mlx5_sf_hw_table_id_alloc(table, usr_sfnum); + sw_id = mlx5_sf_hw_table_id_alloc(table, controller, usr_sfnum); if (sw_id < 0) { err = sw_id; goto exist_err; } - hw_fn_id = mlx5_sf_sw_to_hw_id(dev, sw_id); + hw_fn_id = mlx5_sf_sw_to_hw_id(dev, controller, sw_id); err = mlx5_cmd_alloc_sf(dev, hw_fn_id); if (err) goto err; @@ -104,38 +131,48 @@ int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum) if (err) goto vhca_err; + if (controller) { + /* If this SF is for external controller, SF manager + * needs to arm firmware to receive the events. + */ + err = mlx5_vhca_event_arm(dev, hw_fn_id); + if (err) + goto vhca_err; + } + mutex_unlock(&table->table_lock); return sw_id; vhca_err: mlx5_cmd_dealloc_sf(dev, hw_fn_id); err: - mlx5_sf_hw_table_id_free(table, sw_id); + mlx5_sf_hw_table_id_free(table, controller, sw_id); exist_err: mutex_unlock(&table->table_lock); return err; } -static void _mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id) +void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u32 controller, u16 id) { struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; u16 hw_fn_id; - hw_fn_id = mlx5_sf_sw_to_hw_id(dev, id); + mutex_lock(&table->table_lock); + hw_fn_id = mlx5_sf_sw_to_hw_id(dev, controller, id); mlx5_cmd_dealloc_sf(dev, hw_fn_id); - mlx5_sf_hw_table_id_free(table, id); + mlx5_sf_hw_table_id_free(table, controller, id); + mutex_unlock(&table->table_lock); } -void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id) +static void mlx5_sf_hw_table_hwc_sf_free(struct mlx5_core_dev *dev, + struct mlx5_sf_hwc_table *hwc, int idx) { - struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; - - mutex_lock(&table->table_lock); - _mlx5_sf_hw_table_sf_free(dev, id); - mutex_unlock(&table->table_lock); + mlx5_cmd_dealloc_sf(dev, hwc->start_fn_id + idx); + hwc->sfs[idx].allocated = false; + hwc->sfs[idx].pending_delete = false; } -void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id) +void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u32 controller, u16 id) { struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table; u32 out[MLX5_ST_SZ_DW(query_vhca_state_out)] = {}; @@ -144,8 +181,8 @@ void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id) u8 state; int err; - hw_fn_id = mlx5_sf_sw_to_hw_id(dev, id); - hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; + hw_fn_id = mlx5_sf_sw_to_hw_id(dev, controller, id); + hwc = mlx5_sf_controller_to_hwc(dev, controller); mutex_lock(&table->table_lock); err = mlx5_cmd_query_vhca_state(dev, hw_fn_id, out, sizeof(out)); if (err) @@ -168,12 +205,13 @@ static void mlx5_sf_hw_table_hwc_dealloc_all(struct mlx5_core_dev *dev, for (i = 0; i < hwc->max_fn; i++) { if (hwc->sfs[i].allocated) - _mlx5_sf_hw_table_sf_free(dev, i); + mlx5_sf_hw_table_hwc_sf_free(dev, hwc, i); } } static void mlx5_sf_hw_table_dealloc_all(struct mlx5_sf_hw_table *table) { + mlx5_sf_hw_table_hwc_dealloc_all(table->dev, &table->hwc[MLX5_SF_HWC_EXTERNAL]); mlx5_sf_hw_table_hwc_dealloc_all(table->dev, &table->hwc[MLX5_SF_HWC_LOCAL]); } @@ -181,6 +219,9 @@ static int mlx5_sf_hw_table_hwc_init(struct mlx5_sf_hwc_table *hwc, u16 max_fn, { struct mlx5_sf_hw *sfs; + if (!max_fn) + return 0; + sfs = kcalloc(max_fn, sizeof(*sfs), GFP_KERNEL); if (!sfs) return -ENOMEM; @@ -199,14 +240,25 @@ static void mlx5_sf_hw_table_hwc_cleanup(struct mlx5_sf_hwc_table *hwc) int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) { struct mlx5_sf_hw_table *table; + u16 max_ext_fn = 0; + u16 ext_base_id; + u16 max_fn = 0; u16 base_id; - u16 max_fn; int err; - if (!mlx5_sf_supported(dev) || !mlx5_vhca_event_supported(dev)) + if (!mlx5_vhca_event_supported(dev)) + return 0; + + if (mlx5_sf_supported(dev)) + max_fn = mlx5_sf_max_functions(dev); + + err = mlx5_esw_sf_max_hpf_functions(dev, &max_ext_fn, &ext_base_id); + if (err) + return err; + + if (!max_fn && !max_ext_fn) return 0; - max_fn = mlx5_sf_max_functions(dev); table = kzalloc(sizeof(*table), GFP_KERNEL); if (!table) return -ENOMEM; @@ -220,9 +272,16 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) if (err) goto table_err; - mlx5_core_dbg(dev, "SF HW table: max sfs = %d\n", max_fn); + err = mlx5_sf_hw_table_hwc_init(&table->hwc[MLX5_SF_HWC_EXTERNAL], + max_ext_fn, ext_base_id); + if (err) + goto ext_err; + + mlx5_core_dbg(dev, "SF HW table: max sfs = %d, ext sfs = %d\n", max_fn, max_ext_fn); return 0; +ext_err: + mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]); table_err: mutex_destroy(&table->table_lock); kfree(table); @@ -237,6 +296,7 @@ void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev) return; mutex_destroy(&table->table_lock); + mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_EXTERNAL]); mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]); kfree(table); } @@ -252,8 +312,11 @@ static int mlx5_sf_hw_vhca_event(struct notifier_block *nb, unsigned long opcode if (event->new_vhca_state != MLX5_VHCA_STATE_ALLOCATED) return 0; - hwc = &table->hwc[MLX5_SF_HWC_LOCAL]; - sw_id = mlx5_sf_hw_to_sw_id(table->dev, event->function_id); + hwc = mlx5_sf_table_fn_to_hwc(table, event->function_id); + if (!hwc) + return 0; + + sw_id = mlx5_sf_hw_to_sw_id(hwc, event->function_id); sf_hw = &hwc->sfs[sw_id]; mutex_lock(&table->table_lock); @@ -261,7 +324,7 @@ static int mlx5_sf_hw_vhca_event(struct notifier_block *nb, unsigned long opcode * Hence recycle the sf hardware id for reuse. */ if (sf_hw->allocated && sf_hw->pending_delete) - _mlx5_sf_hw_table_sf_free(table->dev, sw_id); + mlx5_sf_hw_table_hwc_sf_free(table->dev, hwc, sw_id); mutex_unlock(&table->table_lock); return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h index b36be5ecb496..7114f3fc335f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h @@ -12,11 +12,11 @@ int mlx5_cmd_dealloc_sf(struct mlx5_core_dev *dev, u16 function_id); int mlx5_cmd_sf_enable_hca(struct mlx5_core_dev *dev, u16 func_id); int mlx5_cmd_sf_disable_hca(struct mlx5_core_dev *dev, u16 func_id); -u16 mlx5_sf_sw_to_hw_id(const struct mlx5_core_dev *dev, u16 sw_id); +u16 mlx5_sf_sw_to_hw_id(struct mlx5_core_dev *dev, u32 controller, u16 sw_id); -int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 usr_sfnum); -void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u16 id); -void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u16 id); +int mlx5_sf_hw_table_sf_alloc(struct mlx5_core_dev *dev, u32 controller, u32 usr_sfnum); +void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u32 controller, u16 id); +void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u32 controller, u16 id); bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev); #endif