From patchwork Sun Aug 30 15:27:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F40AC433E6 for ; Sun, 30 Aug 2020 15:30:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 485C0208CA for ; Sun, 30 Aug 2020 15:30:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727831AbgH3PaE (ORCPT ); Sun, 30 Aug 2020 11:30:04 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42705 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726434AbgH3P2G (ORCPT ); Sun, 30 Aug 2020 11:28:06 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:54 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRsfd029618; Sun, 30 Aug 2020 18:27:54 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRslY027828; Sun, 30 Aug 2020 18:27:54 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRsMm027827; Sun, 30 Aug 2020 18:27:54 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 01/14] devlink: Add reload action option to devlink reload command Date: Sun, 30 Aug 2020 18:27:21 +0300 Message-Id: <1598801254-27764-2-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add devlink reload action to allow the user to request a specific reload action. The action parameter is optional, if not specified then devlink driver re-init action is used (backward compatible). Note that when required to do firmware activation some drivers may need to reload the driver. On the other hand some drivers may need to reset the firmware to reinitialize the driver entities. Therefore, the devlink reload command returns the actions which were actually done. However, in case fw_activate_no_reset action is selected, then no other reload action is allowed. Reload actions supported are: driver_reinit: driver entities re-initialization, applying devlink-param and devlink-resource values. fw_activate: firmware activate. fw_activate_no_reset: Activate new firmware image without any reset. (also known as: firmware live patching). command examples: $devlink dev reload pci/0000:82:00.0 action driver_reinit reload_actions_done: driver_reinit $devlink dev reload pci/0000:82:00.0 action fw_activate reload_actions_done: driver_reinit fw_activate Signed-off-by: Moshe Shemesh --- v2 -> v3: - Replace fw_live_patch action by fw_activate_no_reset - Devlink reload returns the actions done over netlink reply v1 -> v2: - Instead of reload levels driver,fw_reset,fw_live_patch have reload actions driver_reinit,fw_activate,fw_live_patch - Remove driver default level, the action driver_reinit is the default action for all drivers --- drivers/net/ethernet/mellanox/mlx4/main.c | 14 ++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 15 ++- drivers/net/ethernet/mellanox/mlxsw/core.c | 24 ++-- drivers/net/netdevsim/dev.c | 16 ++- include/net/devlink.h | 7 +- include/uapi/linux/devlink.h | 21 ++++ net/core/devlink.c | 112 +++++++++++++++++- 7 files changed, 182 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 258c7a96f269..3aeaa87556c8 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -3935,6 +3935,7 @@ static int mlx4_restart_one_up(struct pci_dev *pdev, bool reload, struct devlink *devlink); static int mlx4_devlink_reload_down(struct devlink *devlink, bool netns_change, + enum devlink_reload_action action, struct netlink_ext_ack *extack) { struct mlx4_priv *priv = devlink_priv(devlink); @@ -3951,8 +3952,8 @@ static int mlx4_devlink_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int mlx4_devlink_reload_up(struct devlink *devlink, - struct netlink_ext_ack *extack) +static int mlx4_devlink_reload_up(struct devlink *devlink, enum devlink_reload_action action, + struct netlink_ext_ack *extack, unsigned long *actions_done) { struct mlx4_priv *priv = devlink_priv(devlink); struct mlx4_dev *dev = &priv->dev; @@ -3960,15 +3961,20 @@ static int mlx4_devlink_reload_up(struct devlink *devlink, int err; err = mlx4_restart_one_up(persist->pdev, true, devlink); - if (err) + if (err) { mlx4_err(persist->dev, "mlx4_restart_one_up failed, ret=%d\n", err); + return err; + } + if (actions_done) + *actions_done = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); - return err; + return 0; } static const struct devlink_ops mlx4_devlink_ops = { .port_type_set = mlx4_devlink_port_type_set, + .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT), .reload_down = mlx4_devlink_reload_down, .reload_up = mlx4_devlink_reload_up, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index c709e9a385f6..e0f1a1a2c6a9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -89,6 +89,7 @@ mlx5_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, } static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, + enum devlink_reload_action action, struct netlink_ext_ack *extack) { struct mlx5_core_dev *dev = devlink_priv(devlink); @@ -97,12 +98,19 @@ static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int mlx5_devlink_reload_up(struct devlink *devlink, - struct netlink_ext_ack *extack) +static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_action action, + struct netlink_ext_ack *extack, unsigned long *actions_done) { struct mlx5_core_dev *dev = devlink_priv(devlink); + int err; - return mlx5_load_one(dev, false); + err = mlx5_load_one(dev, false); + if (err) + return err; + if (actions_done) + *actions_done = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + + return 0; } static const struct devlink_ops mlx5_devlink_ops = { @@ -118,6 +126,7 @@ static const struct devlink_ops mlx5_devlink_ops = { #endif .flash_update = mlx5_devlink_flash_update, .info_get = mlx5_devlink_info_get, + .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT), .reload_down = mlx5_devlink_reload_down, .reload_up = mlx5_devlink_reload_up, }; diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 08d101138fbe..c42b66d88884 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -1113,7 +1113,7 @@ mlxsw_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, static int mlxsw_devlink_core_bus_device_reload_down(struct devlink *devlink, - bool netns_change, + bool netns_change, enum devlink_reload_action action, struct netlink_ext_ack *extack) { struct mlxsw_core *mlxsw_core = devlink_priv(devlink); @@ -1126,15 +1126,23 @@ mlxsw_devlink_core_bus_device_reload_down(struct devlink *devlink, } static int -mlxsw_devlink_core_bus_device_reload_up(struct devlink *devlink, - struct netlink_ext_ack *extack) +mlxsw_devlink_core_bus_device_reload_up(struct devlink *devlink, enum devlink_reload_action action, + struct netlink_ext_ack *extack, unsigned long *actions_done) { struct mlxsw_core *mlxsw_core = devlink_priv(devlink); + int err; - return mlxsw_core_bus_device_register(mlxsw_core->bus_info, - mlxsw_core->bus, - mlxsw_core->bus_priv, true, - devlink, extack); + err = mlxsw_core_bus_device_register(mlxsw_core->bus_info, + mlxsw_core->bus, + mlxsw_core->bus_priv, true, + devlink, extack); + if (err) + return err; + if (actions_done) + *actions_done = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) | + BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE); + + return 0; } static int mlxsw_devlink_flash_update(struct devlink *devlink, @@ -1268,6 +1276,8 @@ mlxsw_devlink_trap_policer_counter_get(struct devlink *devlink, } static const struct devlink_ops mlxsw_devlink_ops = { + .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) | + BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), .reload_down = mlxsw_devlink_core_bus_device_reload_down, .reload_up = mlxsw_devlink_core_bus_device_reload_up, .port_type_set = mlxsw_devlink_port_type_set, diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index 32f339fedb21..e2166e755659 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -697,7 +697,7 @@ static int nsim_dev_reload_create(struct nsim_dev *nsim_dev, static void nsim_dev_reload_destroy(struct nsim_dev *nsim_dev); static int nsim_dev_reload_down(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack) + enum devlink_reload_action action, struct netlink_ext_ack *extack) { struct nsim_dev *nsim_dev = devlink_priv(devlink); @@ -713,10 +713,11 @@ static int nsim_dev_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int nsim_dev_reload_up(struct devlink *devlink, - struct netlink_ext_ack *extack) +static int nsim_dev_reload_up(struct devlink *devlink, enum devlink_reload_action action, + struct netlink_ext_ack *extack, unsigned long *actions_done) { struct nsim_dev *nsim_dev = devlink_priv(devlink); + int err; if (nsim_dev->fail_reload) { /* For testing purposes, user set debugfs fail_reload @@ -726,7 +727,13 @@ static int nsim_dev_reload_up(struct devlink *devlink, return -EINVAL; } - return nsim_dev_reload_create(nsim_dev, extack); + err = nsim_dev_reload_create(nsim_dev, extack); + if (err) + return err; + if (actions_done) + *actions_done = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + + return 0; } static int nsim_dev_info_get(struct devlink *devlink, @@ -875,6 +882,7 @@ nsim_dev_devlink_trap_policer_counter_get(struct devlink *devlink, } static const struct devlink_ops nsim_dev_devlink_ops = { + .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT), .reload_down = nsim_dev_reload_down, .reload_up = nsim_dev_reload_up, .info_get = nsim_dev_info_get, diff --git a/include/net/devlink.h b/include/net/devlink.h index 8f3c8a443238..b8f0152a1fff 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -991,10 +991,11 @@ enum devlink_trap_group_generic_id { } struct devlink_ops { + unsigned long supported_reload_actions; int (*reload_down)(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack); - int (*reload_up)(struct devlink *devlink, - struct netlink_ext_ack *extack); + enum devlink_reload_action action, struct netlink_ext_ack *extack); + int (*reload_up)(struct devlink *devlink, enum devlink_reload_action action, + struct netlink_ext_ack *extack, unsigned long *actions_done); int (*port_type_set)(struct devlink_port *devlink_port, enum devlink_port_type port_type); int (*port_split)(struct devlink *devlink, unsigned int port_index, diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index cfef4245ea5a..0a438135c3cf 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -272,6 +272,24 @@ enum { DEVLINK_ATTR_TRAP_METADATA_TYPE_FA_COOKIE, }; +/** + * enum devlink_reload_action - Reload action. + * @DEVLINK_RELOAD_ACTION_DRIVER_REINIT: Driver entities re-instantiation. + * @DEVLINK_RELOAD_ACTION_FW_ACTIVATE: FW activate. + * @DEVLINK_RELOAD_ACTION_FW_ACTIVATE_NO_RESET: Activate new FW image withouti any reset + * (firmware live patching). + */ +enum devlink_reload_action { + DEVLINK_RELOAD_ACTION_UNSPEC, + DEVLINK_RELOAD_ACTION_DRIVER_REINIT, + DEVLINK_RELOAD_ACTION_FW_ACTIVATE, + DEVLINK_RELOAD_ACTION_FW_ACTIVATE_NO_RESET, + + /* Add new reload actions above */ + __DEVLINK_RELOAD_ACTION_MAX, + DEVLINK_RELOAD_ACTION_MAX = __DEVLINK_RELOAD_ACTION_MAX - 1 +}; + enum devlink_attr { /* don't change the order or add anything between, this is ABI! */ DEVLINK_ATTR_UNSPEC, @@ -458,6 +476,9 @@ enum devlink_attr { DEVLINK_ATTR_PORT_LANES, /* u32 */ DEVLINK_ATTR_PORT_SPLITTABLE, /* u8 */ + DEVLINK_ATTR_RELOAD_ACTION, /* u8 */ + DEVLINK_ATTR_RELOAD_ACTIONS_DONE, /* nested */ + /* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX, diff --git a/net/core/devlink.c b/net/core/devlink.c index 58c8bb07fa19..8d4137ad40db 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -462,6 +462,12 @@ static int devlink_nl_put_handle(struct sk_buff *msg, struct devlink *devlink) return 0; } +static bool +devlink_reload_action_is_supported(struct devlink *devlink, enum devlink_reload_action action) +{ + return test_bit(action, &devlink->ops->supported_reload_actions); +} + static int devlink_nl_fill(struct sk_buff *msg, struct devlink *devlink, enum devlink_command cmd, u32 portid, u32 seq, int flags) @@ -2964,29 +2970,72 @@ bool devlink_is_reload_failed(const struct devlink *devlink) EXPORT_SYMBOL_GPL(devlink_is_reload_failed); static int devlink_reload(struct devlink *devlink, struct net *dest_net, - struct netlink_ext_ack *extack) + enum devlink_reload_action action, struct netlink_ext_ack *extack, + unsigned long *actions_done) { int err; if (!devlink->reload_enabled) return -EOPNOTSUPP; - err = devlink->ops->reload_down(devlink, !!dest_net, extack); + err = devlink->ops->reload_down(devlink, !!dest_net, action, extack); if (err) return err; if (dest_net && !net_eq(dest_net, devlink_net(devlink))) devlink_reload_netns_change(devlink, dest_net); - err = devlink->ops->reload_up(devlink, extack); + err = devlink->ops->reload_up(devlink, action, extack, actions_done); devlink_reload_failed_set(devlink, !!err); return err; } +static int +devlink_nl_reload_actions_done_fill(struct sk_buff *msg, + struct devlink *devlink, + unsigned long actions_done, + enum devlink_command cmd, u32 portid, + u32 seq, int flags) +{ + struct nlattr *actions_done_attr; + void *hdr; + int i; + + hdr = genlmsg_put(msg, portid, seq, &devlink_nl_family, flags, cmd); + if (!hdr) + return -EMSGSIZE; + + if (devlink_nl_put_handle(msg, devlink)) + goto genlmsg_cancel; + + actions_done_attr = nla_nest_start(msg, DEVLINK_ATTR_RELOAD_ACTIONS_DONE); + if (!actions_done_attr) + goto genlmsg_cancel; + + for (i = 0; i <= DEVLINK_RELOAD_ACTION_MAX; i++) { + if (!test_bit(i, &actions_done)) + continue; + if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_ACTION, i)) + goto actions_done_nest_cancel; + } + nla_nest_end(msg, actions_done_attr); + genlmsg_end(msg, hdr); + return 0; + +actions_done_nest_cancel: + nla_nest_cancel(msg, actions_done_attr); +genlmsg_cancel: + genlmsg_cancel(msg, hdr); + return -EMSGSIZE; +} + static int devlink_nl_cmd_reload(struct sk_buff *skb, struct genl_info *info) { struct devlink *devlink = info->user_ptr[0]; + enum devlink_reload_action action; struct net *dest_net = NULL; + unsigned long actions_done; + struct sk_buff *msg; int err; if (!devlink_reload_supported(devlink)) @@ -3006,12 +3055,42 @@ static int devlink_nl_cmd_reload(struct sk_buff *skb, struct genl_info *info) return PTR_ERR(dest_net); } - err = devlink_reload(devlink, dest_net, info->extack); + if (info->attrs[DEVLINK_ATTR_RELOAD_ACTION]) + action = nla_get_u8(info->attrs[DEVLINK_ATTR_RELOAD_ACTION]); + else + action = DEVLINK_RELOAD_ACTION_DRIVER_REINIT; + + if (action == DEVLINK_RELOAD_ACTION_UNSPEC || action > DEVLINK_RELOAD_ACTION_MAX) { + NL_SET_ERR_MSG_MOD(info->extack, "Invalid reload action"); + return -EINVAL; + } else if (!devlink_reload_action_is_supported(devlink, action)) { + NL_SET_ERR_MSG_MOD(info->extack, "Requested reload action is not supported"); + return -EOPNOTSUPP; + } + + err = devlink_reload(devlink, dest_net, action, info->extack, &actions_done); if (dest_net) put_net(dest_net); - return err; + if (err) + return err; + + WARN_ON(!actions_done); + if (action == DEVLINK_RELOAD_ACTION_FW_ACTIVATE_NO_RESET) + WARN_ON(actions_done != BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE_NO_RESET)); + msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + err = devlink_nl_reload_actions_done_fill(msg, devlink, actions_done, DEVLINK_CMD_RELOAD, + info->snd_portid, info->snd_seq, 0); + if (err) { + nlmsg_free(msg); + return err; + } + + return genlmsg_reply(msg, info); } static int devlink_nl_flash_update_fill(struct sk_buff *msg, @@ -7042,6 +7121,7 @@ static const struct nla_policy devlink_nl_policy[DEVLINK_ATTR_MAX + 1] = { [DEVLINK_ATTR_TRAP_POLICER_RATE] = { .type = NLA_U64 }, [DEVLINK_ATTR_TRAP_POLICER_BURST] = { .type = NLA_U64 }, [DEVLINK_ATTR_PORT_FUNCTION] = { .type = NLA_NESTED }, + [DEVLINK_ATTR_RELOAD_ACTION] = { .type = NLA_U8 }, }; static const struct genl_ops devlink_nl_ops[] = { @@ -7367,6 +7447,20 @@ static struct genl_family devlink_nl_family __ro_after_init = { .n_mcgrps = ARRAY_SIZE(devlink_nl_mcgrps), }; +static int devlink_reload_actions_verify(struct devlink *devlink) +{ + const struct devlink_ops *ops; + + if (!devlink_reload_supported(devlink)) + return 0; + + ops = devlink->ops; + if (WARN_ON(ops->supported_reload_actions >= BIT(__DEVLINK_RELOAD_ACTION_MAX) || + ops->supported_reload_actions <= BIT(DEVLINK_RELOAD_ACTION_UNSPEC))) + return -EINVAL; + return 0; +} + /** * devlink_alloc - Allocate new devlink instance resources * @@ -7387,6 +7481,11 @@ struct devlink *devlink_alloc(const struct devlink_ops *ops, size_t priv_size) if (!devlink) return NULL; devlink->ops = ops; + if (devlink_reload_actions_verify(devlink)) { + kfree(devlink); + return NULL; + } + xa_init_flags(&devlink->snapshot_ids, XA_FLAGS_ALLOC); __devlink_net_set(devlink, &init_net); INIT_LIST_HEAD(&devlink->port_list); @@ -9618,7 +9717,8 @@ static void __net_exit devlink_pernet_pre_exit(struct net *net) if (net_eq(devlink_net(devlink), net)) { if (WARN_ON(!devlink_reload_supported(devlink))) continue; - err = devlink_reload(devlink, &init_net, NULL); + err = devlink_reload(devlink, &init_net, + DEVLINK_RELOAD_ACTION_DRIVER_REINIT, NULL, NULL); if (err && err != -EOPNOTSUPP) pr_warn("Failed to reload devlink instance into init_net\n"); } From patchwork Sun Aug 30 15:27:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AA25C433E7 for ; Sun, 30 Aug 2020 15:30:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7CBE620707 for ; Sun, 30 Aug 2020 15:30:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727940AbgH3Pa4 (ORCPT ); Sun, 30 Aug 2020 11:30:56 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42707 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726479AbgH3P2C (ORCPT ); Sun, 30 Aug 2020 11:28:02 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:54 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRsE5029621; Sun, 30 Aug 2020 18:27:54 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRsqE027830; Sun, 30 Aug 2020 18:27:54 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRsTJ027829; Sun, 30 Aug 2020 18:27:54 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 02/14] devlink: Add reload actions counters Date: Sun, 30 Aug 2020 18:27:22 +0300 Message-Id: <1598801254-27764-3-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add reload actions counters to hold the history per reload action type. For example, the number of times fw_activate has been done on this device since the driver module was added or if the firmware activation was done with or without reset. The function devlink_reload_actions_cnts_update() is exported to enable also drivers update on reload actions done, for example in case firmware activation with reset finished successfully but was initiated by remote host. Signed-off-by: Moshe Shemesh --- v2 -> v3: - New patch --- include/net/devlink.h | 2 ++ net/core/devlink.c | 24 +++++++++++++++++++++--- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/include/net/devlink.h b/include/net/devlink.h index b8f0152a1fff..0547f0707d92 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -38,6 +38,7 @@ struct devlink { struct list_head trap_policer_list; const struct devlink_ops *ops; struct xarray snapshot_ids; + u32 reload_actions_cnts[DEVLINK_RELOAD_ACTION_MAX]; struct device *dev; possible_net_t _net; struct mutex lock; /* Serializes access to devlink instance specific objects such as @@ -1372,6 +1373,7 @@ void devlink_health_reporter_recovery_done(struct devlink_health_reporter *reporter); bool devlink_is_reload_failed(const struct devlink *devlink); +void devlink_reload_actions_cnts_update(struct devlink *devlink, unsigned long actions_done); void devlink_flash_update_begin_notify(struct devlink *devlink); void devlink_flash_update_end_notify(struct devlink *devlink); diff --git a/net/core/devlink.c b/net/core/devlink.c index 8d4137ad40db..20a29c34ff71 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -2969,10 +2969,23 @@ bool devlink_is_reload_failed(const struct devlink *devlink) } EXPORT_SYMBOL_GPL(devlink_is_reload_failed); +void devlink_reload_actions_cnts_update(struct devlink *devlink, unsigned long actions_done) +{ + int action; + + for (action = 0; action < DEVLINK_RELOAD_ACTION_MAX; action++) { + if (!test_bit(action, &actions_done)) + continue; + devlink->reload_actions_cnts[action]++; + } +} +EXPORT_SYMBOL_GPL(devlink_reload_actions_cnts_update); + static int devlink_reload(struct devlink *devlink, struct net *dest_net, enum devlink_reload_action action, struct netlink_ext_ack *extack, - unsigned long *actions_done) + unsigned long *actions_done_out) { + unsigned long actions_done; int err; if (!devlink->reload_enabled) @@ -2985,9 +2998,14 @@ static int devlink_reload(struct devlink *devlink, struct net *dest_net, if (dest_net && !net_eq(dest_net, devlink_net(devlink))) devlink_reload_netns_change(devlink, dest_net); - err = devlink->ops->reload_up(devlink, action, extack, actions_done); + err = devlink->ops->reload_up(devlink, action, extack, &actions_done); devlink_reload_failed_set(devlink, !!err); - return err; + if (err) + return err; + devlink_reload_actions_cnts_update(devlink, actions_done); + if (actions_done_out) + *actions_done_out = actions_done; + return 0; } static int From patchwork Sun Aug 30 15:27:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FF12C433E2 for ; Sun, 30 Aug 2020 15:32:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5608E2071B for ; Sun, 30 Aug 2020 15:32:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727784AbgH3PcW (ORCPT ); Sun, 30 Aug 2020 11:32:22 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42728 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726571AbgH3P2B (ORCPT ); Sun, 30 Aug 2020 11:28:01 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:55 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRtiD029632; Sun, 30 Aug 2020 18:27:55 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRtN4027839; Sun, 30 Aug 2020 18:27:55 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRtXw027838; Sun, 30 Aug 2020 18:27:55 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 05/14] net/mlx5: Set cap for pci sync for fw update event Date: Sun, 30 Aug 2020 18:27:25 +0300 Message-Id: <1598801254-27764-6-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Set capability to notify the firmware that this host driver is capable of handling pci sync for firmware update events. Signed-off-by: Moshe Shemesh --- drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index ce43e3feccd9..871d28b09f8a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -548,6 +548,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx) if (MLX5_CAP_GEN_MAX(dev, dct)) MLX5_SET(cmd_hca_cap, set_hca_cap, dct, 1); + if (MLX5_CAP_GEN_MAX(dev, pci_sync_for_fw_update_event)) + MLX5_SET(cmd_hca_cap, set_hca_cap, pci_sync_for_fw_update_event, 1); + if (MLX5_CAP_GEN_MAX(dev, num_vhca_ports)) MLX5_SET(cmd_hca_cap, set_hca_cap, From patchwork Sun Aug 30 15:27:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A66AEC433E6 for ; Sun, 30 Aug 2020 15:33:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CD392071B for ; Sun, 30 Aug 2020 15:33:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727981AbgH3Pd0 (ORCPT ); Sun, 30 Aug 2020 11:33:26 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42746 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726589AbgH3P2B (ORCPT ); Sun, 30 Aug 2020 11:28:01 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRtXE029649; Sun, 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRtLX027845; Sun, 30 Aug 2020 18:27:55 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRtJx027844; Sun, 30 Aug 2020 18:27:55 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 08/14] net/mlx5: Handle sync reset abort event Date: Sun, 30 Aug 2020 18:27:28 +0300 Message-Id: <1598801254-27764-9-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If firmware sends sync_reset_abort to driver the driver should clear the reset requested mode as reset is not expected any more. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/fw_reset.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index f9e293de9de3..61237f4836cc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -14,6 +14,7 @@ struct mlx5_fw_reset { struct work_struct reset_request_work; struct work_struct reset_reload_work; struct work_struct reset_now_work; + struct work_struct reset_abort_work; unsigned long reset_flags; struct timer_list timer; }; @@ -266,6 +267,16 @@ static void mlx5_sync_reset_now_event(struct work_struct *work) mlx5_load_one(dev, false); } +static void mlx5_sync_reset_abort_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + reset_abort_work); + struct mlx5_core_dev *dev = fw_reset->dev; + + mlx5_sync_reset_clear_reset_requested(dev, true); + mlx5_core_warn(dev, "PCI Sync FW Update Reset Aborted.\n"); +} + static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct mlx5_eqe *eqe) { struct mlx5_eqe_sync_fw_update *sync_fw_update_eqe; @@ -280,6 +291,9 @@ static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct case MLX5_SYNC_RST_STATE_RESET_NOW: queue_work(fw_reset->wq, &fw_reset->reset_now_work); break; + case MLX5_SYNC_RST_STATE_RESET_ABORT: + queue_work(fw_reset->wq, &fw_reset->reset_abort_work); + break; } } @@ -317,6 +331,7 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); INIT_WORK(&fw_reset->reset_reload_work, mlx5_sync_reset_reload_work); INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); + INIT_WORK(&fw_reset->reset_abort_work, mlx5_sync_reset_abort_event); MLX5_NB_INIT(&fw_reset->nb, fw_reset_event_notifier, GENERAL_EVENT); mlx5_eq_notifier_register(dev, &fw_reset->nb); From patchwork Sun Aug 30 15:27:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C750AC433E6 for ; Sun, 30 Aug 2020 15:30:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A85C3206F0 for ; Sun, 30 Aug 2020 15:30:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726447AbgH3PaZ (ORCPT ); Sun, 30 Aug 2020 11:30:25 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42752 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726618AbgH3P2E (ORCPT ); Sun, 30 Aug 2020 11:28:04 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRuTc029653; Sun, 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRuXE027847; Sun, 30 Aug 2020 18:27:56 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRuFA027846; Sun, 30 Aug 2020 18:27:56 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 09/14] net/mlx5: Add support for devlink reload action fw activate Date: Sun, 30 Aug 2020 18:27:29 +0300 Message-Id: <1598801254-27764-10-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for devlink reload action fw_activate. To activate firmware image the mlx5 driver resets the firmware and reloads it from flash. If a new image was stored on flash it will be loaded. Once this reload command is executed the driver initiates fw sync reset flow, where the firmware synchronizes all PFs on coming reset and driver reload. Signed-off-by: Moshe Shemesh --- v2 -> v3: - Return the reload actions done - Update reload action counters if reset initiated by remote host v1 -> v2: - Have fw_activate action instead of fw_reset level --- .../net/ethernet/mellanox/mlx5/core/devlink.c | 63 ++++++++++++++++--- .../ethernet/mellanox/mlx5/core/fw_reset.c | 59 +++++++++++++++-- .../ethernet/mellanox/mlx5/core/fw_reset.h | 1 + 3 files changed, 109 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index e0f1a1a2c6a9..f2f1f4e4ef25 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -4,6 +4,7 @@ #include #include "mlx5_core.h" +#include "fw_reset.h" #include "fs_core.h" #include "eswitch.h" @@ -88,14 +89,49 @@ mlx5_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, return 0; } +static int mlx5_devlink_reload_fw_activate(struct devlink *devlink, struct netlink_ext_ack *extack) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + u8 reset_level, reset_type, net_port_alive; + int err; + + err = mlx5_reg_mfrl_query(dev, &reset_level, &reset_type); + if (err) + return err; + if (!(reset_level & MLX5_MFRL_REG_RESET_LEVEL3)) { + NL_SET_ERR_MSG_MOD(extack, "FW activate requires reboot"); + return -EINVAL; + } + + net_port_alive = !!(reset_type & MLX5_MFRL_REG_RESET_TYPE_NET_PORT_ALIVE); + err = mlx5_fw_set_reset_sync(dev, net_port_alive); + if (err) + goto out; + + err = mlx5_fw_wait_fw_reset_done(dev); +out: + if (err) + NL_SET_ERR_MSG_MOD(extack, "FW activate command failed"); + return err; +} + static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, enum devlink_reload_action action, struct netlink_ext_ack *extack) { struct mlx5_core_dev *dev = devlink_priv(devlink); - mlx5_unload_one(dev, false); - return 0; + switch (action) { + case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: + mlx5_unload_one(dev, false); + return 0; + case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: + return mlx5_devlink_reload_fw_activate(devlink, extack); + default: + /* Unsupported action should not get to this function */ + WARN_ON(1); + return -EOPNOTSUPP; + } } static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_action action, @@ -104,11 +140,21 @@ static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_a struct mlx5_core_dev *dev = devlink_priv(devlink); int err; - err = mlx5_load_one(dev, false); - if (err) - return err; - if (actions_done) - *actions_done = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + switch (action) { + case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: + case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: + err = mlx5_load_one(dev, false); + if (err) + return err; + /* On fw_activate action, also driver is reloaded and reinit done */ + if (actions_done) + *actions_done = BIT(action) | BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + break; + default: + /* Unsupported action should not get to this function */ + WARN_ON(1); + return -EOPNOTSUPP; + } return 0; } @@ -126,7 +172,8 @@ static const struct devlink_ops mlx5_devlink_ops = { #endif .flash_update = mlx5_devlink_flash_update, .info_get = mlx5_devlink_info_get, - .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT), + .supported_reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) | + BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), .reload_down = mlx5_devlink_reload_down, .reload_up = mlx5_devlink_reload_up, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 61237f4836cc..1f9ad1f38d92 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -5,6 +5,7 @@ enum { MLX5_FW_RESET_FLAGS_RESET_REQUESTED, + MLX5_FW_RESET_FLAGS_PENDING_COMP }; struct mlx5_fw_reset { @@ -17,6 +18,8 @@ struct mlx5_fw_reset { struct work_struct reset_abort_work; unsigned long reset_flags; struct timer_list timer; + struct completion done; + int ret; }; static int mlx5_reg_mfrl_set(struct mlx5_core_dev *dev, u8 reset_level, @@ -53,7 +56,14 @@ int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_ty int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel) { - return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, reset_type_sel, 0, true); + struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset; + int err; + + set_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags); + err = mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, reset_type_sel, 0, true); + if (err) + clear_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags); + return err; } int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev) @@ -66,19 +76,35 @@ static int mlx5_fw_set_reset_sync_ack(struct mlx5_core_dev *dev) return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, 0, 1, false); } +static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev) +{ + struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset; + + /* if this is the driver that initiated the fw reset, devlink completed the reload */ + if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags)) { + complete(&fw_reset->done); + } else { + mlx5_load_one(dev, false); + devlink_reload_actions_cnts_update(priv_to_devlink(dev), + BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) | + BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE)); + } +} + static void mlx5_sync_reset_reload_work(struct work_struct *work) { struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, reset_reload_work); struct mlx5_core_dev *dev = fw_reset->dev; + int err; mlx5_enter_error_state(dev, true); mlx5_unload_one(dev, false); - if (mlx5_health_wait_pci_up(dev)) { + err = mlx5_health_wait_pci_up(dev); + if (err) mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n"); - return; - } - mlx5_load_one(dev, false); + fw_reset->ret = err; + mlx5_fw_reset_complete_reload(dev); } static void mlx5_stop_sync_reset_poll(struct mlx5_core_dev *dev) @@ -264,7 +290,8 @@ static void mlx5_sync_reset_now_event(struct work_struct *work) done: if (err) mlx5_start_health_poll(dev); - mlx5_load_one(dev, false); + fw_reset->ret = err; + mlx5_fw_reset_complete_reload(dev); } static void mlx5_sync_reset_abort_event(struct work_struct *work) @@ -313,6 +340,25 @@ static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long acti return NOTIFY_OK; } +#define MLX5_FW_RESET_TIMEOUT_MSEC 5000 +int mlx5_fw_wait_fw_reset_done(struct mlx5_core_dev *dev) +{ + unsigned long timeout = msecs_to_jiffies(MLX5_FW_RESET_TIMEOUT_MSEC); + struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset; + int err; + + if (!wait_for_completion_timeout(&fw_reset->done, timeout)) { + mlx5_core_warn(dev, "FW sync reset timeout after %d seconds\n", + MLX5_FW_RESET_TIMEOUT_MSEC / 1000); + err = -ETIMEDOUT; + goto out; + } + err = fw_reset->ret; +out: + clear_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags); + return err; +} + int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) { struct mlx5_fw_reset *fw_reset = kzalloc(sizeof(*fw_reset), GFP_KERNEL); @@ -336,6 +382,7 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) MLX5_NB_INIT(&fw_reset->nb, fw_reset_event_notifier, GENERAL_EVENT); mlx5_eq_notifier_register(dev, &fw_reset->nb); + init_completion(&fw_reset->done); return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h index 278f538ea92a..d7ee951a2258 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h @@ -10,6 +10,7 @@ int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_ty int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel); int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev); +int mlx5_fw_wait_fw_reset_done(struct mlx5_core_dev *dev); int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev); void mlx5_fw_reset_events_cleanup(struct mlx5_core_dev *dev); From patchwork Sun Aug 30 15:27:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5015C433E2 for ; Sun, 30 Aug 2020 15:29:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA6EB208CA for ; Sun, 30 Aug 2020 15:29:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727078AbgH3P3K (ORCPT ); Sun, 30 Aug 2020 11:29:10 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42768 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726726AbgH3P2E (ORCPT ); Sun, 30 Aug 2020 11:28:04 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRuf1029672; Sun, 30 Aug 2020 18:27:56 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRuBq027853; Sun, 30 Aug 2020 18:27:56 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRute027852; Sun, 30 Aug 2020 18:27:56 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 12/14] net/mlx5: Add support for fw live patch event Date: Sun, 30 Aug 2020 18:27:32 +0300 Message-Id: <1598801254-27764-13-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Firmware live patch event notifies the driver that the firmware was just updated using live patch. In such case the driver should not reload or re-initiate entities, part to updating the firmware version and re-initiate the firmware tracer which can be updated by live patch with new strings database to help debugging an issue. Signed-off-by: Moshe Shemesh --- .../mellanox/mlx5/core/diag/fw_tracer.c | 31 +++++++++++++++++++ .../mellanox/mlx5/core/diag/fw_tracer.h | 1 + .../ethernet/mellanox/mlx5/core/fw_reset.c | 27 ++++++++++++++++ include/linux/mlx5/device.h | 1 + 4 files changed, 60 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c index ad3594c4afcb..08dae045d185 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c @@ -1064,6 +1064,37 @@ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer) kvfree(tracer); } +int mlx5_fw_tracer_recreate_strings_db(struct mlx5_fw_tracer *tracer) +{ + struct mlx5_core_dev *dev; + int err; + + if (IS_ERR_OR_NULL(tracer)) + return -EINVAL; + + cancel_work_sync(&tracer->read_fw_strings_work); + mlx5_fw_tracer_clean_ready_list(tracer); + mlx5_fw_tracer_clean_print_hash(tracer); + mlx5_fw_tracer_clean_saved_traces_array(tracer); + mlx5_fw_tracer_free_strings_db(tracer); + + dev = tracer->dev; + err = mlx5_query_mtrc_caps(tracer); + if (err) { + mlx5_core_dbg(dev, "FWTracer: Failed to query capabilities %d\n", err); + return err; + } + + err = mlx5_fw_tracer_allocate_strings_db(tracer); + if (err) { + mlx5_core_warn(dev, "FWTracer: Allocate strings DB failed %d\n", err); + return err; + } + mlx5_fw_tracer_init_saved_traces_array(tracer); + + return 0; +} + static int fw_tracer_event(struct notifier_block *nb, unsigned long action, void *data) { struct mlx5_fw_tracer *tracer = mlx5_nb_cof(nb, struct mlx5_fw_tracer, nb); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h index 40601fba80ba..1a755098aeeb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h @@ -191,5 +191,6 @@ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer); int mlx5_fw_tracer_trigger_core_dump_general(struct mlx5_core_dev *dev); int mlx5_fw_tracer_get_saved_traces_objects(struct mlx5_fw_tracer *tracer, struct devlink_fmsg *fmsg); +int mlx5_fw_tracer_recreate_strings_db(struct mlx5_fw_tracer *tracer); #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 0df2ad9d555c..13a5e226d331 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -2,6 +2,7 @@ /* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ #include "fw_reset.h" +#include "diag/fw_tracer.h" enum { MLX5_FW_RESET_FLAGS_RESET_REQUESTED, @@ -13,6 +14,7 @@ struct mlx5_fw_reset { struct mlx5_core_dev *dev; struct mlx5_nb nb; struct workqueue_struct *wq; + struct work_struct fw_live_patch_work; struct work_struct reset_request_work; struct work_struct reset_reload_work; struct work_struct reset_now_work; @@ -187,6 +189,27 @@ static void mlx5_sync_reset_set_reset_requested(struct mlx5_core_dev *dev) mlx5_start_sync_reset_poll(dev); } +static void mlx5_fw_live_patch_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + fw_live_patch_work); + struct mlx5_core_dev *dev = fw_reset->dev; + struct mlx5_fw_tracer *tracer; + + mlx5_core_info(dev, "Live patch updated firmware version: %d.%d.%d\n", fw_rev_maj(dev), + fw_rev_min(dev), fw_rev_sub(dev)); + + tracer = dev->tracer; + if (IS_ERR_OR_NULL(tracer)) + return; + + mlx5_fw_tracer_cleanup(tracer); + if (mlx5_fw_tracer_recreate_strings_db(tracer)) + mlx5_core_err(dev, "Failed to recreate FW tracer strings DB\n"); + if (mlx5_fw_tracer_init(tracer)) + mlx5_core_err(dev, "Failed to re-initialize FW tracer\n"); +} + static void mlx5_sync_reset_request_event(struct work_struct *work) { struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, @@ -360,6 +383,9 @@ static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long acti struct mlx5_eqe *eqe = data; switch (eqe->sub_type) { + case MLX5_GENERAL_SUBTYPE_FW_LIVE_PATCH_EVENT: + queue_work(fw_reset->wq, &fw_reset->fw_live_patch_work); + break; case MLX5_GENERAL_SUBTYPE_PCI_SYNC_FOR_FW_UPDATE_EVENT: mlx5_sync_reset_events_handle(fw_reset, eqe); break; @@ -404,6 +430,7 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) fw_reset->dev = dev; dev->priv.fw_reset = fw_reset; + INIT_WORK(&fw_reset->fw_live_patch_work, mlx5_fw_live_patch_event); INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); INIT_WORK(&fw_reset->reset_reload_work, mlx5_sync_reset_reload_work); INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index 4d3376e20f5e..ab5bedd9d3d3 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -366,6 +366,7 @@ enum { enum { MLX5_GENERAL_SUBTYPE_DELAY_DROP_TIMEOUT = 0x1, MLX5_GENERAL_SUBTYPE_PCI_POWER_CHANGE_EVENT = 0x5, + MLX5_GENERAL_SUBTYPE_FW_LIVE_PATCH_EVENT = 0x7, MLX5_GENERAL_SUBTYPE_PCI_SYNC_FOR_FW_UPDATE_EVENT = 0x8, }; From patchwork Sun Aug 30 15:27:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 261758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D1DCC433E2 for ; Sun, 30 Aug 2020 15:28:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1FCF420738 for ; Sun, 30 Aug 2020 15:28:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726968AbgH3P2p (ORCPT ); Sun, 30 Aug 2020 11:28:45 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:42780 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726814AbgH3P2B (ORCPT ); Sun, 30 Aug 2020 11:28:01 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 30 Aug 2020 18:27:57 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 07UFRvvG029680; Sun, 30 Aug 2020 18:27:57 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 07UFRv05027857; Sun, 30 Aug 2020 18:27:57 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 07UFRvd1027856; Sun, 30 Aug 2020 18:27:57 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski Cc: Jiri Pirko , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC v3 14/14] devlink: Add Documentation/networking/devlink/devlink-reload.rst Date: Sun, 30 Aug 2020 18:27:34 +0300 Message-Id: <1598801254-27764-15-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1598801254-27764-1-git-send-email-moshe@mellanox.com> References: <1598801254-27764-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add devlink reload rst documentation file. Update index file to include it. Signed-off-by: Moshe Shemesh --- v2 -> v3: - Devlink reload returns the actions done - Replace fw_live_patch action by fw_activate_no_reset - Explain fw_activate meaning v1 -> v2: - Instead of reload levels driver,fw_reset,fw_live_patch have reload actions driver_reinit,fw_activate,fw_live_patch --- .../networking/devlink/devlink-reload.rst | 68 +++++++++++++++++++ Documentation/networking/devlink/index.rst | 1 + 2 files changed, 69 insertions(+) create mode 100644 Documentation/networking/devlink/devlink-reload.rst diff --git a/Documentation/networking/devlink/devlink-reload.rst b/Documentation/networking/devlink/devlink-reload.rst new file mode 100644 index 000000000000..44a18692bddd --- /dev/null +++ b/Documentation/networking/devlink/devlink-reload.rst @@ -0,0 +1,68 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============== +Devlink Reload +============== + +``devlink-reload`` provides mechanism to either reload driver entities, +applying ``devlink-params`` and ``devlink-resources`` new values or firmware +activation depends on reload action selected. + +Reload actions +============== + +User may select a reload action. +By default ``driver_reinit`` action is done. + +.. list-table:: Possible reload levels + :widths: 5 90 + + * - Name + - Description + * - ``driver-reinit`` + - Devlink driver entities re-initialization, including applying + new values to devlink entities which are used during driver + load such as ``devlink-params`` in configuration mode + ``driverinit`` or ``devlink-resources`` + * - ``fw_activate`` + - Firmware activate. Activates new firmware if such image is stored and + pending activation. This action involves firmware reset, if no new image + pending this action will reload current firmware image. + * - ``fw_activate_no_reset`` + - Activate new firmware image without firmware reset or reload. + This action is also known as live patch, applying firmware changes + without reset. + +Note that when required to do firmware activation some drivers may need +to reload the driver. On the other hand some drivers may need to reset +the firmware to reinitialize the driver entities. Therefore, the devlink +reload command returns the actions which were actually done. +However, in case fw_activate_no_reset action is selected, then no other +reload action is allowed. + +Change namespace +================ + +All devlink instances are created in init_net and stay there for a +lifetime. Allow user to be able to move devlink instances into +namespaces during devlink reload operation. That ensures proper +re-instantiation of driver objects, including netdevices. + +example usage +------------- + +.. code:: shell + + $ devlink dev reload help + $ devlink dev reload DEV [ netns { PID | NAME | ID } ] [ action { driver_reinit | fw_activate [no_reset] } ] + + # Run reload command for devlink driver entities re-initialization: + $ devlink dev reload pci/0000:82:00.0 action driver_reinit + reload_actions_done: + driver_reinit + + # Run reload command to activate firmware: + $ devlink dev reload pci/0000:82:00.0 action fw_activate + reload_actions_done: + driver_reinit fw_activate + # Note that the driver in this example reloads the driver while activating firmware diff --git a/Documentation/networking/devlink/index.rst b/Documentation/networking/devlink/index.rst index 7684ae5c4a4a..d82874760ae2 100644 --- a/Documentation/networking/devlink/index.rst +++ b/Documentation/networking/devlink/index.rst @@ -20,6 +20,7 @@ general. devlink-params devlink-region devlink-resource + devlink-reload devlink-trap Driver-specific documentation