From patchwork Tue Mar 24 22:34:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Keller X-Patchwork-Id: 221925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 158A0C18E5B for ; Tue, 24 Mar 2020 22:35:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E7CB7208CA for ; Tue, 24 Mar 2020 22:35:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728575AbgCXWfB (ORCPT ); Tue, 24 Mar 2020 18:35:01 -0400 Received: from mga04.intel.com ([192.55.52.120]:54988 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728539AbgCXWfA (ORCPT ); Tue, 24 Mar 2020 18:35:00 -0400 IronPort-SDR: 3xjy8ACJvuHz3s8NXF9XvLEGYojeXAPgZd8axkwkNmmajXwy9RW9Fs/idXBFMMozKoEgrlcuDR CDQd65p4CSOg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 15:35:00 -0700 IronPort-SDR: UFl7cGoACukdOS4za7or5bIqVHHPugv/dkQfj8hCnWcTqLj/4/bwVC9d/mqA63lMWu/fPqXS/3 q1KUbBohmd8Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,301,1580803200"; d="scan'208";a="238363135" Received: from jekeller-desk.amr.corp.intel.com ([10.166.241.33]) by fmsmga007.fm.intel.com with ESMTP; 24 Mar 2020 15:35:00 -0700 From: Jacob Keller To: netdev@vger.kernel.org Cc: Jakub Kicinski , Jiri Pirko , Jacob Keller , Jiri Pirko Subject: [PATCH 02/10] devlink: convert snapshot destructor callback to region op Date: Tue, 24 Mar 2020 15:34:37 -0700 Message-Id: <20200324223445.2077900-3-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200324223445.2077900-1-jacob.e.keller@intel.com> References: <20200324223445.2077900-1-jacob.e.keller@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org It does not makes sense that two snapshots for a given region would use different destructors. Simplify snapshot creation by adding a .destructor op for regions. This operation will replace the data_destructor for the snapshot creation, and makes snapshot creation easier. Noticed-by: Jakub Kicinski Signed-off-by: Jacob Keller Reviewed-by: Jiri Pirko --- drivers/net/ethernet/mellanox/mlx4/crdump.c | 6 ++++-- drivers/net/netdevsim/dev.c | 3 ++- include/net/devlink.h | 7 +++---- net/core/devlink.c | 11 +++++------ 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c index cc2bf596c74b..c3f90c0f9554 100644 --- a/drivers/net/ethernet/mellanox/mlx4/crdump.c +++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c @@ -43,10 +43,12 @@ static const char * const region_fw_health_str = "fw-health"; static const struct devlink_region_ops region_cr_space_ops = { .name = region_cr_space_str, + .destructor = &kvfree, }; static const struct devlink_region_ops region_fw_health_ops = { .name = region_fw_health_str, + .destructor = &kvfree, }; /* Set to true in case cr enable bit was set to true before crdump */ @@ -107,7 +109,7 @@ static void mlx4_crdump_collect_crspace(struct mlx4_dev *dev, readl(cr_space + offset); err = devlink_region_snapshot_create(crdump->region_crspace, - crspace_data, id, &kvfree); + crspace_data, id); if (err) { kvfree(crspace_data); mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n", @@ -146,7 +148,7 @@ static void mlx4_crdump_collect_fw_health(struct mlx4_dev *dev, readl(health_buf_start + offset); err = devlink_region_snapshot_create(crdump->region_fw_health, - health_data, id, &kvfree); + health_data, id); if (err) { kvfree(health_data); mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n", diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index 47a8f8c570c4..f7621ccb7b88 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -56,7 +56,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file, id = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev)); err = devlink_region_snapshot_create(nsim_dev->dummy_region, - dummy_data, id, kfree); + dummy_data, id); if (err) { pr_err("Failed to create region snapshot\n"); kfree(dummy_data); @@ -342,6 +342,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink) static const struct devlink_region_ops dummy_region_ops = { .name = "dummy", + .destructor = &kfree, }; static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev, diff --git a/include/net/devlink.h b/include/net/devlink.h index 85db5dd5184d..8869ad75b965 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -496,14 +496,14 @@ enum devlink_param_generic_id { struct devlink_region; struct devlink_info_req; -typedef void devlink_snapshot_data_dest_t(const void *data); - /** * struct devlink_region_ops - Region operations * @name: region name + * @destructor: callback used to free snapshot memory when deleting */ struct devlink_region_ops { const char *name; + void (*destructor)(const void *data); }; struct devlink_fmsg; @@ -978,8 +978,7 @@ devlink_region_create(struct devlink *devlink, void devlink_region_destroy(struct devlink_region *region); u32 devlink_region_snapshot_id_get(struct devlink *devlink); int devlink_region_snapshot_create(struct devlink_region *region, - u8 *data, u32 snapshot_id, - devlink_snapshot_data_dest_t *data_destructor); + u8 *data, u32 snapshot_id); int devlink_info_serial_number_put(struct devlink_info_req *req, const char *sn); int devlink_info_driver_name_put(struct devlink_info_req *req, diff --git a/net/core/devlink.c b/net/core/devlink.c index ca5362530567..84d74fbcff62 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -354,7 +354,6 @@ struct devlink_region { struct devlink_snapshot { struct list_head list; struct devlink_region *region; - devlink_snapshot_data_dest_t *data_destructor; u8 *data; u32 id; }; @@ -3775,7 +3774,7 @@ static void devlink_region_snapshot_del(struct devlink_region *region, devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_DEL); region->cur_snapshots--; list_del(&snapshot->list); - (*snapshot->data_destructor)(snapshot->data); + region->ops->destructor(snapshot->data); kfree(snapshot); } @@ -7659,6 +7658,9 @@ devlink_region_create(struct devlink *devlink, struct devlink_region *region; int err = 0; + if (WARN_ON(!ops) || WARN_ON(!ops->destructor)) + return ERR_PTR(-EINVAL); + mutex_lock(&devlink->lock); if (devlink_region_get_by_name(devlink, ops->name)) { @@ -7745,11 +7747,9 @@ EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get); * @region: devlink region of the snapshot * @data: snapshot data * @snapshot_id: snapshot id to be created - * @data_destructor: pointer to destructor function to free data */ int devlink_region_snapshot_create(struct devlink_region *region, - u8 *data, u32 snapshot_id, - devlink_snapshot_data_dest_t *data_destructor) + u8 *data, u32 snapshot_id) { struct devlink *devlink = region->devlink; struct devlink_snapshot *snapshot; @@ -7777,7 +7777,6 @@ int devlink_region_snapshot_create(struct devlink_region *region, snapshot->id = snapshot_id; snapshot->region = region; snapshot->data = data; - snapshot->data_destructor = data_destructor; list_add_tail(&snapshot->list, ®ion->snapshot_list); From patchwork Tue Mar 24 22:34:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Keller X-Patchwork-Id: 221924 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 497B2C10DCE for ; Tue, 24 Mar 2020 22:35:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 264D420836 for ; Tue, 24 Mar 2020 22:35:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728712AbgCXWfK (ORCPT ); Tue, 24 Mar 2020 18:35:10 -0400 Received: from mga04.intel.com ([192.55.52.120]:54988 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728539AbgCXWfK (ORCPT ); Tue, 24 Mar 2020 18:35:10 -0400 IronPort-SDR: cBR9RuV2m3GnaEhgSM1iU0Q18OgmvnZn11j8MvWZdu8iu0YUip9TiIRLs2tIKMTZbgecyVhcNs sLiwM0UZR3xA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 15:35:09 -0700 IronPort-SDR: /3FvZa2kaCxQtECplDk24pYoriAQ8du99tjMgV79bS0PL4jJRChXG+pBoxQQtyBbuCCTqDdflx xgytXZm6Iwkw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,301,1580803200"; d="scan'208";a="238363179" Received: from jekeller-desk.amr.corp.intel.com ([10.166.241.33]) by fmsmga007.fm.intel.com with ESMTP; 24 Mar 2020 15:35:09 -0700 From: Jacob Keller To: netdev@vger.kernel.org Cc: Jakub Kicinski , Jiri Pirko , Jacob Keller , Jiri Pirko Subject: [PATCH 04/10] devlink: add function to take snapshot while locked Date: Tue, 24 Mar 2020 15:34:39 -0700 Message-Id: <20200324223445.2077900-5-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200324223445.2077900-1-jacob.e.keller@intel.com> References: <20200324223445.2077900-1-jacob.e.keller@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org A future change is going to add a new devlink command to request a snapshot on demand. This function will want to call the devlink_region_snapshot_create function while already holding the devlink instance lock. Extract the logic of this function into a static function prefixed by `__` to indicate that it is an internal helper function. Modify the original function to be implemented in terms of the new locked function. Signed-off-by: Jacob Keller Reviewed-by: Jiri Pirko --- net/core/devlink.c | 78 ++++++++++++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 31 deletions(-) diff --git a/net/core/devlink.c b/net/core/devlink.c index 73e66a779c13..620e9d07ac85 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -3768,6 +3768,52 @@ static void devlink_nl_region_notify(struct devlink_region *region, nlmsg_free(msg); } +/** + * __devlink_region_snapshot_create - create a new snapshot + * This will add a new snapshot of a region. The snapshot + * will be stored on the region struct and can be accessed + * from devlink. This is useful for future analyses of snapshots. + * Multiple snapshots can be created on a region. + * The @snapshot_id should be obtained using the getter function. + * + * Must be called only while holding the devlink instance lock. + * + * @region: devlink region of the snapshot + * @data: snapshot data + * @snapshot_id: snapshot id to be created + */ +static int +__devlink_region_snapshot_create(struct devlink_region *region, + u8 *data, u32 snapshot_id) +{ + struct devlink *devlink = region->devlink; + struct devlink_snapshot *snapshot; + + lockdep_assert_held(&devlink->lock); + + /* check if region can hold one more snapshot */ + if (region->cur_snapshots == region->max_snapshots) + return -ENOMEM; + + if (devlink_region_snapshot_get_by_id(region, snapshot_id)) + return -EEXIST; + + snapshot = kzalloc(sizeof(*snapshot), GFP_KERNEL); + if (!snapshot) + return -ENOMEM; + + snapshot->id = snapshot_id; + snapshot->region = region; + snapshot->data = data; + + list_add_tail(&snapshot->list, ®ion->snapshot_list); + + region->cur_snapshots++; + + devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_NEW); + return 0; +} + static void devlink_region_snapshot_del(struct devlink_region *region, struct devlink_snapshot *snapshot) { @@ -7752,42 +7798,12 @@ int devlink_region_snapshot_create(struct devlink_region *region, u8 *data, u32 snapshot_id) { struct devlink *devlink = region->devlink; - struct devlink_snapshot *snapshot; int err; mutex_lock(&devlink->lock); - - /* check if region can hold one more snapshot */ - if (region->cur_snapshots == region->max_snapshots) { - err = -ENOMEM; - goto unlock; - } - - if (devlink_region_snapshot_get_by_id(region, snapshot_id)) { - err = -EEXIST; - goto unlock; - } - - snapshot = kzalloc(sizeof(*snapshot), GFP_KERNEL); - if (!snapshot) { - err = -ENOMEM; - goto unlock; - } - - snapshot->id = snapshot_id; - snapshot->region = region; - snapshot->data = data; - - list_add_tail(&snapshot->list, ®ion->snapshot_list); - - region->cur_snapshots++; - - devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_NEW); + err = __devlink_region_snapshot_create(region, data, snapshot_id); mutex_unlock(&devlink->lock); - return 0; -unlock: - mutex_unlock(&devlink->lock); return err; } EXPORT_SYMBOL_GPL(devlink_region_snapshot_create); From patchwork Tue Mar 24 22:34:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Keller X-Patchwork-Id: 221923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25990C10DCE for ; Tue, 24 Mar 2020 22:35:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0449F20836 for ; Tue, 24 Mar 2020 22:35:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728728AbgCXWfR (ORCPT ); Tue, 24 Mar 2020 18:35:17 -0400 Received: from mga04.intel.com ([192.55.52.120]:54988 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728539AbgCXWfQ (ORCPT ); Tue, 24 Mar 2020 18:35:16 -0400 IronPort-SDR: KNepQk2j0/xTXx3L+g5uklZoCQ6LgA3hVEcWqIOh8lKkBOl6yAnqo3bHqqrkB1NGEc1anDq1Ep 7KTNttczoFPg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 15:35:16 -0700 IronPort-SDR: ijd9ZTV/a9iE8wc3c784okHKSRCIYCUlOn9/JI3XUGDoK/3zX8Kj4X9sUfP8CTrHB9q0MBHXe4 cbfxHd4XCh3w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,301,1580803200"; d="scan'208";a="238363195" Received: from jekeller-desk.amr.corp.intel.com ([10.166.241.33]) by fmsmga007.fm.intel.com with ESMTP; 24 Mar 2020 15:35:16 -0700 From: Jacob Keller To: netdev@vger.kernel.org Cc: Jakub Kicinski , Jiri Pirko , Jacob Keller , Jiri Pirko Subject: [PATCH 06/10] devlink: convert snapshot id getter to return an error Date: Tue, 24 Mar 2020 15:34:41 -0700 Message-Id: <20200324223445.2077900-7-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200324223445.2077900-1-jacob.e.keller@intel.com> References: <20200324223445.2077900-1-jacob.e.keller@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Modify the devlink_snapshot_id_get function to return a signed value, enabling reporting an error on failure. This enables easily refactoring how IDs are generated and kept track of in the future. For now, just report ENOSPC once INT_MAX snapshot ids have been returned. Signed-off-by: Jacob Keller Reviewed-by: Jiri Pirko --- drivers/net/ethernet/mellanox/mlx4/crdump.c | 10 +++++++--- drivers/net/netdevsim/dev.c | 7 +++++-- include/net/devlink.h | 2 +- net/core/devlink.c | 16 +++++++++++----- 4 files changed, 24 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c index c3f90c0f9554..723a66efdf32 100644 --- a/drivers/net/ethernet/mellanox/mlx4/crdump.c +++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c @@ -169,7 +169,7 @@ int mlx4_crdump_collect(struct mlx4_dev *dev) struct pci_dev *pdev = dev->persist->pdev; unsigned long cr_res_size; u8 __iomem *cr_space; - u32 id; + int id; if (!dev->caps.health_buffer_addrs) { mlx4_info(dev, "crdump: FW doesn't support health buffer access, skipping\n"); @@ -189,10 +189,14 @@ int mlx4_crdump_collect(struct mlx4_dev *dev) return -ENODEV; } - crdump_enable_crspace_access(dev, cr_space); - /* Get the available snapshot ID for the dumps */ id = devlink_region_snapshot_id_get(devlink); + if (id < 0) { + mlx4_err(dev, "crdump: devlink get snapshot id err %d\n", id); + return id; + } + + crdump_enable_crspace_access(dev, cr_space); /* Try to capture dumps */ mlx4_crdump_collect_crspace(dev, cr_space, id); diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index f7621ccb7b88..f9420b77e5fd 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -45,8 +45,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file, { struct nsim_dev *nsim_dev = file->private_data; void *dummy_data; - int err; - u32 id; + int err, id; dummy_data = kmalloc(NSIM_DEV_DUMMY_REGION_SIZE, GFP_KERNEL); if (!dummy_data) @@ -55,6 +54,10 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file, get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE); id = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev)); + if (id < 0) { + pr_err("Failed to get snapshot id\n"); + return id; + } err = devlink_region_snapshot_create(nsim_dev->dummy_region, dummy_data, id); if (err) { diff --git a/include/net/devlink.h b/include/net/devlink.h index 8869ad75b965..df9f6ddf6c66 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -976,7 +976,7 @@ devlink_region_create(struct devlink *devlink, const struct devlink_region_ops *ops, u32 region_max_snapshots, u64 region_size); void devlink_region_destroy(struct devlink_region *region); -u32 devlink_region_snapshot_id_get(struct devlink *devlink); +int devlink_region_snapshot_id_get(struct devlink *devlink); int devlink_region_snapshot_create(struct devlink_region *region, u8 *data, u32 snapshot_id); int devlink_info_serial_number_put(struct devlink_info_req *req, diff --git a/net/core/devlink.c b/net/core/devlink.c index 6dc14eb2a5f7..62a8566e9851 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -3772,12 +3772,16 @@ static void devlink_nl_region_notify(struct devlink_region *region, * __devlink_region_snapshot_id_get - get snapshot ID * @devlink: devlink instance * - * Returns a new snapshot id. Must be called while holding the - * devlink instance lock. + * Returns a new snapshot id or a negative error code on failure. Must be + * called while holding the devlink instance lock. */ -static u32 __devlink_region_snapshot_id_get(struct devlink *devlink) +static int __devlink_region_snapshot_id_get(struct devlink *devlink) { lockdep_assert_held(&devlink->lock); + + if (devlink->snapshot_id >= INT_MAX) + return -ENOSPC; + return ++devlink->snapshot_id; } @@ -7781,11 +7785,13 @@ EXPORT_SYMBOL_GPL(devlink_region_destroy); * Driver should use the same id for multiple snapshots taken * on multiple regions at the same time/by the same trigger. * + * Returns a positive id or a negative error code on failure. + * * @devlink: devlink */ -u32 devlink_region_snapshot_id_get(struct devlink *devlink) +int devlink_region_snapshot_id_get(struct devlink *devlink) { - u32 id; + int id; mutex_lock(&devlink->lock); id = __devlink_region_snapshot_id_get(devlink); From patchwork Tue Mar 24 22:34:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Keller X-Patchwork-Id: 221922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F55C54FCE for ; Tue, 24 Mar 2020 22:35:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E0C572078A for ; Tue, 24 Mar 2020 22:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728755AbgCXWfW (ORCPT ); Tue, 24 Mar 2020 18:35:22 -0400 Received: from mga04.intel.com ([192.55.52.120]:54988 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728539AbgCXWfV (ORCPT ); Tue, 24 Mar 2020 18:35:21 -0400 IronPort-SDR: aQZwco4AtgYkAaRf+eG8D/16F3jYgL+DYN4X7nb6VJsG3OF2hXGckVcsLzNsKAk3g4lhZvRc6w Tz8qvPukESbQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 15:35:21 -0700 IronPort-SDR: 9Iz2pA9FutsP62xtT1hBhlxVKvcQBHwbvIM5HEbdnYYy87SagKtqIvFErZMl8O54ArsmOyzlDv pszXh8hiWPtQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,301,1580803200"; d="scan'208";a="238363208" Received: from jekeller-desk.amr.corp.intel.com ([10.166.241.33]) by fmsmga007.fm.intel.com with ESMTP; 24 Mar 2020 15:35:20 -0700 From: Jacob Keller To: netdev@vger.kernel.org Cc: Jakub Kicinski , Jiri Pirko , Jacob Keller Subject: [PATCH 08/10] devlink: implement DEVLINK_CMD_REGION_NEW Date: Tue, 24 Mar 2020 15:34:43 -0700 Message-Id: <20200324223445.2077900-9-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200324223445.2077900-1-jacob.e.keller@intel.com> References: <20200324223445.2077900-1-jacob.e.keller@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Implement support for the DEVLINK_CMD_REGION_NEW command for creating snapshots. This new command parallels the existing DEVLINK_CMD_REGION_DEL. In order for DEVLINK_CMD_REGION_NEW to work for a region, the new ".snapshot" operation must be implemented in the region's ops structure. The desired snapshot id must be provided. This helps avoid confusion on the purpose of DEVLINK_CMD_REGION_NEW, and keeps the API simpler. The requested id will be inserted into the xarray tracking the number of snapshots using each id. If this id is already used by another snapshot on any region, an error will be returned. Signed-off-by: Jacob Keller --- .../networking/devlink/devlink-region.rst | 8 ++ include/net/devlink.h | 6 + net/core/devlink.c | 103 ++++++++++++++++++ 3 files changed, 117 insertions(+) diff --git a/Documentation/networking/devlink/devlink-region.rst b/Documentation/networking/devlink/devlink-region.rst index 8b46e8591fe0..9d2d4c95a5c4 100644 --- a/Documentation/networking/devlink/devlink-region.rst +++ b/Documentation/networking/devlink/devlink-region.rst @@ -20,6 +20,11 @@ address regions that are otherwise inaccessible to the user. Regions may also be used to provide an additional way to debug complex error states, but see also :doc:`devlink-health` +Regions may optionally support capturing a snapshot on demand via the +``DEVLINK_CMD_REGION_NEW`` netlink message. A driver wishing to allow +requested snapshots must implement the ``.snapshot`` callback for the region +in its ``devlink_region_ops`` structure. + example usage ------------- @@ -40,6 +45,9 @@ example usage # Delete a snapshot using: $ devlink region del pci/0000:00:05.0/cr-space snapshot 1 + # Request an immediate snapshot, if supported by the region + $ devlink region new pci/0000:00:05.0/cr-space snapshot 5 + # Dump a snapshot: $ devlink region dump pci/0000:00:05.0/fw-health snapshot 1 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 diff --git a/include/net/devlink.h b/include/net/devlink.h index 30306e62fe73..ae894e073050 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -502,10 +502,16 @@ struct devlink_info_req; * struct devlink_region_ops - Region operations * @name: region name * @destructor: callback used to free snapshot memory when deleting + * @snapshot: callback to request an immediate snapshot. On success, + * the data variable must be updated to point to the snapshot data. + * The function will be called while the devlink instance lock is + * held. */ struct devlink_region_ops { const char *name; void (*destructor)(const void *data); + int (*snapshot)(struct devlink *devlink, struct netlink_ext_ack *extack, + u8 **data); }; struct devlink_fmsg; diff --git a/net/core/devlink.c b/net/core/devlink.c index b3698228a6ed..16129ec27913 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -3849,6 +3849,35 @@ static void __devlink_snapshot_id_decrement(struct devlink *devlink, u32 id) } } +/** + * __devlink_snapshot_id_insert - Insert a specific snapshot ID + * @devlink: devlink instance + * @id: the snapshot id + * + * Mark the given snapshot id as used by inserting a zero value into the + * snapshot xarray. + * + * Returns zero on success, or an error code if the snapshot id could not + * be inserted. + */ +static int __devlink_snapshot_id_insert(struct devlink *devlink, u32 id) +{ + int err; + + lockdep_assert_held(&devlink->lock); + + /* Check to make sure it's empty first */ + if (xa_load(&devlink->snapshot_ids, id)) + return -EBUSY; + + err = xa_err(xa_store(&devlink->snapshot_ids, id, xa_mk_value(0), + GFP_KERNEL)); + if (err) + return err; + + return 0; +} + /** * __devlink_region_snapshot_id_get - get snapshot ID * @devlink: devlink instance @@ -4048,6 +4077,73 @@ static int devlink_nl_cmd_region_del(struct sk_buff *skb, return 0; } +static int +devlink_nl_cmd_region_new(struct sk_buff *skb, struct genl_info *info) +{ + struct devlink *devlink = info->user_ptr[0]; + struct devlink_region *region; + const char *region_name; + u32 snapshot_id; + u8 *data; + int err; + + if (!info->attrs[DEVLINK_ATTR_REGION_NAME]) { + NL_SET_ERR_MSG_MOD(info->extack, "No region name provided"); + return -EINVAL; + } + + if (!info->attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) { + NL_SET_ERR_MSG_MOD(info->extack, "No snapshot id provided"); + return -EINVAL; + } + + region_name = nla_data(info->attrs[DEVLINK_ATTR_REGION_NAME]); + region = devlink_region_get_by_name(devlink, region_name); + if (!region) { + NL_SET_ERR_MSG_MOD(info->extack, "The requested region does not exist"); + return -EINVAL; + } + + if (!region->ops->snapshot) { + NL_SET_ERR_MSG_MOD(info->extack, "The requested region does not support taking an immediate snapshot"); + return -EOPNOTSUPP; + } + + if (region->cur_snapshots == region->max_snapshots) { + NL_SET_ERR_MSG_MOD(info->extack, "The region has reached the maximum number of stored snapshots"); + return -ENOMEM; + } + + snapshot_id = nla_get_u32(info->attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]); + + if (devlink_region_snapshot_get_by_id(region, snapshot_id)) { + NL_SET_ERR_MSG_MOD(info->extack, "The requested snapshot id is already in use"); + return -EEXIST; + } + + err = __devlink_snapshot_id_insert(devlink, snapshot_id); + if (err) { + NL_SET_ERR_MSG_MOD(info->extack, "The requested snapshot id is already in used"); + return err; + } + + err = region->ops->snapshot(devlink, info->extack, &data); + if (err) + goto err_decrement_snapshot_count; + + err = __devlink_region_snapshot_create(region, data, snapshot_id); + if (err) + goto err_free_snapshot_data; + + return 0; + +err_decrement_snapshot_count: + __devlink_snapshot_id_decrement(devlink, snapshot_id); +err_free_snapshot_data: + region->ops->destructor(data); + return err; +} + static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg, struct devlink *devlink, u8 *chunk, u32 chunk_size, @@ -6455,6 +6551,13 @@ static const struct genl_ops devlink_nl_ops[] = { .flags = GENL_ADMIN_PERM, .internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK, }, + { + .cmd = DEVLINK_CMD_REGION_NEW, + .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, + .doit = devlink_nl_cmd_region_new, + .flags = GENL_ADMIN_PERM, + .internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK, + }, { .cmd = DEVLINK_CMD_REGION_DEL, .validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP, From patchwork Tue Mar 24 22:34:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacob Keller X-Patchwork-Id: 221921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA950C10DCE for ; Tue, 24 Mar 2020 22:35:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 804702078A for ; Tue, 24 Mar 2020 22:35:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728774AbgCXWf1 (ORCPT ); Tue, 24 Mar 2020 18:35:27 -0400 Received: from mga04.intel.com ([192.55.52.120]:54988 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728445AbgCXWf0 (ORCPT ); Tue, 24 Mar 2020 18:35:26 -0400 IronPort-SDR: V4qb0h92VzG8m2ousDlmdvQ0zUfyRiaDYdQxjorQnZkSRBdwA37a9J+PmbDvzk0DM1xucRqV7J sBgZ8LM0QXEQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 15:35:25 -0700 IronPort-SDR: vjZUuchommjLouPJRXNnA8rYoVL5Itd8V26EJFnuLqye4mnWA+OwTmz3ebjhxC1Ycq9bJWWE7i pMFGLBbPti/w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,301,1580803200"; d="scan'208";a="238363222" Received: from jekeller-desk.amr.corp.intel.com ([10.166.241.33]) by fmsmga007.fm.intel.com with ESMTP; 24 Mar 2020 15:35:25 -0700 From: Jacob Keller To: netdev@vger.kernel.org Cc: Jakub Kicinski , Jiri Pirko , Jacob Keller Subject: [PATCH 10/10] ice: add a devlink region for dumping NVM contents Date: Tue, 24 Mar 2020 15:34:45 -0700 Message-Id: <20200324223445.2077900-11-jacob.e.keller@intel.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200324223445.2077900-1-jacob.e.keller@intel.com> References: <20200324223445.2077900-1-jacob.e.keller@intel.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a devlink region for exposing the device's Non Volatime Memory flash contents. Support the recently added .snapshot operation, enabling userspace to request a snapshot of the NVM contents via DEVLINK_CMD_REGION_NEW. Signed-off-by: Jacob Keller --- Documentation/networking/devlink/ice.rst | 26 +++++ drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 99 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + 5 files changed, 134 insertions(+) diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst index 37fbbd40a5e5..f3d6a3b50342 100644 --- a/Documentation/networking/devlink/ice.rst +++ b/Documentation/networking/devlink/ice.rst @@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions - The version of the DDP package that is active in the device. Note that both the name (as reported by ``fw.app.name``) and version are required to uniquely identify the package. + +Regions +======= + +The ``ice`` driver enables access to the contents of the Non Volatile Memory +flash chip via the ``nvm-flash`` region. + +Users can request an immediate capture of a snapshot via the +``DEVLINK_CMD_REGION_NEW`` + +.. code:: shell + + $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1 + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8 + 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc + 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5 + + $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 + length 16 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + + $ devlink region delete pci/0000:01:00.0/nvm-flash snapshot 1 diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 8ce3afcfeca0..5c11448bfbb3 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -351,6 +351,8 @@ struct ice_pf { /* devlink port data */ struct devlink_port devlink_port; + struct devlink_region *nvm_region; + /* OS reserved IRQ details */ struct msix_entry *msix_entries; struct ice_res_tracker *irq_tracker; diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c index 27c5034c039a..9dd1b21820bf 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.c +++ b/drivers/net/ethernet/intel/ice/ice_devlink.c @@ -318,3 +318,102 @@ void ice_devlink_destroy_port(struct ice_pf *pf) devlink_port_type_clear(&pf->devlink_port); devlink_port_unregister(&pf->devlink_port); } + +/** + * ice_devlink_nvm_snapshot - Capture a snapshot of the Shadow RAM contents + * @devlink: the devlink instance + * @extack: extended ACK response structure + * @data: on exit points to snapshot data buffer + * + * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for + * the shadow-ram devlink region. It captures a snapshot of the shadow ram + * contents. This snapshot can later be viewed via the devlink-region + * interface. + * + * @returns zero on success, and updates the data pointer. Returns a non-zero + * error code on failure. + */ +static int +ice_devlink_nvm_snapshot(struct devlink *devlink, struct netlink_ext_ack *extack, + u8 **data) +{ + struct ice_pf *pf = devlink_priv(devlink); + struct device *dev = ice_pf_to_dev(pf); + struct ice_hw *hw = &pf->hw; + enum ice_status status; + void *nvm_data; + u32 nvm_size; + + nvm_size = hw->nvm.flash_size; + nvm_data = vzalloc(nvm_size); + if (!nvm_data) { + NL_SET_ERR_MSG_MOD(extack, "Out of memory"); + return -ENOMEM; + } + + status = ice_acquire_nvm(hw, ICE_RES_READ); + if (status) { + dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n", + status, hw->adminq.sq_last_status); + NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore"); + vfree(nvm_data); + return -EIO; + } + + status = ice_read_flat_nvm(hw, 0, &nvm_size, nvm_data, false); + if (status) { + dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n", + nvm_size, status, hw->adminq.sq_last_status); + NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents"); + ice_release_nvm(hw); + vfree(nvm_data); + return -EIO; + } + + ice_release_nvm(hw); + + *data = nvm_data; + + return 0; +} + +static const struct devlink_region_ops ice_nvm_region_ops = { + .name = "nvm-flash", + .destructor = vfree, + .snapshot = ice_devlink_nvm_snapshot, +}; + +/** + * ice_devlink_init_regions - Initialize devlink regions + * @pf: the PF device structure + * + * Create devlink regions used to enable access to dump the contents of the + * flash memory on the device. + */ +void ice_devlink_init_regions(struct ice_pf *pf) +{ + struct devlink *devlink = priv_to_devlink(pf); + struct device *dev = ice_pf_to_dev(pf); + u64 nvm_size; + + nvm_size = pf->hw.nvm.flash_size; + pf->nvm_region = devlink_region_create(devlink, &ice_nvm_region_ops, 1, + nvm_size); + if (IS_ERR(pf->nvm_region)) { + dev_warn(dev, "failed to create NVM devlink region, err %ld\n", + PTR_ERR(pf->nvm_region)); + pf->nvm_region = NULL; + } +} + +/** + * ice_devlink_destroy_regions - Destroy devlink regions + * @pf: the PF device structure + * + * Remove previously created regions for this PF. + */ +void ice_devlink_destroy_regions(struct ice_pf *pf) +{ + if (pf->nvm_region) + devlink_region_destroy(pf->nvm_region); +} diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.h b/drivers/net/ethernet/intel/ice/ice_devlink.h index f94dc93c24c5..6e806a08dc23 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.h +++ b/drivers/net/ethernet/intel/ice/ice_devlink.h @@ -11,4 +11,7 @@ void ice_devlink_unregister(struct ice_pf *pf); int ice_devlink_create_port(struct ice_pf *pf); void ice_devlink_destroy_port(struct ice_pf *pf); +void ice_devlink_init_regions(struct ice_pf *pf); +void ice_devlink_destroy_regions(struct ice_pf *pf); + #endif /* _ICE_DEVLINK_H_ */ diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 359ff8544773..306a4e5b2320 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -3276,6 +3276,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) goto err_init_pf_unroll; } + ice_devlink_init_regions(pf); + pf->num_alloc_vsi = hw->func_caps.guar_num_vsi; if (!pf->num_alloc_vsi) { err = -EIO; @@ -3385,6 +3387,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) devm_kfree(dev, pf->vsi); err_init_pf_unroll: ice_deinit_pf(pf); + ice_devlink_destroy_regions(pf); ice_deinit_hw(hw); err_exit_unroll: ice_devlink_unregister(pf); @@ -3427,6 +3430,7 @@ static void ice_remove(struct pci_dev *pdev) ice_vsi_free_q_vectors(pf->vsi[i]); } ice_deinit_pf(pf); + ice_devlink_destroy_regions(pf); ice_deinit_hw(&pf->hw); ice_devlink_unregister(pf);