From patchwork Mon Jul 19 21:26:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AD78C6377A for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DDBD36113A for ; Mon, 19 Jul 2021 22:00:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359120AbhGSVQO (ORCPT ); Mon, 19 Jul 2021 17:16:14 -0400 Received: from mail-sn1anam02on2051.outbound.protection.outlook.com ([40.107.96.51]:60260 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388227AbhGSUqj (ORCPT ); Mon, 19 Jul 2021 16:46:39 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eWZjTkHJmDbl/27RxhTyHfGFHhzLLV0/5lSPn/6zK2B9fiI6af49ccd2mGZRYv4S9mn/J5cvWPRPTil+CIEOPwkykuPhcpR5c0czfWRbJgMrZsxghlqx2wbe1D93SsP5fibPlRLoPM0wczxkGp0w0O/iWlOcm3toeEn0Ncl1l9oZgF0j3UfZE+zHNE4bdo2n+G2NVeQ+e0geQH6RR9LlIeyn9j9Szxf/RvJyZjMTKmAdkUcfeTct+/Jd3ivpBF3C4Vp+UWzV9jtO3v8IHQDNlGm1tSO4tpXjc5H6rV7rGhFALzEO/HYrfV3axNUOEGGHjMrzriWBPbg0SLbEUvsJMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8aculnYNcbYtJ/YX+CXlSMntVEwyU0cas/MeOV7Fz+U=; b=V835xbVIsfq9Has0QMqp2utMZha1F5Ns7MOeM7+yfCfEODNlOJxd2jmTaUMHHQHVaVPnQaI+Yc+BtMwrCwzp808NZnkOAUZWVntF/S6avIEW6Y8iXTmcowGirlK25cnsHncOGYaQDHt37z5J9BdERpz5TYjzn3W4fJMFlxoEgLO1d+YJ+4ZCGb5dY45jLstFMrsZuv0FdqLSUP8egYz2rB66GuRjM2gKrRMBjp3it0eALGaDQF6SiVP6uiOJ4RNFMNoq5EcS9VrYVpvmb2d3gmZAX3isByLpjTEMXn6cJ+qVLQf8Ro2TY60GCE2rP8GVXsXD+sEywJU6yNZXSsQL7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8aculnYNcbYtJ/YX+CXlSMntVEwyU0cas/MeOV7Fz+U=; b=ion7eoJJayEPqqUjcnWOuTKL5GyI/Cs3dDaZeUREowbDmNAO+Ldj1sJtejcx2++YGSlhkS4AyTDhbPUVS4CHmy3NvccNzt/6B7xll/aFhCQ1/I4WQEhXNw6kyPqvAf0afjuGwlRpGJyc2gUxQBxDLefZyd3uky/aKNH7kZ8Apkg= Received: from DM6PR12CA0034.namprd12.prod.outlook.com (2603:10b6:5:1c0::47) by CH0PR02MB8108.namprd02.prod.outlook.com (2603:10b6:610:10d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 21:27:16 +0000 Received: from DM3NAM02FT012.eop-nam02.prod.protection.outlook.com (2603:10b6:5:1c0:cafe::ac) by DM6PR12CA0034.outlook.office365.com (2603:10b6:5:1c0::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Mon, 19 Jul 2021 21:27:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT012.mail.protection.outlook.com (10.13.5.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:16 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:27:08 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:27:08 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38270 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5an6-000F7s-1U; Mon, 19 Jul 2021 14:27:08 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 740136020BA; Mon, 19 Jul 2021 14:26:31 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 03/14] fpga: xrt: xclbin file helper functions Date: Mon, 19 Jul 2021 14:26:17 -0700 Message-ID: <20210719212628.134129-4-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8f9ab168-7a2f-44d0-8c80-08d94afbfba5 X-MS-TrafficTypeDiagnostic: CH0PR02MB8108: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1824; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: P7R1P+hH2z3UhSWNiLIVXoSLp3kvEamxcm8L7sjHUZY8s+NYwc/QBS19WNb3YOM4R9A2F/U1r+aOPn8EAYXooEKlAYkkMCOrf3AW+jhuOTUH0EN4yxEnBZnkMVNss1NqYDgBz0RbAoe4fsRL3Hr1WTcCLnTU3KH/IVyXqkaBxIRVP5hAmJegUGsiM6kDKSv54tT65ydb9UZuXIhF8kQuUV8R3jLSI+RJD/ktrhjVVtCidrkx4rtt4weUDxHjVvC03sMNTS/eeEqyVS4T7nglvUPQ3v0XBd3MVQLOMW7GRsdef5rq+GJAg8z4gXdTDh00MWpf4ifplO5ezc7Lj2Tct7tpit4B+WElrmNLrhcfkG1ApMQ4KNME4gX2V8he3iiDTaXXhkqwtnXqGPfRweFZbUU/QO34Ul8Qyqt8nbtSIHKD8gGF5UZ7wYZD7aPcX3ay+VJm0nVZHW0O+3DG158sIcaZTq4WcIYAGlzKzfv/SaVAKn8GGNYTjhmdgKNzUp4mX5PjVSkmKbJFz0nsBBq/anNc9RnZyWMk3PVvabK9RN3O1zdzAepKy1zU//mm5VJEmVw7ZyjcrAw9U147m4/6PXw9oByKBqrijnMcKUUHXtdrWlx/91uxoseOYf/NroqhtIeyKrW21GHNAIkqn/TGH00Y/lVUhPXCJqQt7L6dKRLPXujePQ1I/i78iX+GBmYR2q4WlIrolwSkzY1yFO/A3g== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(136003)(376002)(396003)(36840700001)(46966006)(26005)(107886003)(316002)(6916009)(70206006)(70586007)(6666004)(8936002)(5660300002)(1076003)(6266002)(4326008)(83380400001)(36906005)(36860700001)(186003)(2906002)(82310400003)(42186006)(82740400003)(30864003)(54906003)(426003)(336012)(478600001)(8676002)(2616005)(44832011)(7636003)(356005)(47076005)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:27:16.7893 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8f9ab168-7a2f-44d0-8c80-08d94afbfba5 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT012.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR02MB8108 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Alveo FPGA firmware and partial reconfigure file are in xclbin format. This code enumerates and extracts sections from xclbin files. xclbin.h is cross platform and used across all platforms and OS. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/include/xclbin-helper.h | 48 +++ drivers/fpga/xrt/lib/xclbin.c | 381 ++++++++++++++++++++ include/uapi/linux/fpga-xrt.h | 428 +++++++++++++++++++++++ 3 files changed, 857 insertions(+) create mode 100644 drivers/fpga/xrt/include/xclbin-helper.h create mode 100644 drivers/fpga/xrt/lib/xclbin.c create mode 100644 include/uapi/linux/fpga-xrt.h diff --git a/drivers/fpga/xrt/include/xclbin-helper.h b/drivers/fpga/xrt/include/xclbin-helper.h new file mode 100644 index 000000000000..518d55f74c25 --- /dev/null +++ b/drivers/fpga/xrt/include/xclbin-helper.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * David Zhang + * Sonal Santan + */ + +#ifndef _XCLBIN_HELPER_H_ +#define _XCLBIN_HELPER_H_ + +#include +#include +#include + +#define XCLBIN_VERSION2 "xclbin2" +#define XCLBIN_HWICAP_BITFILE_BUF_SZ 1024 +#define XCLBIN_MAX_SZ_1G (1024 * 1024 * 1024) /* Assuming xclbin <= 1G, always */ + +enum axlf_section_kind; +struct axlf; + +/** + * Bitstream header information as defined by Xilinx tools. + * Please note that this struct definition is not owned by the driver. + */ +struct xclbin_bit_head_info { + u32 header_length; /* Length of header in 32 bit words */ + u32 bitstream_length; /* Length of bitstream to read in bytes */ + const unchar *design_name; /* Design name get from bitstream */ + const unchar *part_name; /* Part name read from bitstream */ + const unchar *date; /* Date read from bitstream header */ + const unchar *time; /* Bitstream creation time */ + u32 magic_length; /* Length of the magic numbers */ + const unchar *version; /* Version string */ +}; + +/* caller must free the allocated memory for **data. len could be NULL. */ +int xrt_xclbin_get_section(struct device *dev, const struct axlf *xclbin, + enum axlf_section_kind kind, void **data, + uint64_t *len); +int xrt_xclbin_get_metadata(struct device *dev, const struct axlf *xclbin, char **dtb); +int xrt_xclbin_parse_bitstream_header(struct device *dev, const unchar *data, + u32 size, struct xclbin_bit_head_info *head_info); +const char *xrt_clock_type2epname(enum XCLBIN_CLOCK_TYPE type); + +#endif /* _XCLBIN_HELPER_H_ */ diff --git a/drivers/fpga/xrt/lib/xclbin.c b/drivers/fpga/xrt/lib/xclbin.c new file mode 100644 index 000000000000..6edac3d418be --- /dev/null +++ b/drivers/fpga/xrt/lib/xclbin.c @@ -0,0 +1,381 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx Alveo FPGA Driver XCLBIN parser + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: David Zhang + */ + +#include +#include +#include +#include "xclbin-helper.h" +#include "metadata.h" + +/* Used for parsing bitstream header */ +#define BITSTREAM_EVEN_MAGIC_BYTE 0x0f +#define BITSTREAM_ODD_MAGIC_BYTE 0xf0 + +static inline u16 bitstream_read16(const char *data, u32 *offset) +{ + u16 val; + + val = be16_to_cpu(*(__be16 *)(data + *offset)); + *offset += sizeof(__be16); + + return val; +} + +static inline u32 bitstream_read32(const char *data, u32 *offset) +{ + u32 val; + + val = be32_to_cpu(*(__be32 *)(data + *offset)); + *offset += sizeof(__be32); + + return val; +} + +static int xrt_xclbin_get_section_hdr(const struct axlf *xclbin, + enum axlf_section_kind kind, + const struct axlf_section_header **header) +{ + const struct axlf_section_header *phead = NULL; + u64 xclbin_len; + int i; + + *header = NULL; + for (i = 0; i < xclbin->header.num_sections; i++) { + if (xclbin->sections[i].section_kind == kind) { + phead = &xclbin->sections[i]; + break; + } + } + + if (!phead) + return -ENOENT; + + xclbin_len = xclbin->header.length; + if (xclbin_len > XCLBIN_MAX_SZ_1G || !phead->section_size || + phead->section_offset + phead->section_size > xclbin_len) + return -EINVAL; + + *header = phead; + return 0; +} + +static int xrt_xclbin_section_info(const struct axlf *xclbin, + enum axlf_section_kind kind, + u64 *offset, u64 *size) +{ + const struct axlf_section_header *mem_header = NULL; + int rc; + + rc = xrt_xclbin_get_section_hdr(xclbin, kind, &mem_header); + if (rc) + return rc; + + *offset = mem_header->section_offset; + *size = mem_header->section_size; + + return 0; +} + +/* caller must free the allocated memory for **data */ +int xrt_xclbin_get_section(struct device *dev, + const struct axlf *buf, + enum axlf_section_kind kind, + void **data, u64 *len) +{ + const struct axlf *xclbin = (const struct axlf *)buf; + void *section = NULL; + u64 offset; + u64 size; + int err; + + if (!data) { + dev_err(dev, "invalid data pointer"); + return -EINVAL; + } + + err = xrt_xclbin_section_info(xclbin, kind, &offset, &size); + if (err) { + dev_dbg(dev, "parsing section failed. kind %d, err = %d", kind, err); + return err; + } + + section = vzalloc(size); + if (!section) + return -ENOMEM; + + memcpy(section, ((const char *)xclbin) + offset, size); + + *data = section; + if (len) + *len = size; + + return 0; +} +EXPORT_SYMBOL_GPL(xrt_xclbin_get_section); + +static inline int xclbin_bit_get_string(const unchar *data, u32 size, + u32 offset, unchar prefix, + const unchar **str) +{ + int len; + u32 tmp; + + /* prefix and length will be 3 bytes */ + if (offset + 3 > size) + return -EINVAL; + + /* Read prefix */ + tmp = data[offset++]; + if (tmp != prefix) + return -EINVAL; + + /* Get string length */ + len = bitstream_read16(data, &offset); + if (offset + len > size) + return -EINVAL; + + if (data[offset + len - 1] != '\0') + return -EINVAL; + + *str = data + offset; + + return len + 3; +} + +/* parse bitstream header */ +int xrt_xclbin_parse_bitstream_header(struct device *dev, const unchar *data, + u32 size, struct xclbin_bit_head_info *head_info) +{ + u32 offset = 0; + int len, i; + u16 magic; + + memset(head_info, 0, sizeof(*head_info)); + + /* Get "Magic" length */ + if (size < sizeof(u16)) { + dev_err(dev, "invalid size"); + return -EINVAL; + } + + len = bitstream_read16(data, &offset); + if (offset + len > size) { + dev_err(dev, "invalid magic len"); + return -EINVAL; + } + head_info->magic_length = len; + + for (i = 0; i < head_info->magic_length - 1; i++) { + magic = data[offset++]; + if (!(i % 2) && magic != BITSTREAM_EVEN_MAGIC_BYTE) { + dev_err(dev, "invalid magic even byte at %d", offset); + return -EINVAL; + } + + if ((i % 2) && magic != BITSTREAM_ODD_MAGIC_BYTE) { + dev_err(dev, "invalid magic odd byte at %d", offset); + return -EINVAL; + } + } + + if (offset + 3 > size) { + dev_err(dev, "invalid length of magic end"); + return -EINVAL; + } + /* Read null end of magic data. */ + if (data[offset++]) { + dev_err(dev, "invalid magic end"); + return -EINVAL; + } + + /* Read 0x01 (short) */ + magic = bitstream_read16(data, &offset); + + /* Check the "0x01" half word */ + if (magic != 0x01) { + dev_err(dev, "invalid magic end"); + return -EINVAL; + } + + len = xclbin_bit_get_string(data, size, offset, 'a', &head_info->design_name); + if (len < 0) { + dev_err(dev, "get design name failed"); + return -EINVAL; + } + + head_info->version = strstr(head_info->design_name, "Version=") + strlen("Version="); + offset += len; + + len = xclbin_bit_get_string(data, size, offset, 'b', &head_info->part_name); + if (len < 0) { + dev_err(dev, "get part name failed"); + return -EINVAL; + } + offset += len; + + len = xclbin_bit_get_string(data, size, offset, 'c', &head_info->date); + if (len < 0) { + dev_err(dev, "get data failed"); + return -EINVAL; + } + offset += len; + + len = xclbin_bit_get_string(data, size, offset, 'd', &head_info->time); + if (len < 0) { + dev_err(dev, "get time failed"); + return -EINVAL; + } + offset += len; + + if (offset + 5 >= size) { + dev_err(dev, "can not get bitstream length"); + return -EINVAL; + } + + /* Read 'e' */ + if (data[offset++] != 'e') { + dev_err(dev, "invalid prefix of bitstream length"); + return -EINVAL; + } + + /* Get byte length of bitstream */ + head_info->bitstream_length = bitstream_read32(data, &offset); + + head_info->header_length = offset; + + return 0; +} +EXPORT_SYMBOL_GPL(xrt_xclbin_parse_bitstream_header); + +static struct xrt_clock_desc { + char *clock_ep_name; + u32 clock_xclbin_type; + char *clkfreq_ep_name; +} clock_desc[] = { + { + .clock_ep_name = XRT_MD_NODE_CLK_KERNEL1, + .clock_xclbin_type = CT_DATA, + .clkfreq_ep_name = XRT_MD_NODE_CLKFREQ_K1, + }, + { + .clock_ep_name = XRT_MD_NODE_CLK_KERNEL2, + .clock_xclbin_type = CT_KERNEL, + .clkfreq_ep_name = XRT_MD_NODE_CLKFREQ_K2, + }, + { + .clock_ep_name = XRT_MD_NODE_CLK_KERNEL3, + .clock_xclbin_type = CT_SYSTEM, + .clkfreq_ep_name = XRT_MD_NODE_CLKFREQ_HBM, + }, +}; + +const char *xrt_clock_type2epname(enum XCLBIN_CLOCK_TYPE type) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(clock_desc); i++) { + if (clock_desc[i].clock_xclbin_type == type) + return clock_desc[i].clock_ep_name; + } + return NULL; +} +EXPORT_SYMBOL_GPL(xrt_clock_type2epname); + +static const char *clock_type2clkfreq_name(enum XCLBIN_CLOCK_TYPE type) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(clock_desc); i++) { + if (clock_desc[i].clock_xclbin_type == type) + return clock_desc[i].clkfreq_ep_name; + } + return NULL; +} + +static int xrt_xclbin_add_clock_metadata(struct device *dev, + const struct axlf *xclbin, + char *dtb) +{ + struct clock_freq_topology *clock_topo; + u16 freq; + int rc; + int i; + + /* if clock section does not exist, add nothing and return success */ + rc = xrt_xclbin_get_section(dev, xclbin, CLOCK_FREQ_TOPOLOGY, + (void **)&clock_topo, NULL); + if (rc == -ENOENT) + return 0; + else if (rc) + return rc; + + for (i = 0; i < clock_topo->count; i++) { + u8 type = clock_topo->clock_freq[i].type; + const char *ep_name = xrt_clock_type2epname(type); + const char *counter_name = clock_type2clkfreq_name(type); + + if (!ep_name || !counter_name) + continue; + + freq = be16_to_cpu((__force __be16)clock_topo->clock_freq[i].freq_MHZ); + rc = xrt_md_set_prop(dev, dtb, ep_name, NULL, XRT_MD_PROP_CLK_FREQ, + &freq, sizeof(freq)); + if (rc) + break; + + rc = xrt_md_set_prop(dev, dtb, ep_name, NULL, XRT_MD_PROP_CLK_CNT, + counter_name, strlen(counter_name) + 1); + if (rc) + break; + } + + vfree(clock_topo); + + return rc; +} + +int xrt_xclbin_get_metadata(struct device *dev, const struct axlf *xclbin, char **dtb) +{ + char *md = NULL, *newmd = NULL; + u64 len, md_len; + int rc; + + *dtb = NULL; + + rc = xrt_xclbin_get_section(dev, xclbin, PARTITION_METADATA, (void **)&md, &len); + if (rc) + goto done; + + md_len = xrt_md_size(dev, md); + + /* Sanity check the dtb section. */ + if (md_len > len) { + rc = -EINVAL; + goto done; + } + + /* use dup function here to convert incoming metadata to writable */ + newmd = xrt_md_dup(dev, md); + if (!newmd) { + rc = -EFAULT; + goto done; + } + + /* Convert various needed xclbin sections into dtb. */ + rc = xrt_xclbin_add_clock_metadata(dev, xclbin, newmd); + + if (!rc) + *dtb = newmd; + else + vfree(newmd); +done: + vfree(md); + return rc; +} +EXPORT_SYMBOL_GPL(xrt_xclbin_get_metadata); diff --git a/include/uapi/linux/fpga-xrt.h b/include/uapi/linux/fpga-xrt.h new file mode 100644 index 000000000000..ace7d3d09897 --- /dev/null +++ b/include/uapi/linux/fpga-xrt.h @@ -0,0 +1,428 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Xilinx FPGA compiled binary container format + * + * Copyright (C) 2015-2021, Xilinx Inc + */ + +#ifndef _UAPI_LINUX_FPGA_XRT_H +#define _UAPI_LINUX_FPGA_XRT_H + +#if defined(__KERNEL__) + +#include + +#elif defined(__cplusplus) + +#include +#include +#include +#include + +#else + +#include +#include +#include + +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +/** + * DOC: Container format for Xilinx FPGA images + * The container stores bitstreams, metadata and firmware images. + * xclbin/xsabin is an ELF-like binary container format. It is a structured + * series of sections. There is a file header followed by several section + * headers which is followed by sections. A section header points to an + * actual section. There is an optional signature at the end. The + * following figure illustrates a typical xclbin: + * + * +---------------------+ + * | | + * | HEADER | + * +---------------------+ + * | SECTION HEADER | + * | | + * +---------------------+ + * | ... | + * | | + * +---------------------+ + * | SECTION HEADER | + * | | + * +---------------------+ + * | SECTION | + * | | + * +---------------------+ + * | ... | + * | | + * +---------------------+ + * | SECTION | + * | | + * +---------------------+ + * | SIGNATURE | + * | (OPTIONAL) | + * +---------------------+ + */ + +enum XCLBIN_MODE { + XCLBIN_FLAT = 0, + XCLBIN_PR, + XCLBIN_TANDEM_STAGE2, + XCLBIN_TANDEM_STAGE2_WITH_PR, + XCLBIN_HW_EMU, + XCLBIN_SW_EMU, + XCLBIN_MODE_MAX +}; + +enum axlf_section_kind { + BITSTREAM = 0, + CLEARING_BITSTREAM, + EMBEDDED_METADATA, + FIRMWARE, + DEBUG_DATA, + SCHED_FIRMWARE, + MEM_TOPOLOGY, + CONNECTIVITY, + IP_LAYOUT, + DEBUG_IP_LAYOUT, + DESIGN_CHECK_POINT, + CLOCK_FREQ_TOPOLOGY, + MCS, + BMC, + BUILD_METADATA, + KEYVALUE_METADATA, + USER_METADATA, + DNA_CERTIFICATE, + PDI, + BITSTREAM_PARTIAL_PDI, + PARTITION_METADATA, + EMULATION_DATA, + SYSTEM_METADATA, + SOFT_KERNEL, + ASK_FLASH, + AIE_METADATA, + ASK_GROUP_TOPOLOGY, + ASK_GROUP_CONNECTIVITY +}; + +enum MEM_TYPE { + MEM_DDR3 = 0, + MEM_DDR4, + MEM_DRAM, + MEM_STREAMING, + MEM_PREALLOCATED_GLOB, + MEM_ARE, + MEM_HBM, + MEM_BRAM, + MEM_URAM, + MEM_STREAMING_CONNECTION +}; + +enum IP_TYPE { + IP_MB = 0, + IP_KERNEL, + IP_DNASC, + IP_DDR4_CONTROLLER, + IP_MEM_DDR4, + IP_MEM_HBM +}; + +struct axlf_section_header { + uint32_t section_kind; /* Section type */ + char section_name[16]; /* Section name */ + char rsvd[4]; + uint64_t section_offset; /* File offset of section data */ + uint64_t section_size; /* Size of section data */ +} __packed; + +struct axlf_header { + uint64_t length; /* Total size of the xclbin file */ + uint64_t time_stamp; /* Timestamp when xclbin was created */ + uint64_t feature_rom_timestamp; /* TimeSinceEpoch of the featureRom */ + uint16_t version_patch; /* Patch Version */ + uint8_t version_major; /* Major Version */ + uint8_t version_minor; /* Minor Version */ + uint32_t mode; /* Xclbin mode. See enum XCLBIN_MODE */ + union { + struct { + uint64_t platform_id; /* 64 bit platform ID */ + uint64_t feature_id; /* 64 bit feature ID */ + } rom; + unsigned char rom_uuid[16]; /* feature ROM UUID */ + }; + unsigned char platform_vbnv[64]; /* String of Vendor:Board:Name:Version */ + union { + char next_axlf[16]; /* Name of next xclbin file */ + unsigned char uuid[16]; /* uuid of this xclbin */ + }; + char debug_bin[16]; /* Name of binary with debug information */ + uint32_t num_sections; /* Number of section headers */ + char rsvd[4]; +} __packed; + +struct axlf { + char magic[8]; /* Magic word: xclbin2\0 */ + int32_t signature_length; /* Length. -1 indicates no signature */ + uint8_t reserved[28]; + uint8_t key_block[256]; /* Signature for validation of binary */ + uint64_t unique_id; /* Unique ID */ + struct axlf_header header; /* Inline header */ + struct axlf_section_header sections[1]; /* One or more section headers follow */ +} __packed; + +/* bitstream information */ +struct xlnx_bitstream { + uint8_t freq[8]; + char bits[1]; +} __packed; + +/**** MEMORY TOPOLOGY SECTION ****/ +struct mem_data { + uint8_t type; /* Memory type. See enum MEM_TYPE */ + uint8_t used; /* Memory presence. 0 means not present */ + uint8_t rsvd[6]; + union { + uint64_t size; /* Memory size. Counted in KB */ + uint64_t route_id; /* Stream route ID */ + }; + union { + uint64_t base_address; /* Memory base address */ + uint64_t flow_id; /* Stream flow ID */ + }; + unsigned char tag[16]; /* Memory tag string */ +} __packed; + +struct mem_topology { + int32_t count; /* Number of mem_data */ + struct mem_data mem_data[1]; /* mem_data array, sorted on mem_type */ +} __packed; + +/**** CONNECTIVITY SECTION ****/ +/* Connectivity of each argument of CU(Compute Unit). It will be in terms + * of argument index associated. For associating CU instances with arguments + * and banks, start at the connectivity section. Using the ip_layout_index + * access the ip_data.name. Now we can associate this CU instance with its + * original CU name and get the connectivity as well. This enables us to form + * related groups of CU instances. + */ + +struct connection { + int32_t arg_index; /* Index of CU argument */ + int32_t ip_layout_index; /* Index into the ip_layout section */ + int32_t mem_data_index; /* Index into the mem_data section */ +} __packed; + +struct connectivity { + int32_t count; + struct connection connection[1]; +} __packed; + +/**** IP_LAYOUT SECTION ****/ + +/* IP Kernel */ +#define IP_INT_ENABLE_MASK 0x0001 +#define IP_INTERRUPT_ID_MASK 0x00FE +#define IP_INTERRUPT_ID_SHIFT 0x1 + +enum IP_CONTROL { + AP_CTRL_HS = 0, + AP_CTRL_CHAIN, + AP_CTRL_NONE, + AP_CTRL_ME, + ACCEL_ADAPTER +}; + +#define IP_CONTROL_MASK 0xFF00 +#define IP_CONTROL_SHIFT 0x8 + +/* + * IPs on AXI lite - their types, names, and base addresses. + * + * The defination of 32-bit follows IP_TYPE is based on IP_TYPE + * For IP_KERNEL + * int_enable : Bit - 0x0000_0001; + * interrupt_id : Bits - 0x0000_00FE; + * ip_control : Bits - 0x0000_FF00; + * For IP_MEM_* + * index : Bits - 0x0000_FFFF; + * pc_index : Bits - 0x00FF_0000; + */ +struct ip_data { + uint32_t type; /* Type. See enum IP_TYPE */ + union { + uint32_t properties; + struct { + uint16_t index; + uint8_t pc_index; + uint8_t unused; + } indices; + }; + uint64_t base_address; + uint8_t name[64]; /* Name of IP */ +} __packed; + +struct ip_layout { + int32_t count; + struct ip_data ip_data[1]; /* ip_data array, sorted by base_address */ +} __packed; + +/*** Debug IP section layout ****/ +enum DEBUG_IP_TYPE { + UNDEFINED = 0, + LAPC, + ILA, + AXI_MM_MONITOR, + AXI_TRACE_FUNNEL, + AXI_MONITOR_FIFO_LITE, + AXI_MONITOR_FIFO_FULL, + ACCEL_MONITOR, + AXI_STREAM_MONITOR, + AXI_STREAM_PROTOCOL_CHECKER, + TRACE_S2MM, + AXI_DMA, + TRACE_S2MM_FULL +}; + +struct debug_ip_data { + uint8_t type; /* Type. See enum DEBUG_IP_TYPE */ + uint8_t index_lowbyte; + uint8_t properties; + uint8_t major; + uint8_t minor; + uint8_t index_highbyte; + uint8_t reserved[2]; + uint64_t base_address; + char name[128]; +} __packed; + +struct debug_ip_layout { + uint16_t count; + struct debug_ip_data debug_ip_data[1]; +} __packed; + +/* Supported clock frequency types */ +enum XCLBIN_CLOCK_TYPE { + CT_UNUSED = 0, /* Initialized value */ + CT_DATA = 1, /* Data clock */ + CT_KERNEL = 2, /* Kernel clock */ + CT_SYSTEM = 3 /* System Clock */ +}; + +/* Clock Frequency Entry */ +struct clock_freq { + uint16_t freq_MHZ; /* Frequency in MHz */ + uint8_t type; /* Clock type (enum CLOCK_TYPE) */ + uint8_t unused[5]; + char name[128]; /* Clock Name */ +} __packed; + +/* Clock frequency section */ +struct clock_freq_topology { + int16_t count; /* Number of entries */ + struct clock_freq clock_freq[1]; /* Clock array */ +} __packed; + +/* Supported MCS file types */ +enum MCS_TYPE { + MCS_UNKNOWN = 0, /* Initialized value */ + MCS_PRIMARY = 1, /* The primary mcs file data */ + MCS_SECONDARY = 2, /* The secondary mcs file data */ +}; + +/* One chunk of MCS data */ +struct mcs_chunk { + uint8_t type; /* MCS data type */ + uint8_t unused[7]; + uint64_t offset; /* Data offset */ + uint64_t size; /* Data size */ +} __packed; + +/* MCS data section */ +struct mcs { + int8_t count; /* Number of chunks */ + int8_t unused[7]; + struct mcs_chunk chunk[1]; /* MCS chunks followed by data */ +} __packed; + +/* bmc data section */ +struct bmc { + uint64_t offset; /* Data offset */ + uint64_t size; /* Data size (bytes) */ + char image_name[64]; /* Name of the image */ + char device_name[64]; /* Name of the device */ + char version[64]; + char md5value[33]; /* MD5 checksum */ + char padding[7]; +} __packed; + +/* + * soft kernel data section, used by classic driver + * Prefix Syntax: + * mpo - member, pointer, offset + * This variable represents a zero terminated string + * that is offseted from the beginning of the section. + * The pointer to access the string is initialized as follows: + * char * pCharString = (address_of_section) + (mpo value) + */ +struct soft_kernel { + uint32_t mpo_name; /* Name of the soft kernel */ + uint32_t image_offset; /* Image offset */ + uint32_t image_size; /* Image size */ + uint32_t mpo_version; /* Version */ + uint32_t mpo_md5_value; /* MD5 checksum */ + uint32_t mpo_symbol_name; /* Symbol name */ + uint32_t num_instances; /* Number of instances */ + uint8_t padding[36]; /* Reserved for future use */ + uint8_t reserved_ext[16]; /* Reserved for future extended data */ +} __packed; + +enum CHECKSUM_TYPE { + CST_UNKNOWN = 0, + CST_SDBM = 1, + CST_LAST +}; + +/** + * DOC: PCIe Kernel Driver for Management Physical Function + * Interfaces exposed by *xrt-mgmt* driver are defined in file, *xmgmt-ioctl.h*. + * Core functionality provided by *xrt-mgmt* driver is described in the following table: + * + * =========== ============================== ================================== + * Functionality ioctl request code data format + * =========== ============================== ================================== + * 1 FPGA image download XMGMT_IOCICAPDOWNLOAD_AXLF xmgmt_ioc_bitstream_axlf + * =========== ============================== ================================== + */ + +#define XMGMT_IOC_MAGIC 'X' +#define XMGMT_IOC_ICAP_DOWNLOAD_AXLF 0x6 + +/** + * struct xmgmt_ioc_bitstream_axlf - load xclbin (AXLF) device image + * used with XMGMT_IOCICAPDOWNLOAD_AXLF ioctl + * + * @xclbin: Pointer to user's xclbin structure in memory + */ +struct xmgmt_ioc_bitstream_axlf { + struct axlf *xclbin; +}; + +#define XMGMT_IOCICAPDOWNLOAD_AXLF \ + _IOW(XMGMT_IOC_MAGIC, XMGMT_IOC_ICAP_DOWNLOAD_AXLF, struct xmgmt_ioc_bitstream_axlf) + +/* + * The following definitions are for binary compatibility with classic XRT management driver + */ +#define XCLMGMT_IOCICAPDOWNLOAD_AXLF XMGMT_IOCICAPDOWNLOAD_AXLF +#define xclmgmt_ioc_bitstream_axlf xmgmt_ioc_bitstream_axlf + +#ifdef __cplusplus +} +#endif + +#endif From patchwork Mon Jul 19 21:26:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480112 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21D52C6377C for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE19A6113B for ; Mon, 19 Jul 2021 22:00:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378855AbhGSVQW (ORCPT ); Mon, 19 Jul 2021 17:16:22 -0400 Received: from mail-co1nam11on2049.outbound.protection.outlook.com ([40.107.220.49]:51259 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388229AbhGSUqo (ORCPT ); Mon, 19 Jul 2021 16:46:44 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RGys49Y0BlDf1FdKXJwMaGfwL3d66lG3kCVjvb/V1CEcFDHqR1KNia7jjOty7Hqkz2Jsq4b1Q2piKGJPy+CGsj7D+N10QQVukAPBTwJt5NLQEtSKxPWNiIsQ50oG6aCsmnVkJcWHiSy4yunzb7vVKjsL1Fxh+Z1wC/4TKroGHG0dnOOGCHf+yYMFbg4R/ZkLV24/psaTbIauUDmpTvMuloQDKVPzpVlGWKvHt/dtNO4jqFI22WpADQsoYoMuUZfCM/p/MKT1uS7+pPd/4usA4tWZKwBHUvRwTu57pQlvSXZQ8t3TQbyexumb2TE/vnyBfT07JxYanIQodw3VOcpe3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Dm2nAW14gh672fBo9jFiWRzUNF3pg1VZekmPsYyrVY=; b=lujQBwtvn47DC6LNM+nIFhitW0nDJRZDoShiVJhN5jX1RoAk0/HP4MsyoWD7A3wFdvClx0+IvFZjDkp0KAWI35BhUiFT1lqVrQzxCT9oSHGZdKX4qBeVPamDMz4SeNu5cBLM6kXkLcHthTqahPWo5KsfV2gyUPe8P4ozYcLxU1sncQ/MbiOu6Evrc8Z5wU07T00S5adPEbyTHkcPI17G9FGp4JTiE1JSPjKqc1dgGnIrhSmFm8+kzc01ri5nI2KOBCdPLRnjxe2zY5j+7nuQ6Nf0xCKnf6DyMlNz/6Bt9Vwo1Qe4iLpTpybWFI+6ZjdvIYzA8voav3Z8qB+5RQLGsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Dm2nAW14gh672fBo9jFiWRzUNF3pg1VZekmPsYyrVY=; b=sXbS3AXbj//KqQU6USfsEYTNjmtC8uzAbb/+p1hzUdtnlMnSgqWdklceKV6CwOUlOeoyTVqrzsnaTvoVz/jqFtT19Yq4zaxhA5Nql7EPr8PDdyq86OTHAHCP5KecZuQRTptZRjnrChPYXCYwoj3fqpJQg/WITRn1DpnTvm9r2LI= Received: from DM5PR12CA0052.namprd12.prod.outlook.com (2603:10b6:3:103::14) by BN7PR02MB3956.namprd02.prod.outlook.com (2603:10b6:406:f4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 21:27:21 +0000 Received: from DM3NAM02FT024.eop-nam02.prod.protection.outlook.com (2603:10b6:3:103:cafe::b5) by DM5PR12CA0052.outlook.office365.com (2603:10b6:3:103::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT024.mail.protection.outlook.com (10.13.5.128) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:21 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:27:20 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:27:20 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38272 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5anI-000F8r-2w; Mon, 19 Jul 2021 14:27:20 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 866E56008D8; Mon, 19 Jul 2021 14:26:31 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 04/14] fpga: xrt: xrt-lib driver manager Date: Mon, 19 Jul 2021 14:26:18 -0700 Message-ID: <20210719212628.134129-5-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bdc5d411-2f79-458f-2edc-08d94afbfe2f X-MS-TrafficTypeDiagnostic: BN7PR02MB3956: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:132; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2TiGVpJ9lKuclZwoJdDqgpY1VtAOscxe7agAsTTnQ6gvX44uno0uZqTR8IysDw189mnPiegV4NvLHS90aYQ1kDESfLddtzrasjV8wZ8msS3IeVISlYfH1p2eFquz21CRg3IqAW0PDnXAHDCKWfNqQzy80kU23nt28rPuM27Yxl8riYeY2D8LpWLpFvImd8JlY/Isf+9dW9VNzotnX2D/A1qJhQkkXZoJDa6BlyxqLcBi+it7HPrWQqeLJSm1L1OvyyyTPICHsxclap/mA8w0yrySDS1E4BJRK86SEd+kiX9doiDjZbr5WPjgSKF1ihIzhS+EZniGbYGjcl6Ky0ddvTdF3R0ntwUOW6wPi5CP9bPt3cL5Y4FiVCn+7ltvnezRTXgUkv+7uNCUMLvuLRokuxyE0MyxaHhx4X2AwoK1Zcwo8lbDFnh65KIY+8ADx5Zp8/0mlgBYdZj2s9ln+ViH5y7NSMC7iIxnwh6WF+zn1O7W5Pua8OuNI5TCgnQsbt1K9rpHCrFY5SYuzZ7dfDVzO9NvKF7glLz/BCcnQgcZnB/PJr6ueW8kkkO47ZCpHR+DvB2JiPHIp+UXINL1FBuU7kYHx85RS5n9mkDj9P9+kIKlw6tgBGqTJiheP6yuqd5xmZFg+3lk5/5SrQFASWl44iM7nub+pf3w+hIomK7rCDNg5oW61hl6/+z5KaY7crRRBeDcvwDxsl8G7IA+zjEbfA== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(136003)(39860400002)(346002)(376002)(396003)(36840700001)(46966006)(6266002)(82310400003)(6666004)(4326008)(186003)(1076003)(8936002)(478600001)(36756003)(8676002)(83380400001)(26005)(2906002)(6916009)(36860700001)(7636003)(30864003)(36906005)(426003)(2616005)(54906003)(44832011)(47076005)(356005)(70586007)(70206006)(42186006)(316002)(82740400003)(5660300002)(107886003)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:27:21.0507 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bdc5d411-2f79-458f-2edc-08d94afbfe2f X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT024.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR02MB3956 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org xrt-lib kernel module infrastructure code to register and manage all leaf driver modules. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/include/subdev_id.h | 39 +++ drivers/fpga/xrt/include/xdevice.h | 131 +++++++ drivers/fpga/xrt/include/xleaf.h | 205 +++++++++++ drivers/fpga/xrt/include/xleaf/clkfreq.h | 21 ++ drivers/fpga/xrt/include/xleaf/clock.h | 29 ++ .../fpga/xrt/include/xleaf/ddr_calibration.h | 28 ++ drivers/fpga/xrt/include/xleaf/devctl.h | 40 +++ drivers/fpga/xrt/lib/lib-drv.c | 322 ++++++++++++++++++ drivers/fpga/xrt/lib/lib-drv.h | 21 ++ 9 files changed, 836 insertions(+) create mode 100644 drivers/fpga/xrt/include/subdev_id.h create mode 100644 drivers/fpga/xrt/include/xdevice.h create mode 100644 drivers/fpga/xrt/include/xleaf.h create mode 100644 drivers/fpga/xrt/include/xleaf/clkfreq.h create mode 100644 drivers/fpga/xrt/include/xleaf/clock.h create mode 100644 drivers/fpga/xrt/include/xleaf/ddr_calibration.h create mode 100644 drivers/fpga/xrt/include/xleaf/devctl.h create mode 100644 drivers/fpga/xrt/lib/lib-drv.c create mode 100644 drivers/fpga/xrt/lib/lib-drv.h diff --git a/drivers/fpga/xrt/include/subdev_id.h b/drivers/fpga/xrt/include/subdev_id.h new file mode 100644 index 000000000000..02df4b939a1b --- /dev/null +++ b/drivers/fpga/xrt/include/subdev_id.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#ifndef _XRT_SUBDEV_ID_H_ +#define _XRT_SUBDEV_ID_H_ + +/* + * Every subdev driver has an ID for others to refer to it. There can be multiple number of + * instances of a subdev driver. A tuple is a unique identification + * of a specific instance of a subdev driver. + */ +enum xrt_subdev_id { + XRT_SUBDEV_INVALID = 0, + XRT_SUBDEV_GRP, + XRT_SUBDEV_VSEC, + XRT_SUBDEV_VSEC_GOLDEN, + XRT_SUBDEV_DEVCTL, + XRT_SUBDEV_AXIGATE, + XRT_SUBDEV_ICAP, + XRT_SUBDEV_TEST, + XRT_SUBDEV_MGMT_MAIN, + XRT_SUBDEV_QSPI, + XRT_SUBDEV_MAILBOX, + XRT_SUBDEV_CMC, + XRT_SUBDEV_CALIB, + XRT_SUBDEV_CLKFREQ, + XRT_SUBDEV_CLOCK, + XRT_SUBDEV_SRSR, + XRT_SUBDEV_UCS, + XRT_SUBDEV_NUM, /* Total number of subdevs. */ + XRT_ROOT = -1, /* Special ID for root driver. */ +}; + +#endif /* _XRT_SUBDEV_ID_H_ */ diff --git a/drivers/fpga/xrt/include/xdevice.h b/drivers/fpga/xrt/include/xdevice.h new file mode 100644 index 000000000000..3afd96989fc5 --- /dev/null +++ b/drivers/fpga/xrt/include/xdevice.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Lizhi Hou + */ + +#ifndef _XRT_DEVICE_H_ +#define _XRT_DEVICE_H_ + +#include +#include + +#define XRT_MAX_DEVICE_NODES 128 +#define XRT_INVALID_DEVICE_INST (XRT_MAX_DEVICE_NODES + 1) + +enum { + XRT_DEVICE_STATE_NONE = 0, + XRT_DEVICE_STATE_ADDED +}; + +/* + * struct xrt_device - represent an xrt device on xrt bus + * + * dev: generic device interface. + * id: id of the xrt device. + */ +struct xrt_device { + struct device dev; + u32 subdev_id; + const char *name; + u32 instance; + u32 state; + u32 num_resources; + struct resource *resource; + void *sdev_data; +}; + +/* + * If populated by xrt device driver, infra will handle the mechanics of + * char device (un)registration. + */ +enum xrt_dev_file_mode { + /* Infra create cdev, default file name */ + XRT_DEV_FILE_DEFAULT = 0, + /* Infra create cdev, need to encode inst num in file name */ + XRT_DEV_FILE_MULTI_INST, + /* No auto creation of cdev by infra, leaf handles it by itself */ + XRT_DEV_FILE_NO_AUTO, +}; + +struct xrt_dev_file_ops { + const struct file_operations xsf_ops; + dev_t xsf_dev_t; + const char *xsf_dev_name; + enum xrt_dev_file_mode xsf_mode; +}; + +/* + * this struct define the endpoints belong to the same xrt device + */ +struct xrt_dev_ep_names { + const char *ep_name; + const char *compat; +}; + +struct xrt_dev_endpoints { + struct xrt_dev_ep_names *xse_names; + /* minimum number of endpoints to support the subdevice */ + u32 xse_min_ep; +}; + +/* + * struct xrt_driver - represent a xrt device driver + * + * drv: driver model structure. + * id_table: pointer to table of device IDs the driver is interested in. + * { } member terminated. + * probe: mandatory callback for device binding. + * remove: callback for device unbinding. + */ +struct xrt_driver { + struct device_driver driver; + u32 subdev_id; + struct xrt_dev_file_ops file_ops; + struct xrt_dev_endpoints *endpoints; + + /* + * Subdev driver callbacks populated by subdev driver. + */ + int (*probe)(struct xrt_device *xrt_dev); + void (*remove)(struct xrt_device *xrt_dev); + /* + * If leaf_call is defined, these are called by other leaf drivers. + * Note that root driver may call into leaf_call of a group driver. + */ + int (*leaf_call)(struct xrt_device *xrt_dev, u32 cmd, void *arg); +}; + +#define to_xrt_dev(d) container_of(d, struct xrt_device, dev) +#define to_xrt_drv(d) container_of(d, struct xrt_driver, driver) + +static inline void *xrt_get_drvdata(const struct xrt_device *xdev) +{ + return dev_get_drvdata(&xdev->dev); +} + +static inline void xrt_set_drvdata(struct xrt_device *xdev, void *data) +{ + dev_set_drvdata(&xdev->dev, data); +} + +static inline void *xrt_get_xdev_data(struct device *dev) +{ + struct xrt_device *xdev = to_xrt_dev(dev); + + return xdev->sdev_data; +} + +struct xrt_device * +xrt_device_register(struct device *parent, u32 id, + struct resource *res, u32 res_num, + void *pdata, size_t data_sz); +void xrt_device_unregister(struct xrt_device *xdev); +int xrt_register_driver(struct xrt_driver *drv); +void xrt_unregister_driver(struct xrt_driver *drv); +void *xrt_get_xdev_data(struct device *dev); +struct resource *xrt_get_resource(struct xrt_device *xdev, u32 type, u32 num); + +#endif /* _XRT_DEVICE_H_ */ diff --git a/drivers/fpga/xrt/include/xleaf.h b/drivers/fpga/xrt/include/xleaf.h new file mode 100644 index 000000000000..f065fc766e0f --- /dev/null +++ b/drivers/fpga/xrt/include/xleaf.h @@ -0,0 +1,205 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + * Sonal Santan + */ + +#ifndef _XRT_XLEAF_H_ +#define _XRT_XLEAF_H_ + +#include +#include "xdevice.h" +#include "subdev_id.h" +#include "xroot.h" +#include "events.h" + +/* All subdev drivers should use below common routines to print out msg. */ +#define DEV(xdev) (&(xdev)->dev) +#define DEV_PDATA(xdev) \ + ((struct xrt_subdev_platdata *)xrt_get_xdev_data(DEV(xdev))) +#define DEV_FILE_OPS(xdev) \ + (&(to_xrt_drv((xdev)->dev.driver))->file_ops) +#define FMT_PRT(prt_fn, xdev, fmt, args...) \ + ({typeof(xdev) (_xdev) = (xdev); \ + prt_fn(DEV(_xdev), "%s %s: " fmt, \ + DEV_PDATA(_xdev)->xsp_root_name, __func__, ##args); }) +#define xrt_err(xdev, fmt, args...) FMT_PRT(dev_err, xdev, fmt, ##args) +#define xrt_warn(xdev, fmt, args...) FMT_PRT(dev_warn, xdev, fmt, ##args) +#define xrt_info(xdev, fmt, args...) FMT_PRT(dev_info, xdev, fmt, ##args) +#define xrt_dbg(xdev, fmt, args...) FMT_PRT(dev_dbg, xdev, fmt, ##args) + +#define XRT_DEFINE_REGMAP_CONFIG(config_name) \ + static const struct regmap_config config_name = { \ + .reg_bits = 32, \ + .val_bits = 32, \ + .reg_stride = 4, \ + .max_register = 0x1000, \ + } + +enum { + /* Starting cmd for common leaf cmd implemented by all leaves. */ + XRT_XLEAF_COMMON_BASE = 0, + /* Starting cmd for leaves' specific leaf cmds. */ + XRT_XLEAF_CUSTOM_BASE = 64, +}; + +enum xrt_xleaf_common_leaf_cmd { + XRT_XLEAF_EVENT = XRT_XLEAF_COMMON_BASE, +}; + +/* + * Partially initialized by the parent driver, then, passed in as subdev driver's + * platform data when creating subdev driver instance by calling platform + * device register API (xrt_device_register_data() or the likes). + * + * Once device register API returns, platform driver framework makes a copy of + * this buffer and maintains its life cycle. The content of the buffer is + * completely owned by subdev driver. + * + * Thus, parent driver should be very careful when it touches this buffer + * again once it's handed over to subdev driver. And the data structure + * should not contain pointers pointing to buffers that is managed by + * other or parent drivers since it could have been freed before platform + * data buffer is freed by platform driver framework. + */ +struct xrt_subdev_platdata { + /* + * Per driver instance callback. The xdev points to the instance. + * Should always be defined for subdev driver to get service from root. + */ + xrt_subdev_root_cb_t xsp_root_cb; + void *xsp_root_cb_arg; + + /* Something to associate w/ root for msg printing. */ + const char *xsp_root_name; + + /* + * Char dev support for this subdev instance. + * Initialized by subdev driver. + */ + struct cdev xsp_cdev; + struct device *xsp_sysdev; + struct mutex xsp_devnode_lock; /* devnode lock */ + struct completion xsp_devnode_comp; + int xsp_devnode_ref; + bool xsp_devnode_online; + bool xsp_devnode_excl; + + /* + * Subdev driver specific init data. The buffer should be embedded + * in this data structure buffer after dtb, so that it can be freed + * together with platform data. + */ + loff_t xsp_priv_off; /* Offset into this platform data buffer. */ + size_t xsp_priv_len; + + /* + * Populated by parent driver to describe the device tree for + * the subdev driver to handle. Should always be last one since it's + * of variable length. + */ + bool xsp_dtb_valid; + char xsp_dtb[0]; +}; + +struct subdev_match_arg { + enum xrt_subdev_id id; + int instance; +}; + +bool xleaf_has_endpoint(struct xrt_device *xdev, const char *endpoint_name); +struct xrt_device *xleaf_get_leaf(struct xrt_device *xdev, + xrt_subdev_match_t cb, void *arg); + +static inline bool subdev_match(enum xrt_subdev_id id, struct xrt_device *xdev, void *arg) +{ + const struct subdev_match_arg *a = (struct subdev_match_arg *)arg; + int instance = a->instance; + + if (id != a->id) + return false; + if (instance != xdev->instance && instance != XRT_INVALID_DEVICE_INST) + return false; + return true; +} + +static inline bool xrt_subdev_match_epname(enum xrt_subdev_id id, + struct xrt_device *xdev, void *arg) +{ + return xleaf_has_endpoint(xdev, arg); +} + +static inline struct xrt_device * +xleaf_get_leaf_by_id(struct xrt_device *xdev, + enum xrt_subdev_id id, int instance) +{ + struct subdev_match_arg arg = { id, instance }; + + return xleaf_get_leaf(xdev, subdev_match, &arg); +} + +static inline struct xrt_device * +xleaf_get_leaf_by_epname(struct xrt_device *xdev, const char *name) +{ + return xleaf_get_leaf(xdev, xrt_subdev_match_epname, (void *)name); +} + +static inline int xleaf_call(struct xrt_device *tgt, u32 cmd, void *arg) +{ + return (to_xrt_drv(tgt->dev.driver)->leaf_call)(tgt, cmd, arg); +} + +int xleaf_broadcast_event(struct xrt_device *xdev, enum xrt_events evt, bool async); +int xleaf_create_group(struct xrt_device *xdev, char *dtb); +int xleaf_destroy_group(struct xrt_device *xdev, int instance); +void xleaf_get_root_res(struct xrt_device *xdev, u32 region_id, struct resource **res); +void xleaf_get_root_id(struct xrt_device *xdev, unsigned short *vendor, unsigned short *device, + unsigned short *subvendor, unsigned short *subdevice); +void xleaf_hot_reset(struct xrt_device *xdev); +int xleaf_put_leaf(struct xrt_device *xdev, struct xrt_device *leaf); +struct device *xleaf_register_hwmon(struct xrt_device *xdev, const char *name, void *drvdata, + const struct attribute_group **grps); +void xleaf_unregister_hwmon(struct xrt_device *xdev, struct device *hwmon); +int xleaf_wait_for_group_bringup(struct xrt_device *xdev); + +/* + * Character device helper APIs for use by leaf drivers + */ +static inline bool xleaf_devnode_enabled(struct xrt_device *xdev) +{ + return DEV_FILE_OPS(xdev)->xsf_ops.open; +} + +int xleaf_devnode_create(struct xrt_device *xdev, + const char *file_name, const char *inst_name); +void xleaf_devnode_destroy(struct xrt_device *xdev); + +struct xrt_device *xleaf_devnode_open_excl(struct inode *inode); +struct xrt_device *xleaf_devnode_open(struct inode *inode); +void xleaf_devnode_close(struct inode *inode); + +/* Module's init/fini routines for leaf driver in xrt-lib module */ +#define XRT_LEAF_INIT_FINI_FUNC(name) \ +void name##_leaf_init_fini(bool init) \ +{ \ + if (init) \ + xrt_register_driver(&xrt_##name##_driver); \ + else \ + xrt_unregister_driver(&xrt_##name##_driver); \ +} + +/* Module's init/fini routines for leaf driver in xrt-lib module */ +void group_leaf_init_fini(bool init); +void vsec_leaf_init_fini(bool init); +void devctl_leaf_init_fini(bool init); +void axigate_leaf_init_fini(bool init); +void icap_leaf_init_fini(bool init); +void calib_leaf_init_fini(bool init); +void clkfreq_leaf_init_fini(bool init); +void clock_leaf_init_fini(bool init); +void ucs_leaf_init_fini(bool init); + +#endif /* _XRT_LEAF_H_ */ diff --git a/drivers/fpga/xrt/include/xleaf/clkfreq.h b/drivers/fpga/xrt/include/xleaf/clkfreq.h new file mode 100644 index 000000000000..005441d5df78 --- /dev/null +++ b/drivers/fpga/xrt/include/xleaf/clkfreq.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Lizhi Hou + */ + +#ifndef _XRT_CLKFREQ_H_ +#define _XRT_CLKFREQ_H_ + +#include "xleaf.h" + +/* + * CLKFREQ driver leaf calls. + */ +enum xrt_clkfreq_leaf_cmd { + XRT_CLKFREQ_READ = XRT_XLEAF_CUSTOM_BASE, /* See comments in xleaf.h */ +}; + +#endif /* _XRT_CLKFREQ_H_ */ diff --git a/drivers/fpga/xrt/include/xleaf/clock.h b/drivers/fpga/xrt/include/xleaf/clock.h new file mode 100644 index 000000000000..1379e24fa5d0 --- /dev/null +++ b/drivers/fpga/xrt/include/xleaf/clock.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Lizhi Hou + */ + +#ifndef _XRT_CLOCK_H_ +#define _XRT_CLOCK_H_ + +#include "xleaf.h" +#include + +/* + * CLOCK driver leaf calls. + */ +enum xrt_clock_leaf_cmd { + XRT_CLOCK_SET = XRT_XLEAF_CUSTOM_BASE, /* See comments in xleaf.h */ + XRT_CLOCK_GET, + XRT_CLOCK_VERIFY, +}; + +struct xrt_clock_get { + u16 freq; + u32 freq_cnter; +}; + +#endif /* _XRT_CLOCK_H_ */ diff --git a/drivers/fpga/xrt/include/xleaf/ddr_calibration.h b/drivers/fpga/xrt/include/xleaf/ddr_calibration.h new file mode 100644 index 000000000000..c44ae30f939a --- /dev/null +++ b/drivers/fpga/xrt/include/xleaf/ddr_calibration.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#ifndef _XRT_DDR_CALIBRATION_H_ +#define _XRT_DDR_CALIBRATION_H_ + +#include "xleaf.h" +#include + +/* + * Memory calibration driver leaf calls. + */ +enum xrt_calib_results { + XRT_CALIB_UNKNOWN = 0, + XRT_CALIB_SUCCEEDED, + XRT_CALIB_FAILED, +}; + +enum xrt_calib_leaf_cmd { + XRT_CALIB_RESULT = XRT_XLEAF_CUSTOM_BASE, /* See comments in xleaf.h */ +}; + +#endif /* _XRT_DDR_CALIBRATION_H_ */ diff --git a/drivers/fpga/xrt/include/xleaf/devctl.h b/drivers/fpga/xrt/include/xleaf/devctl.h new file mode 100644 index 000000000000..b97f3b6d9326 --- /dev/null +++ b/drivers/fpga/xrt/include/xleaf/devctl.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Lizhi Hou + */ + +#ifndef _XRT_DEVCTL_H_ +#define _XRT_DEVCTL_H_ + +#include "xleaf.h" + +/* + * DEVCTL driver leaf calls. + */ +enum xrt_devctl_leaf_cmd { + XRT_DEVCTL_READ = XRT_XLEAF_CUSTOM_BASE, /* See comments in xleaf.h */ +}; + +enum xrt_devctl_id { + XRT_DEVCTL_ROM_UUID = 0, + XRT_DEVCTL_DDR_CALIB, + XRT_DEVCTL_GOLDEN_VER, + XRT_DEVCTL_MAX +}; + +struct xrt_devctl_rw { + u32 xdr_id; + void *xdr_buf; + u32 xdr_len; + u32 xdr_offset; +}; + +struct xrt_devctl_intf_uuid { + u32 uuid_num; + uuid_t *uuids; +}; + +#endif /* _XRT_DEVCTL_H_ */ diff --git a/drivers/fpga/xrt/lib/lib-drv.c b/drivers/fpga/xrt/lib/lib-drv.c new file mode 100644 index 000000000000..2c2f9fe3e07e --- /dev/null +++ b/drivers/fpga/xrt/lib/lib-drv.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + * Lizhi Hou + */ + +#include +#include +#include +#include "xleaf.h" +#include "xroot.h" +#include "lib-drv.h" + +#define XRT_IPLIB_MODULE_NAME "xrt-lib" +#define XRT_IPLIB_MODULE_VERSION "4.0.0" +#define XRT_DRVNAME(drv) ((drv)->driver.name) + +#define XRT_SUBDEV_ID_SHIFT 16 +#define XRT_SUBDEV_ID_MASK ((1 << XRT_SUBDEV_ID_SHIFT) - 1) + +struct xrt_find_drv_data { + enum xrt_subdev_id id; + struct xrt_driver *xdrv; +}; + +struct class *xrt_class; +static DEFINE_IDA(xrt_device_ida); + +static inline u32 xrt_instance_to_id(enum xrt_subdev_id id, u32 instance) +{ + return (id << XRT_SUBDEV_ID_SHIFT) | instance; +} + +static inline u32 xrt_id_to_instance(u32 id) +{ + return (id & XRT_SUBDEV_ID_MASK); +} + +static int xrt_bus_match(struct device *dev, struct device_driver *drv) +{ + struct xrt_device *xdev = to_xrt_dev(dev); + struct xrt_driver *xdrv = to_xrt_drv(drv); + + if (xdev->subdev_id == xdrv->subdev_id) + return 1; + + return 0; +} + +static int xrt_bus_probe(struct device *dev) +{ + struct xrt_driver *xdrv = to_xrt_drv(dev->driver); + struct xrt_device *xdev = to_xrt_dev(dev); + + return xdrv->probe(xdev); +} + +static int xrt_bus_remove(struct device *dev) +{ + struct xrt_driver *xdrv = to_xrt_drv(dev->driver); + struct xrt_device *xdev = to_xrt_dev(dev); + + if (xdrv->remove) + xdrv->remove(xdev); + + return 0; +} + +struct bus_type xrt_bus_type = { + .name = "xrt", + .match = xrt_bus_match, + .probe = xrt_bus_probe, + .remove = xrt_bus_remove, +}; + +int xrt_register_driver(struct xrt_driver *drv) +{ + const char *drvname = XRT_DRVNAME(drv); + int rc = 0; + + /* Initialize dev_t for char dev node. */ + if (drv->file_ops.xsf_ops.open) { + rc = alloc_chrdev_region(&drv->file_ops.xsf_dev_t, 0, + XRT_MAX_DEVICE_NODES, drvname); + if (rc) { + pr_err("failed to alloc dev minor for %s: %d\n", drvname, rc); + return rc; + } + } else { + drv->file_ops.xsf_dev_t = (dev_t)-1; + } + + drv->driver.owner = THIS_MODULE; + drv->driver.bus = &xrt_bus_type; + + rc = driver_register(&drv->driver); + if (rc) { + pr_err("register %s xrt driver failed\n", drvname); + if (drv->file_ops.xsf_dev_t != (dev_t)-1) { + unregister_chrdev_region(drv->file_ops.xsf_dev_t, + XRT_MAX_DEVICE_NODES); + } + return rc; + } + + pr_info("%s registered successfully\n", drvname); + + return 0; +} +EXPORT_SYMBOL_GPL(xrt_register_driver); + +void xrt_unregister_driver(struct xrt_driver *drv) +{ + driver_unregister(&drv->driver); + + if (drv->file_ops.xsf_dev_t != (dev_t)-1) + unregister_chrdev_region(drv->file_ops.xsf_dev_t, XRT_MAX_DEVICE_NODES); + + pr_info("%s unregistered successfully\n", XRT_DRVNAME(drv)); +} +EXPORT_SYMBOL_GPL(xrt_unregister_driver); + +static int __find_driver(struct device_driver *drv, void *_data) +{ + struct xrt_driver *xdrv = to_xrt_drv(drv); + struct xrt_find_drv_data *data = _data; + + if (xdrv->subdev_id == data->id) { + data->xdrv = xdrv; + return 1; + } + + return 0; +} + +const char *xrt_drv_name(enum xrt_subdev_id id) +{ + struct xrt_find_drv_data data = { 0 }; + + data.id = id; + bus_for_each_drv(&xrt_bus_type, NULL, &data, __find_driver); + + if (data.xdrv) + return XRT_DRVNAME(data.xdrv); + + return NULL; +} + +static int xrt_drv_get_instance(enum xrt_subdev_id id) +{ + int ret; + + ret = ida_alloc_range(&xrt_device_ida, xrt_instance_to_id(id, 0), + xrt_instance_to_id(id, XRT_MAX_DEVICE_NODES), + GFP_KERNEL); + if (ret < 0) + return ret; + + return xrt_id_to_instance((u32)ret); +} + +static void xrt_drv_put_instance(enum xrt_subdev_id id, int instance) +{ + ida_free(&xrt_device_ida, xrt_instance_to_id(id, instance)); +} + +struct xrt_dev_endpoints *xrt_drv_get_endpoints(enum xrt_subdev_id id) +{ + struct xrt_find_drv_data data = { 0 }; + + data.id = id; + bus_for_each_drv(&xrt_bus_type, NULL, &data, __find_driver); + + if (data.xdrv) + return data.xdrv->endpoints; + + return NULL; +} + +static void xrt_device_release(struct device *dev) +{ + struct xrt_device *xdev = container_of(dev, struct xrt_device, dev); + + kfree(xdev); +} + +void xrt_device_unregister(struct xrt_device *xdev) +{ + if (xdev->state == XRT_DEVICE_STATE_ADDED) + device_del(&xdev->dev); + + vfree(xdev->sdev_data); + kfree(xdev->resource); + + if (xdev->instance != XRT_INVALID_DEVICE_INST) + xrt_drv_put_instance(xdev->subdev_id, xdev->instance); + + if (xdev->dev.release == xrt_device_release) + put_device(&xdev->dev); +} + +struct xrt_device * +xrt_device_register(struct device *parent, u32 id, + struct resource *res, u32 res_num, + void *pdata, size_t data_sz) +{ + struct xrt_device *xdev = NULL; + int ret; + + xdev = kzalloc(sizeof(*xdev), GFP_KERNEL); + if (!xdev) + return NULL; + xdev->instance = XRT_INVALID_DEVICE_INST; + + /* Obtain dev instance number. */ + ret = xrt_drv_get_instance(id); + if (ret < 0) { + dev_err(parent, "failed get instance, ret %d", ret); + goto fail; + } + + xdev->instance = ret; + xdev->name = xrt_drv_name(id); + xdev->subdev_id = id; + device_initialize(&xdev->dev); + xdev->dev.release = xrt_device_release; + xdev->dev.parent = parent; + + xdev->dev.bus = &xrt_bus_type; + dev_set_name(&xdev->dev, "%s.%d", xdev->name, xdev->instance); + + xdev->num_resources = res_num; + xdev->resource = kmemdup(res, sizeof(*res) * res_num, GFP_KERNEL); + if (!xdev->resource) + goto fail; + + xdev->sdev_data = vzalloc(data_sz); + if (!xdev->sdev_data) + goto fail; + + memcpy(xdev->sdev_data, pdata, data_sz); + + ret = device_add(&xdev->dev); + if (ret) { + dev_err(parent, "failed add device, ret %d", ret); + goto fail; + } + xdev->state = XRT_DEVICE_STATE_ADDED; + + return xdev; + +fail: + xrt_device_unregister(xdev); + kfree(xdev); + + return NULL; +} + +struct resource *xrt_get_resource(struct xrt_device *xdev, u32 type, u32 num) +{ + u32 i; + + for (i = 0; i < xdev->num_resources; i++) { + struct resource *r = &xdev->resource[i]; + + if (type == resource_type(r) && num-- == 0) + return r; + } + return NULL; +} + +/* + * Leaf driver's module init/fini callbacks. This is not a open infrastructure for dynamic + * plugging in drivers. All drivers should be statically added. + */ +static void (*leaf_init_fini_cbs[])(bool) = { + group_leaf_init_fini, + axigate_leaf_init_fini, + icap_leaf_init_fini, +}; + +static __init int xrt_lib_init(void) +{ + int ret; + int i; + + ret = bus_register(&xrt_bus_type); + if (ret) + return ret; + + xrt_class = class_create(THIS_MODULE, XRT_IPLIB_MODULE_NAME); + if (IS_ERR(xrt_class)) { + bus_unregister(&xrt_bus_type); + return PTR_ERR(xrt_class); + } + + for (i = 0; i < ARRAY_SIZE(leaf_init_fini_cbs); i++) + leaf_init_fini_cbs[i](true); + return 0; +} + +static __exit void xrt_lib_fini(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(leaf_init_fini_cbs); i++) + leaf_init_fini_cbs[i](false); + + class_destroy(xrt_class); + bus_unregister(&xrt_bus_type); +} + +module_init(xrt_lib_init); +module_exit(xrt_lib_fini); + +MODULE_VERSION(XRT_IPLIB_MODULE_VERSION); +MODULE_AUTHOR("XRT Team "); +MODULE_DESCRIPTION("Xilinx Alveo IP Lib driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/fpga/xrt/lib/lib-drv.h b/drivers/fpga/xrt/lib/lib-drv.h new file mode 100644 index 000000000000..0276c28e009f --- /dev/null +++ b/drivers/fpga/xrt/lib/lib-drv.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#ifndef _LIB_DRV_H_ +#define _LIB_DRV_H_ + +#include +#include + +extern struct class *xrt_class; +extern struct bus_type xrt_bus_type; + +const char *xrt_drv_name(enum xrt_subdev_id id); +struct xrt_dev_endpoints *xrt_drv_get_endpoints(enum xrt_subdev_id id); + +#endif /* _LIB_DRV_H_ */ From patchwork Mon Jul 19 21:26:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28179C6377B for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DF616112D for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1379481AbhGSVQa (ORCPT ); Mon, 19 Jul 2021 17:16:30 -0400 Received: from mail-bn7nam10on2067.outbound.protection.outlook.com ([40.107.92.67]:3009 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388232AbhGSUq5 (ORCPT ); Mon, 19 Jul 2021 16:46:57 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e2ajGfUnWt86g4aCTzn4juwTskswVPnAMAUIbPWgkEAzdMBYGQEE8W1lWr7MmEBwWvFAPV5QAxfm+V736bO9asMxE7K5xj2skRbTiabNbb7F3qgfFMsHhV8AjxaVjJu3nhwWoIGzU2+iFIoXpmlDAl3sfc91N8UGtkMUJn5v+7EWVIZ5jmbvzoazNta09zyRu4/rhFQ9gy5T3P7kZidGb2ynYhHJCF6Q2tlP2JOLxj/9yIeoicGEwBCXFE5QS5a418q28TvqGvKFZoJ/+rqd5yb6SajxnkqxEQnt970cMuU6R3eVneprhH62xEwVBGoTdhQluoRGw3YR+SeUHGObag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PCX0uLyI8DyDALmjDSx/DCq6ZEGpSQm1m8kEhGOkgTs=; b=Skg/BpETiH79Fuwg2/hSJQWCYjX4wNJDt0/6TD2TPpsOpuOAdUtj/EIOPXYY9ZStdCU0KtdBa8YIZ5+8cXlFwXRo4aSA3ra1xlQ0xht3MtcVVYM0YMe6jVoJNVAa7r9h25a0uS0Qb2OFy+DKmKGJdKmjqCgCvvZI4zD67SZC62qPyFhVGtbmLxAiK5B+QYbJ+1YvdmyQXK2ImU5qnUDqdRKcccncOhFhb9aBx90rz8VWtFYNAHX4HO6wr/+TS7OQi9/QEycqjGV/j+/QWdiGUR167z6mcLex/Fqhlaxd8soPwzfJkYIEaYPDuUXlBuMVnt4I5m9FS4u4XSf/zENfpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PCX0uLyI8DyDALmjDSx/DCq6ZEGpSQm1m8kEhGOkgTs=; b=h4jJ81ICyu2kLUAWu1RB5gc4FzAYRNTHG8TvfzrP46inyrOIA+Ze9FJyXQoe0hydYGbPJ0KHm0JZZunaO4+Dq4Z0+8MQxQNndaTtgdQ9bqHmLECklu6aG8WWTpBls72Lhd+8knYux+xgJcidEE5JeJZfjDmYXRotKXtrAdEkZBc= Received: from DM5PR20CA0024.namprd20.prod.outlook.com (2603:10b6:3:93::34) by BN6PR02MB2482.namprd02.prod.outlook.com (2603:10b6:404:55::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.29; Mon, 19 Jul 2021 21:27:33 +0000 Received: from DM3NAM02FT054.eop-nam02.prod.protection.outlook.com (2603:10b6:3:93:cafe::3d) by DM5PR20CA0024.outlook.office365.com (2603:10b6:3:93::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT054.mail.protection.outlook.com (10.13.5.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:33 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:27:32 -0700 Received: from smtp.xilinx.com (172.19.127.95) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:27:32 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38274 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5anU-0004xG-4P; Mon, 19 Jul 2021 14:27:32 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 9CFBB6020BC; Mon, 19 Jul 2021 14:26:31 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 05/14] fpga: xrt: group driver Date: Mon, 19 Jul 2021 14:26:19 -0700 Message-ID: <20210719212628.134129-6-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d686a084-fdc6-449a-1274-08d94afc056e X-MS-TrafficTypeDiagnostic: BN6PR02MB2482: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:126; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lDFArRUDLVkS0PnGGc0rYqSMK/L7ydy3Yn0RPzuGKpfBvcTsCIVwJJQaIDCH8f3ChpkSImZKybSqE04u2mowU9Cw8e+ejq6GTwgpJJLp/DofQdcABlr5SzrzpzSQE3/DgvTBuPt4eXSXTq+gXoSoL2DRWxkqlicX2QjmR5Sauf2ygF0YhHTDJpZGWgojr/l//7tsEcLjY4tf//opvyjwqmKOhkJmryiDfv+TnGb0ercCNZbIVubUmPDZtvJN7OOoGXpuTUzEZ9gnm8z1WMJxuaqr/lbyEIZlggTHOp2QtG5EoBVLf/G7+CGVa/uKWQtwOS3NCZMU4nbqJGBVdCYvv/L+3zJzEG25Av3kqJtd3cy7D9iTpyVOr3euakwftN0h6sHpibBij5TLPFrJJKhyW04oVUNiOG7vGY9uKOVMfExeKEoyjhatTYR5n4bbGHrOlco3KJ4acU5gIzsOhBa2xA81mFCKQJ+j1EsSBUUbRLAYf3lgDH2HfrmO6iBm4evyyVHzX0flD5c+6wRb2KINPmj14XaoFQBsRIj5ccRZJahqYe9UU1fdxj5MWHAA7gSOkqvi0OzkEfCsL/eXHpzN9iy+3akunB4gGSzQ3uxilqk/8xMDbTCvsljuetzQb3YjHnLXbTQruVbeneHTTvdX+23k+CgXi12Oq7wWZbayS6DO7ksioZXOTvn+T6W4entPntYXgSSDo0qwRSoYPbHrKQ== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(6916009)(186003)(2906002)(1076003)(36906005)(6266002)(8676002)(107886003)(42186006)(54906003)(426003)(8936002)(508600001)(44832011)(4326008)(70586007)(5660300002)(2616005)(336012)(316002)(70206006)(26005)(6666004)(7636003)(36860700001)(36756003)(82310400003)(356005)(47076005)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:27:33.2030 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d686a084-fdc6-449a-1274-08d94afc056e X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT054.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR02MB2482 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org group driver that manages life cycle of a bunch of leaf driver instances and bridges them with root. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/include/group.h | 25 +++ drivers/fpga/xrt/lib/group.c | 278 +++++++++++++++++++++++++++++++ 2 files changed, 303 insertions(+) create mode 100644 drivers/fpga/xrt/include/group.h create mode 100644 drivers/fpga/xrt/lib/group.c diff --git a/drivers/fpga/xrt/include/group.h b/drivers/fpga/xrt/include/group.h new file mode 100644 index 000000000000..09e9d03f53fe --- /dev/null +++ b/drivers/fpga/xrt/include/group.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#ifndef _XRT_GROUP_H_ +#define _XRT_GROUP_H_ + +#include "xleaf.h" + +/* + * Group driver leaf calls. + */ +enum xrt_group_leaf_cmd { + XRT_GROUP_GET_LEAF = XRT_XLEAF_CUSTOM_BASE, /* See comments in xleaf.h */ + XRT_GROUP_PUT_LEAF, + XRT_GROUP_INIT_CHILDREN, + XRT_GROUP_FINI_CHILDREN, + XRT_GROUP_TRIGGER_EVENT, +}; + +#endif /* _XRT_GROUP_H_ */ diff --git a/drivers/fpga/xrt/lib/group.c b/drivers/fpga/xrt/lib/group.c new file mode 100644 index 000000000000..b45f05449e0b --- /dev/null +++ b/drivers/fpga/xrt/lib/group.c @@ -0,0 +1,278 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx Alveo FPGA Group Driver + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#include +#include "xleaf.h" +#include "subdev_pool.h" +#include "group.h" +#include "metadata.h" +#include "lib-drv.h" + +#define XRT_GRP "xrt_group" + +struct xrt_group { + struct xrt_device *xdev; + struct xrt_subdev_pool leaves; + bool leaves_created; + struct mutex lock; /* lock for group */ +}; + +static int xrt_grp_root_cb(struct device *dev, void *parg, + enum xrt_root_cmd cmd, void *arg) +{ + int rc; + struct xrt_device *xdev = + container_of(dev, struct xrt_device, dev); + struct xrt_group *xg = (struct xrt_group *)parg; + + switch (cmd) { + case XRT_ROOT_GET_LEAF_HOLDERS: { + struct xrt_root_get_holders *holders = + (struct xrt_root_get_holders *)arg; + rc = xrt_subdev_pool_get_holders(&xg->leaves, + holders->xpigh_xdev, + holders->xpigh_holder_buf, + holders->xpigh_holder_buf_len); + break; + } + default: + /* Forward parent call to root. */ + rc = xrt_subdev_root_request(xdev, cmd, arg); + break; + } + + return rc; +} + +/* + * Cut subdev's dtb from group's dtb based on passed-in endpoint descriptor. + * Return the subdev's dtb through dtbp, if found. + */ +static int xrt_grp_cut_subdev_dtb(struct xrt_group *xg, struct xrt_dev_endpoints *eps, + char *grp_dtb, char **dtbp) +{ + int ret, i, ep_count = 0; + char *dtb = NULL; + + ret = xrt_md_create(DEV(xg->xdev), &dtb); + if (ret) + return ret; + + for (i = 0; eps->xse_names[i].ep_name || eps->xse_names[i].compat; i++) { + const char *ep_name = eps->xse_names[i].ep_name; + const char *compat = eps->xse_names[i].compat; + + if (!ep_name) + xrt_md_get_compatible_endpoint(DEV(xg->xdev), grp_dtb, compat, &ep_name); + if (!ep_name) + continue; + + ret = xrt_md_copy_endpoint(DEV(xg->xdev), dtb, grp_dtb, ep_name, compat, NULL); + if (ret) + continue; + xrt_md_del_endpoint(DEV(xg->xdev), grp_dtb, ep_name, compat); + ep_count++; + } + /* Found enough endpoints, return the subdev's dtb. */ + if (ep_count >= eps->xse_min_ep) { + *dtbp = dtb; + return 0; + } + + /* Cleanup - Restore all endpoints that has been deleted, if any. */ + if (ep_count > 0) { + xrt_md_copy_endpoint(DEV(xg->xdev), grp_dtb, dtb, + XRT_MD_NODE_ENDPOINTS, NULL, NULL); + } + vfree(dtb); + *dtbp = NULL; + return 0; +} + +static int xrt_grp_create_leaves(struct xrt_group *xg) +{ + struct xrt_subdev_platdata *pdata = DEV_PDATA(xg->xdev); + struct xrt_dev_endpoints *eps = NULL; + int ret = 0, failed = 0; + enum xrt_subdev_id did; + char *grp_dtb = NULL; + unsigned long mlen; + + if (!pdata) + return -EINVAL; + + mlen = xrt_md_size(DEV(xg->xdev), pdata->xsp_dtb); + if (mlen == XRT_MD_INVALID_LENGTH) { + xrt_err(xg->xdev, "invalid dtb, len %ld", mlen); + return -EINVAL; + } + + mutex_lock(&xg->lock); + + if (xg->leaves_created) { + /* + * This is expected since caller does not keep track of the state of the group + * and may, in some cases, still try to create leaves after it has already been + * created. This special error code will let the caller know what is going on. + */ + mutex_unlock(&xg->lock); + return -EEXIST; + } + + grp_dtb = vmalloc(mlen); + if (!grp_dtb) { + mutex_unlock(&xg->lock); + return -ENOMEM; + } + + /* Create all leaves based on dtb. */ + xrt_info(xg->xdev, "bringing up leaves..."); + memcpy(grp_dtb, pdata->xsp_dtb, mlen); + for (did = 0; did < XRT_SUBDEV_NUM; did++) { + eps = xrt_drv_get_endpoints(did); + while (eps && eps->xse_names) { + char *dtb = NULL; + + ret = xrt_grp_cut_subdev_dtb(xg, eps, grp_dtb, &dtb); + if (ret) { + failed++; + xrt_err(xg->xdev, "failed to cut subdev dtb for drv %s: %d", + xrt_drv_name(did), ret); + } + if (!dtb) { + /* + * No more dtb to cut or bad things happened for this instance, + * switch to the next one. + */ + eps++; + continue; + } + + /* Found a dtb for this instance, let's add it. */ + ret = xrt_subdev_pool_add(&xg->leaves, did, xrt_grp_root_cb, xg, dtb); + if (ret < 0) { + /* + * It is not a fatal error here. Some functionality is not usable + * due to this missing device, but the error can be handled + * when the functionality is used. + */ + failed++; + xrt_err(xg->xdev, "failed to add %s: %d", xrt_drv_name(did), ret); + } + vfree(dtb); + /* Continue searching for the same instance from grp_dtb. */ + } + } + + xg->leaves_created = true; + vfree(grp_dtb); + mutex_unlock(&xg->lock); + return failed == 0 ? 0 : -ECHILD; +} + +static void xrt_grp_remove_leaves(struct xrt_group *xg) +{ + mutex_lock(&xg->lock); + + if (!xg->leaves_created) { + mutex_unlock(&xg->lock); + return; + } + + xrt_info(xg->xdev, "tearing down leaves..."); + xrt_subdev_pool_fini(&xg->leaves); + xg->leaves_created = false; + + mutex_unlock(&xg->lock); +} + +static int xrt_grp_probe(struct xrt_device *xdev) +{ + struct xrt_group *xg; + + xrt_info(xdev, "probing..."); + + xg = devm_kzalloc(&xdev->dev, sizeof(*xg), GFP_KERNEL); + if (!xg) + return -ENOMEM; + + xg->xdev = xdev; + mutex_init(&xg->lock); + xrt_subdev_pool_init(DEV(xdev), &xg->leaves); + xrt_set_drvdata(xdev, xg); + + return 0; +} + +static void xrt_grp_remove(struct xrt_device *xdev) +{ + struct xrt_group *xg = xrt_get_drvdata(xdev); + + xrt_info(xdev, "leaving..."); + xrt_grp_remove_leaves(xg); +} + +static int xrt_grp_leaf_call(struct xrt_device *xdev, u32 cmd, void *arg) +{ + int rc = 0; + struct xrt_group *xg = xrt_get_drvdata(xdev); + + switch (cmd) { + case XRT_XLEAF_EVENT: + /* Simply forward to every child. */ + xrt_subdev_pool_handle_event(&xg->leaves, + (struct xrt_event *)arg); + break; + case XRT_GROUP_GET_LEAF: { + struct xrt_root_get_leaf *get_leaf = + (struct xrt_root_get_leaf *)arg; + + rc = xrt_subdev_pool_get(&xg->leaves, get_leaf->xpigl_match_cb, + get_leaf->xpigl_match_arg, + DEV(get_leaf->xpigl_caller_xdev), + &get_leaf->xpigl_tgt_xdev); + break; + } + case XRT_GROUP_PUT_LEAF: { + struct xrt_root_put_leaf *put_leaf = + (struct xrt_root_put_leaf *)arg; + + rc = xrt_subdev_pool_put(&xg->leaves, put_leaf->xpipl_tgt_xdev, + DEV(put_leaf->xpipl_caller_xdev)); + break; + } + case XRT_GROUP_INIT_CHILDREN: + rc = xrt_grp_create_leaves(xg); + break; + case XRT_GROUP_FINI_CHILDREN: + xrt_grp_remove_leaves(xg); + break; + case XRT_GROUP_TRIGGER_EVENT: + xrt_subdev_pool_trigger_event(&xg->leaves, (enum xrt_events)(uintptr_t)arg); + break; + default: + xrt_err(xdev, "unknown IOCTL cmd %d", cmd); + rc = -EINVAL; + break; + } + return rc; +} + +static struct xrt_driver xrt_group_driver = { + .driver = { + .name = XRT_GRP, + }, + .subdev_id = XRT_SUBDEV_GRP, + .probe = xrt_grp_probe, + .remove = xrt_grp_remove, + .leaf_call = xrt_grp_leaf_call, +}; + +XRT_LEAF_INIT_FINI_FUNC(group); From patchwork Mon Jul 19 21:26:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AEE0C6377E for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D4CC61002 for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385208AbhGSVQi (ORCPT ); Mon, 19 Jul 2021 17:16:38 -0400 Received: from mail-bn8nam12on2055.outbound.protection.outlook.com ([40.107.237.55]:58176 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388238AbhGSUrL (ORCPT ); Mon, 19 Jul 2021 16:47:11 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IAUoiqXqRThuy1qeFKNsIY+HoRGK/Cqg0Uiq69OOfg/n2fgRoC/qvgXd72gKJTLykZixYq3LXIoy+evoBzTvIJ4KSupe88WgRK9kOV7WQrCdS63fkWKCfOeOVOCpyGwDE0r6I54ob3znyr+juJ0LR9fcCKAyioQv22WV9y0k99fXZN/EdfsFZlmUZSzki2ObYHK2gfNVEfuxzG5QidNQMcQ5aodD9eHLnWCaENE0dP/X8I5uycnjXPG0kjxPn/N7B6kcHBVbJGNzFXSPRl3XhSisxfgGCpSnrXvwahVOjr2SQYYu5Y+apoOsYAlG4S2LB7KRApyF7Nogb3YhkQkJsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FJGnxQJe6FQRgxaRo8LXtiTIBNZmF318AW7uIhkSTbU=; b=AQgf9/jb74ONMxHEs+xP7wEO/YOS+VuTB9xzNvCwLqKWi/RjOuMaDU4o23BvbOPpHw4j62j4j50gj+V/OQy12O7H+374IAV9Xr2CxMS6NTZvfzRfWFEkAcTk4RdqgmVNLpMrccnwxbLvYzFVRqXPQDis6MRK11Up537OBDVAzPZk/5CFgb67g8urY0GHB+JvLM7S67jhfaoSDlrh6e8t9uzZc/GxpN7QPA2LtOladuvAYQn7nM90yRF7SWvN/DtUCSDng9Lgvno6H2Y2mCIbbBZTz/AeW9oEeWDpPTNPaR0bWxk24Ky/fUtGTFk4g3RCi24E3dflV0w2UaIBxiKOeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FJGnxQJe6FQRgxaRo8LXtiTIBNZmF318AW7uIhkSTbU=; b=h/i+burOVdhPs9kQfnx28NH67PklHUATRcL3fSGSZ16ilS/73yNtlcuHbgUYg6zob/c8PpJjVKcvlBNmSuW3IVdwLQNzgu/qmSm8wTtTVcaT6AwXA9MkZV+R8g4IxE81nna1EkaOck6JAhaqU9QPdNKVCyJ1QLoSJ5ko4Wd36oY= Received: from DM5PR16CA0043.namprd16.prod.outlook.com (2603:10b6:4:15::29) by DM5PR02MB3260.namprd02.prod.outlook.com (2603:10b6:4:6c::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.31; Mon, 19 Jul 2021 21:27:45 +0000 Received: from DM3NAM02FT056.eop-nam02.prod.protection.outlook.com (2603:10b6:4:15:cafe::19) by DM5PR16CA0043.outlook.office365.com (2603:10b6:4:15::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT056.mail.protection.outlook.com (10.13.4.177) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:27:44 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:27:44 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:27:44 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38276 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5ang-000FAq-69; Mon, 19 Jul 2021 14:27:44 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id BE7056020BD; Mon, 19 Jul 2021 14:26:31 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 06/14] fpga: xrt: char dev node helper functions Date: Mon, 19 Jul 2021 14:26:20 -0700 Message-ID: <20210719212628.134129-7-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7916b9fe-bd77-461c-5cfe-08d94afc0c75 X-MS-TrafficTypeDiagnostic: DM5PR02MB3260: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:67; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wWYcJ+zwyZha9uRLgfZa4a7LokSuSAZ7hh7mHBDM/+9PD99FHH0+T120IUtw2I8YFXC3RuTyFzkRZWf0HdPz2qA5znUzzTbNZibihqsJPVsSv+SaYfwUzQKj8X2WH9UzUYaYRqL3ZbnZon+zk1j/aGWs2Zf3tbxIfxgNxyjVahaDD6PdnPeUl2sOcOK4NVdja1k3tBOuz0JQgtBxrHVLSyfM8g6ex6KeZhbjTqVUJOegSNrz+ogn/JhQnCCugpXVnnOmivniJv7VKbp0IeLHqlVzvY8r0nLXg3UqaAwfRTJsFMvpvJfHxDlrQN/PNDD70wWCoF3dJPHgpJQF2fZOWCW01kxjx/TsKG/OQ9puyxItlviwZrCmhC+8seq9UuGh3pTLfcScOHTybdv89WlX5PBZvEa2NzzWFKe/S9lx/c2p8DtQXulNOOweQ71P04kCRZbYDVsnhQjFu+fXnRpJaULh+Yxo7jbwxaLf+Zq8Izk6LgzLWC3mo/Sba9euyovCBIJ2HadxMuMgNpMp1LxXLENO9n/fTm/EBgAYrt29icLGJtRgRO+hXan0hgDZlkMZL1zj8QfvswZzHlwpVAAJGDZtUkkokBPCLUNROB7FvFmDj4SxG2xv0KZuhn701k7XwZpuieJSjLzCooIHYvRtYLkW3efRykZZ3MkLlpxfQlOcToLq11Q9FkvXKT0U25ctu9RA4EwMHOGUXH9O3igrEg== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(26005)(82310400003)(47076005)(8936002)(36756003)(83380400001)(5660300002)(2616005)(508600001)(2906002)(6666004)(44832011)(70586007)(316002)(7636003)(70206006)(8676002)(186003)(54906003)(356005)(426003)(1076003)(36860700001)(4326008)(42186006)(36906005)(107886003)(6266002)(6916009)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:27:44.9958 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7916b9fe-bd77-461c-5cfe-08d94afc0c75 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT056.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR02MB3260 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Helper functions for char device node creation / removal for xrt drivers. This is part of xrt driver infrastructure. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/lib/cdev.c | 209 ++++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 drivers/fpga/xrt/lib/cdev.c diff --git a/drivers/fpga/xrt/lib/cdev.c b/drivers/fpga/xrt/lib/cdev.c new file mode 100644 index 000000000000..3c20adac8c03 --- /dev/null +++ b/drivers/fpga/xrt/lib/cdev.c @@ -0,0 +1,209 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx Alveo FPGA device node helper functions. + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#include "xleaf.h" +#include "lib-drv.h" + +#define XRT_CDEV_DIR "xrt" +#define INODE2PDATA(inode) \ + container_of((inode)->i_cdev, struct xrt_subdev_platdata, xsp_cdev) +#define INODE2PDEV(inode) \ + to_xrt_dev(kobj_to_dev((inode)->i_cdev->kobj.parent)) +#define CDEV_NAME(sysdev) (strchr((sysdev)->kobj.name, '!') + 1) + +/* Allow it to be accessed from cdev. */ +static void xleaf_devnode_allowed(struct xrt_device *xdev) +{ + struct xrt_subdev_platdata *pdata = DEV_PDATA(xdev); + + /* Allow new opens. */ + mutex_lock(&pdata->xsp_devnode_lock); + pdata->xsp_devnode_online = true; + mutex_unlock(&pdata->xsp_devnode_lock); +} + +/* Turn off access from cdev and wait for all existing user to go away. */ +static void xleaf_devnode_disallowed(struct xrt_device *xdev) +{ + struct xrt_subdev_platdata *pdata = DEV_PDATA(xdev); + + mutex_lock(&pdata->xsp_devnode_lock); + + /* Prevent new opens. */ + pdata->xsp_devnode_online = false; + /* Wait for existing user to close. */ + while (pdata->xsp_devnode_ref) { + mutex_unlock(&pdata->xsp_devnode_lock); + wait_for_completion(&pdata->xsp_devnode_comp); + mutex_lock(&pdata->xsp_devnode_lock); + } + + mutex_unlock(&pdata->xsp_devnode_lock); +} + +static struct xrt_device * +__xleaf_devnode_open(struct inode *inode, bool excl) +{ + struct xrt_subdev_platdata *pdata = INODE2PDATA(inode); + struct xrt_device *xdev = INODE2PDEV(inode); + bool opened = false; + + mutex_lock(&pdata->xsp_devnode_lock); + + if (pdata->xsp_devnode_online) { + if (excl && pdata->xsp_devnode_ref) { + xrt_err(xdev, "%s has already been opened exclusively", + CDEV_NAME(pdata->xsp_sysdev)); + } else if (!excl && pdata->xsp_devnode_excl) { + xrt_err(xdev, "%s has been opened exclusively", + CDEV_NAME(pdata->xsp_sysdev)); + } else { + pdata->xsp_devnode_ref++; + pdata->xsp_devnode_excl = excl; + opened = true; + xrt_info(xdev, "opened %s, ref=%d", + CDEV_NAME(pdata->xsp_sysdev), + pdata->xsp_devnode_ref); + } + } else { + xrt_err(xdev, "%s is offline", CDEV_NAME(pdata->xsp_sysdev)); + } + + mutex_unlock(&pdata->xsp_devnode_lock); + + xdev = opened ? xdev : NULL; + return xdev; +} + +struct xrt_device * +xleaf_devnode_open_excl(struct inode *inode) +{ + return __xleaf_devnode_open(inode, true); +} + +struct xrt_device * +xleaf_devnode_open(struct inode *inode) +{ + return __xleaf_devnode_open(inode, false); +} +EXPORT_SYMBOL_GPL(xleaf_devnode_open); + +void xleaf_devnode_close(struct inode *inode) +{ + struct xrt_subdev_platdata *pdata = INODE2PDATA(inode); + struct xrt_device *xdev = INODE2PDEV(inode); + bool notify = false; + + mutex_lock(&pdata->xsp_devnode_lock); + + WARN_ON(pdata->xsp_devnode_ref == 0); + pdata->xsp_devnode_ref--; + if (pdata->xsp_devnode_ref == 0) { + pdata->xsp_devnode_excl = false; + notify = true; + } + if (notify) + xrt_info(xdev, "closed %s", CDEV_NAME(pdata->xsp_sysdev)); + else + xrt_info(xdev, "closed %s, notifying waiter", CDEV_NAME(pdata->xsp_sysdev)); + + mutex_unlock(&pdata->xsp_devnode_lock); + + if (notify) + complete(&pdata->xsp_devnode_comp); +} +EXPORT_SYMBOL_GPL(xleaf_devnode_close); + +static inline enum xrt_dev_file_mode +devnode_mode(struct xrt_device *xdev) +{ + return DEV_FILE_OPS(xdev)->xsf_mode; +} + +int xleaf_devnode_create(struct xrt_device *xdev, const char *file_name, + const char *inst_name) +{ + struct xrt_subdev_platdata *pdata = DEV_PDATA(xdev); + struct xrt_dev_file_ops *fops = DEV_FILE_OPS(xdev); + struct cdev *cdevp; + struct device *sysdev; + int ret = 0; + char fname[256]; + + mutex_init(&pdata->xsp_devnode_lock); + init_completion(&pdata->xsp_devnode_comp); + + cdevp = &DEV_PDATA(xdev)->xsp_cdev; + cdev_init(cdevp, &fops->xsf_ops); + cdevp->owner = fops->xsf_ops.owner; + cdevp->dev = MKDEV(MAJOR(fops->xsf_dev_t), xdev->instance); + + /* + * Set xdev as parent of cdev so that when xdev (and its platform + * data) will not be freed when cdev is not freed. + */ + cdev_set_parent(cdevp, &DEV(xdev)->kobj); + + ret = cdev_add(cdevp, cdevp->dev, 1); + if (ret) { + xrt_err(xdev, "failed to add cdev: %d", ret); + goto failed; + } + if (!file_name) + file_name = xdev->name; + if (!inst_name) { + if (devnode_mode(xdev) == XRT_DEV_FILE_MULTI_INST) { + snprintf(fname, sizeof(fname), "%s/%s/%s.%u", + XRT_CDEV_DIR, DEV_PDATA(xdev)->xsp_root_name, + file_name, xdev->instance); + } else { + snprintf(fname, sizeof(fname), "%s/%s/%s", + XRT_CDEV_DIR, DEV_PDATA(xdev)->xsp_root_name, + file_name); + } + } else { + snprintf(fname, sizeof(fname), "%s/%s/%s.%s", XRT_CDEV_DIR, + DEV_PDATA(xdev)->xsp_root_name, file_name, inst_name); + } + sysdev = device_create(xrt_class, NULL, cdevp->dev, NULL, "%s", fname); + if (IS_ERR(sysdev)) { + ret = PTR_ERR(sysdev); + xrt_err(xdev, "failed to create device node: %d", ret); + goto failed_cdev_add; + } + pdata->xsp_sysdev = sysdev; + + xleaf_devnode_allowed(xdev); + + xrt_info(xdev, "created (%d, %d): /dev/%s", + MAJOR(cdevp->dev), xdev->instance, fname); + return 0; + +failed_cdev_add: + cdev_del(cdevp); +failed: + cdevp->owner = NULL; + return ret; +} + +void xleaf_devnode_destroy(struct xrt_device *xdev) +{ + struct xrt_subdev_platdata *pdata = DEV_PDATA(xdev); + struct cdev *cdevp = &pdata->xsp_cdev; + dev_t dev = cdevp->dev; + + xleaf_devnode_disallowed(xdev); + + xrt_info(xdev, "removed (%d, %d): /dev/%s/%s", MAJOR(dev), MINOR(dev), + XRT_CDEV_DIR, CDEV_NAME(pdata->xsp_sysdev)); + device_destroy(xrt_class, cdevp->dev); + pdata->xsp_sysdev = NULL; + cdev_del(cdevp); +} From patchwork Mon Jul 19 21:26:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480110 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 783F0C63798 for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 58655611ED for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1389145AbhGSVQo (ORCPT ); Mon, 19 Jul 2021 17:16:44 -0400 Received: from mail-bn8nam11on2073.outbound.protection.outlook.com ([40.107.236.73]:29152 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388254AbhGSUrk (ORCPT ); Mon, 19 Jul 2021 16:47:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HDfIFj9DjEjEkPtphd2pOVDx/5bHx4m+oyfSj4NaHlfO3QFVyoKtIMijK4DjOflJvKSkfn8SI2e6pq9ZkqL934tT9R+H4rkcCFZVY+dRIMYPvRTgV843Ln05lSCOGBhQuz+s9Fis/cn26ZChGp5/XSszSEzeMPlSkmA0lkModYRZgVW2F35XBNtshaaTbSyOYytqmJLO1/WngrOaoaB5X91okrofKyTKTH3snAH1Ows0BSqJzJf5SoYMAjzjT5LVDiWkTuGL6iR3sLwUPxeUtMgA5vtA9wOoi04VqrbVveovYk5puBpDAvH3biMGigJALafwD7uIAFA9fp0k5PNHxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t4phI/p2XdBLZgLW4Q+NuAnAlvtjXs4aJPXk/LWbEFQ=; b=Oidnj9DRcDGEKu3AKNO7WHOLYpid71UzpbZTzRJRmVYWJ/aOSOLuagZGTm4UJbWbNRiOyfIRiCrzaCs4OQVMwdl8K8RpxzVcO0/hXPPn5vwnnLmElVFHr7raS3j2HQBPj8A4GOIY1c6obfzlnb2LKjiJvExd8GSr1oNdmeOkL/oqtQLTeQYGEPHj9e8P5TiWEeRaZyxgEmi93HALlXxdosgdLkKkhRP+vreCFhr/cUqfBH08pPaZe8eFW2JhT3PVko9FcC6AdN2pLdpBvffKHC3p01zfDBi9qlrceUw0QOXWvBzhdRr+xUoDtOWWK66DeaLkIs/7+Vg7tC24qp5gnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t4phI/p2XdBLZgLW4Q+NuAnAlvtjXs4aJPXk/LWbEFQ=; b=YpJVbHc0kSqywQFRQwLG1Dh1cxFqOWYtAbYl8IBDSFlQwnNrSrlf8MYbqYPMovxjtUyGJi+VQqeWey/FjzISWcIXsVTspwy+In92CVe+uc+5eSzil2mnCdq6EVeIfH7VVLhKn7r+EhYtX7gQQE1IdCHMd8HK150cOE5bpe4PmOY= Received: from DM3PR08CA0003.namprd08.prod.outlook.com (2603:10b6:0:52::13) by DM6PR02MB5034.namprd02.prod.outlook.com (2603:10b6:5:4d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.29; Mon, 19 Jul 2021 21:28:11 +0000 Received: from DM3NAM02FT026.eop-nam02.prod.protection.outlook.com (2603:10b6:0:52:cafe::53) by DM3PR08CA0003.outlook.office365.com (2603:10b6:0:52::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22 via Frontend Transport; Mon, 19 Jul 2021 21:28:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT026.mail.protection.outlook.com (10.13.5.129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:28:10 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:28:08 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:28:08 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38280 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5ao4-000FDr-8t; Mon, 19 Jul 2021 14:28:08 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id EDB606020C1; Mon, 19 Jul 2021 14:26:31 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 08/14] fpga: xrt: driver infrastructure Date: Mon, 19 Jul 2021 14:26:22 -0700 Message-ID: <20210719212628.134129-9-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fe6a75cb-dac4-4b01-269d-08d94afc1bee X-MS-TrafficTypeDiagnostic: DM6PR02MB5034: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:525; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ro87S/wiL4LqdhVAQlzUIFyXk1HmSh5xrZ+tbqRB9UuYLHFPksNieqZy0Buskub1uj2LTnTcGBabBgBn6WslDb+DSMQKJauWxVA0T4kEUbhQycRCB0ACDhZbV2W2UyOxxC7XIjrYkWIVLvW3DY8994Bs5LRgmnBsjup1ATXW774F/f+YjKVDI0Q35AES72vFoWXxXUCSLXpmHBet/7uNI/P+eKWL0t8prY/J8rVDKgekYVkdkD87Feq0eX6xO7qW5P0HIU9aiMW1HINa1tAZMqd7qvo48eSwxUiaGwbWIMqODQibmavhzPpMNaY8hB+CLJNVTKFfxfOcUtVXtsQxAC3oHCmkalYHP/YyQbnyHtB94P7/SKRK4028JfQ9Vabe648cVNTFTGjnARWysbY3JcowNuPUEBlHjKdozeTJLzAtNlnVvG86Y2VHFc93iH/tMGnIKvnvlyqsAdd9Rd4MDwPyTpE/JOCJ+LO0jnEcOW3/zDK8lt2ZkR93RKCW0pqzANxX9xWAeGcMkLS8H30cGfLGuU2KoY3+67EMT9fyNFxPk1noT9oUgyaz2ZEbQuOjEzRl7uYujy6+Rjr+8cZ5f0NBK0e2aX2mFCBIV8ttLlsyqc6FQdwtffFTQ19l/Qa05ZbFhkpkXgZbjFNKXbjM03o0uqO5io34M3o8nwL5dvuLJRJAooEJji9perEyX2gxCbacUfe+6KzzHaPzjiSlNw== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(376002)(346002)(136003)(396003)(39860400002)(46966006)(36840700001)(70206006)(82310400003)(36860700001)(107886003)(6666004)(1076003)(82740400003)(26005)(70586007)(186003)(6266002)(2906002)(30864003)(83380400001)(36756003)(54906003)(47076005)(316002)(356005)(5660300002)(7636003)(4326008)(36906005)(44832011)(6916009)(42186006)(8676002)(478600001)(8936002)(426003)(336012)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:28:10.9532 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fe6a75cb-dac4-4b01-269d-08d94afc1bee X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT026.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR02MB5034 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Infrastructure code providing APIs for managing leaf driver instance groups, facilitating inter-leaf driver calls and root calls. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/lib/subdev.c | 859 ++++++++++++++++++++++++++++++++++ 1 file changed, 859 insertions(+) create mode 100644 drivers/fpga/xrt/lib/subdev.c diff --git a/drivers/fpga/xrt/lib/subdev.c b/drivers/fpga/xrt/lib/subdev.c new file mode 100644 index 000000000000..350f67d5eb1e --- /dev/null +++ b/drivers/fpga/xrt/lib/subdev.c @@ -0,0 +1,859 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#include +#include +#include "xleaf.h" +#include "subdev_pool.h" +#include "lib-drv.h" +#include "metadata.h" + +extern struct bus_type xrt_bus_type; + +#define IS_ROOT_DEV(dev) ((dev)->bus != &xrt_bus_type) +#define XRT_HOLDER_BUF_SZ 1024 + +static inline struct device *find_root(struct xrt_device *xdev) +{ + struct device *d = DEV(xdev); + + while (!IS_ROOT_DEV(d)) + d = d->parent; + return d; +} + +/* + * It represents a holder of a subdev. One holder can repeatedly hold a subdev + * as long as there is a unhold corresponding to a hold. + */ +struct xrt_subdev_holder { + struct list_head xsh_holder_list; + struct device *xsh_holder; + int xsh_count; + struct kref xsh_kref; +}; + +/* + * It represents a specific instance of platform driver for a subdev, which + * provides services to its clients (another subdev driver or root driver). + */ +struct xrt_subdev { + struct list_head xs_dev_list; + struct list_head xs_holder_list; + enum xrt_subdev_id xs_id; /* type of subdev */ + struct xrt_device *xs_xdev; + struct completion xs_holder_comp; +}; + +static struct xrt_subdev *xrt_subdev_alloc(void) +{ + struct xrt_subdev *sdev = kzalloc(sizeof(*sdev), GFP_KERNEL); + + if (!sdev) + return NULL; + + INIT_LIST_HEAD(&sdev->xs_dev_list); + INIT_LIST_HEAD(&sdev->xs_holder_list); + init_completion(&sdev->xs_holder_comp); + return sdev; +} + +int xrt_subdev_root_request(struct xrt_device *self, u32 cmd, void *arg) +{ + struct device *dev = DEV(self); + struct xrt_subdev_platdata *pdata = DEV_PDATA(self); + + if (!pdata->xsp_root_cb) { + dev_err(dev, "invalid root callback"); + return -EINVAL; + } + return (*pdata->xsp_root_cb)(dev->parent, pdata->xsp_root_cb_arg, cmd, arg); +} + +/* + * Subdev common sysfs nodes. + */ +static ssize_t holders_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + ssize_t len; + struct xrt_device *xdev = to_xrt_dev(dev); + struct xrt_root_get_holders holders = { xdev, buf, XRT_HOLDER_BUF_SZ }; + + len = xrt_subdev_root_request(xdev, XRT_ROOT_GET_LEAF_HOLDERS, &holders); + if (len >= holders.xpigh_holder_buf_len) + return len; + buf[len] = '\n'; + return len + 1; +} +static DEVICE_ATTR_RO(holders); + +static struct attribute *xrt_subdev_attrs[] = { + &dev_attr_holders.attr, + NULL, +}; + +static ssize_t metadata_output(struct file *filp, struct kobject *kobj, + struct bin_attribute *attr, char *buf, loff_t off, size_t count) +{ + struct device *dev = kobj_to_dev(kobj); + struct xrt_device *xdev = to_xrt_dev(dev); + struct xrt_subdev_platdata *pdata = DEV_PDATA(xdev); + unsigned char *blob; + unsigned long size; + ssize_t ret = 0; + + blob = pdata->xsp_dtb; + size = xrt_md_size(dev, blob); + if (size == XRT_MD_INVALID_LENGTH) { + ret = -EINVAL; + goto failed; + } + + if (off >= size) { + dev_dbg(dev, "offset (%lld) beyond total size: %ld\n", off, size); + goto failed; + } + + if (off + count > size) { + dev_dbg(dev, "count (%ld) beyond left bytes: %lld\n", + (unsigned long)count, size - off); + count = size - off; + } + memcpy(buf, blob + off, count); + + ret = count; +failed: + return ret; +} + +static struct bin_attribute meta_data_attr = { + .attr = { + .name = "metadata", + .mode = 0400 + }, + .read = metadata_output, + .size = 0 +}; + +static struct bin_attribute *xrt_subdev_bin_attrs[] = { + &meta_data_attr, + NULL, +}; + +static const struct attribute_group xrt_subdev_attrgroup = { + .attrs = xrt_subdev_attrs, + .bin_attrs = xrt_subdev_bin_attrs, +}; + +/* + * Given the device metadata, parse it to get IO ranges and construct + * resource array. + */ +static int +xrt_subdev_getres(struct device *parent, enum xrt_subdev_id id, + char *dtb, struct resource **res, int *res_num) +{ + struct xrt_subdev_platdata *pdata; + struct resource *pci_res = NULL; + const __be64 *bar_range; + const __be32 *bar_idx; + char *ep_name = NULL, *compat = NULL; + uint bar; + int count1 = 0, count2 = 0, ret; + + if (!dtb) + return -EINVAL; + + pdata = DEV_PDATA(to_xrt_dev(parent)); + + /* go through metadata and count endpoints in it */ + xrt_md_get_next_endpoint(parent, dtb, NULL, NULL, &ep_name, &compat); + while (ep_name) { + ret = xrt_md_get_prop(parent, dtb, ep_name, compat, + XRT_MD_PROP_IO_OFFSET, (const void **)&bar_range, NULL); + if (!ret) + count1++; + xrt_md_get_next_endpoint(parent, dtb, ep_name, compat, &ep_name, &compat); + } + if (!count1) + return 0; + + /* allocate resource array for all endpoints been found in metadata */ + *res = vzalloc(sizeof(**res) * count1); + + /* go through all endpoints again and get IO range for each endpoint */ + ep_name = NULL; + xrt_md_get_next_endpoint(parent, dtb, NULL, NULL, &ep_name, &compat); + while (ep_name) { + ret = xrt_md_get_prop(parent, dtb, ep_name, compat, + XRT_MD_PROP_IO_OFFSET, (const void **)&bar_range, NULL); + if (ret) + continue; + xrt_md_get_prop(parent, dtb, ep_name, compat, + XRT_MD_PROP_BAR_IDX, (const void **)&bar_idx, NULL); + bar = bar_idx ? be32_to_cpu(*bar_idx) : 0; + xleaf_get_root_res(to_xrt_dev(parent), bar, &pci_res); + if (!pci_res) { + dev_err(parent, "Invalid bar defined %d", bar); + ret = -EINVAL; + goto failed; + } + (*res)[count2].start = pci_res->start + be64_to_cpu(bar_range[0]); + (*res)[count2].end = pci_res->start + be64_to_cpu(bar_range[0]) + + be64_to_cpu(bar_range[1]) - 1; + (*res)[count2].flags = IORESOURCE_MEM; + /* check if there is conflicted resource */ + ret = request_resource(pci_res, *res + count2); + if (ret) { + dev_err(parent, "Conflict resource %pR\n", *res + count2); + goto failed; + } + release_resource(*res + count2); + + (*res)[count2].parent = pci_res; + + xrt_md_find_endpoint(parent, pdata->xsp_dtb, ep_name, + compat, &(*res)[count2].name); + + count2++; + xrt_md_get_next_endpoint(parent, dtb, ep_name, compat, &ep_name, &compat); + } + + WARN_ON(count1 != count2); + *res_num = count2; + + return 0; + +failed: + vfree(*res); + *res_num = 0; + *res = NULL; + return ret; +} + +static inline enum xrt_dev_file_mode +xleaf_devnode_mode(struct xrt_device *xdev) +{ + return DEV_FILE_OPS(xdev)->xsf_mode; +} + +static bool xrt_subdev_cdev_auto_creation(struct xrt_device *xdev) +{ + enum xrt_dev_file_mode mode = xleaf_devnode_mode(xdev); + + if (!xleaf_devnode_enabled(xdev)) + return false; + + return (mode == XRT_DEV_FILE_DEFAULT || mode == XRT_DEV_FILE_MULTI_INST); +} + +static struct xrt_subdev * +xrt_subdev_create(struct device *parent, enum xrt_subdev_id id, + xrt_subdev_root_cb_t pcb, void *pcb_arg, char *dtb) +{ + struct xrt_subdev_platdata *pdata = NULL; + struct xrt_subdev *sdev = NULL; + struct xrt_device *xdev = NULL; + struct resource *res = NULL; + unsigned long dtb_len = 0; + bool dtb_alloced = false; + int res_num = 0; + size_t pdata_sz; + int ret; + + sdev = xrt_subdev_alloc(); + if (!sdev) { + dev_err(parent, "failed to alloc subdev for ID %d", id); + return NULL; + } + sdev->xs_id = id; + + if (!dtb) { + ret = xrt_md_create(parent, &dtb); + if (ret) { + dev_err(parent, "can't create empty dtb: %d", ret); + goto fail; + } + dtb_alloced = true; + } + xrt_md_pack(parent, dtb); + dtb_len = xrt_md_size(parent, dtb); + if (dtb_len == XRT_MD_INVALID_LENGTH) { + dev_err(parent, "invalid metadata len %ld", dtb_len); + goto fail1; + } + pdata_sz = sizeof(struct xrt_subdev_platdata) + dtb_len; + + /* Prepare platform data passed to subdev. */ + pdata = vzalloc(pdata_sz); + if (!pdata) + goto fail1; + + pdata->xsp_root_cb = pcb; + pdata->xsp_root_cb_arg = pcb_arg; + memcpy(pdata->xsp_dtb, dtb, dtb_len); + if (id == XRT_SUBDEV_GRP) { + /* Group can only be created by root driver. */ + pdata->xsp_root_name = dev_name(parent); + } else { + struct xrt_device *grp = to_xrt_dev(parent); + + /* Leaf can only be created by group driver. */ + WARN_ON(to_xrt_drv(parent->driver)->subdev_id != XRT_SUBDEV_GRP); + pdata->xsp_root_name = DEV_PDATA(grp)->xsp_root_name; + } + + /* Create subdev. */ + if (id != XRT_SUBDEV_GRP) { + int rc = xrt_subdev_getres(parent, id, dtb, &res, &res_num); + + if (rc) { + dev_err(parent, "failed to get resource for %s: %d", + xrt_drv_name(id), rc); + goto fail2; + } + } + xdev = xrt_device_register(parent, id, res, res_num, pdata, pdata_sz); + vfree(res); + if (!xdev) { + dev_err(parent, "failed to create subdev for %s", xrt_drv_name(id)); + goto fail2; + } + sdev->xs_xdev = xdev; + + if (device_attach(DEV(xdev)) != 1) { + xrt_err(xdev, "failed to attach"); + goto fail3; + } + + if (sysfs_create_group(&DEV(xdev)->kobj, &xrt_subdev_attrgroup)) + xrt_err(xdev, "failed to create sysfs group"); + + /* + * Create sysfs sym link under root for leaves + * under random groups for easy access to them. + */ + if (id != XRT_SUBDEV_GRP) { + if (sysfs_create_link(&find_root(xdev)->kobj, + &DEV(xdev)->kobj, dev_name(DEV(xdev)))) { + xrt_err(xdev, "failed to create sysfs link"); + } + } + + /* All done, ready to handle req thru cdev. */ + if (xrt_subdev_cdev_auto_creation(xdev)) + xleaf_devnode_create(xdev, DEV_FILE_OPS(xdev)->xsf_dev_name, NULL); + + vfree(pdata); + return sdev; + +fail3: + xrt_device_unregister(sdev->xs_xdev); +fail2: + vfree(pdata); +fail1: + if (dtb_alloced) + vfree(dtb); +fail: + kfree(sdev); + return NULL; +} + +static void xrt_subdev_destroy(struct xrt_subdev *sdev) +{ + struct xrt_device *xdev = sdev->xs_xdev; + struct device *dev = DEV(xdev); + + /* Take down the device node */ + if (xrt_subdev_cdev_auto_creation(xdev)) + xleaf_devnode_destroy(xdev); + if (sdev->xs_id != XRT_SUBDEV_GRP) + sysfs_remove_link(&find_root(xdev)->kobj, dev_name(dev)); + sysfs_remove_group(&dev->kobj, &xrt_subdev_attrgroup); + xrt_device_unregister(xdev); + kfree(sdev); +} + +struct xrt_device * +xleaf_get_leaf(struct xrt_device *xdev, xrt_subdev_match_t match_cb, void *match_arg) +{ + int rc; + struct xrt_root_get_leaf get_leaf = { + xdev, match_cb, match_arg, }; + + rc = xrt_subdev_root_request(xdev, XRT_ROOT_GET_LEAF, &get_leaf); + if (rc) + return NULL; + return get_leaf.xpigl_tgt_xdev; +} +EXPORT_SYMBOL_GPL(xleaf_get_leaf); + +bool xleaf_has_endpoint(struct xrt_device *xdev, const char *endpoint_name) +{ + struct resource *res; + int i = 0; + + do { + res = xrt_get_resource(xdev, IORESOURCE_MEM, i); + if (res && !strncmp(res->name, endpoint_name, strlen(res->name) + 1)) + return true; + ++i; + } while (res); + + return false; +} +EXPORT_SYMBOL_GPL(xleaf_has_endpoint); + +int xleaf_put_leaf(struct xrt_device *xdev, struct xrt_device *leaf) +{ + struct xrt_root_put_leaf put_leaf = { xdev, leaf }; + + return xrt_subdev_root_request(xdev, XRT_ROOT_PUT_LEAF, &put_leaf); +} +EXPORT_SYMBOL_GPL(xleaf_put_leaf); + +int xleaf_create_group(struct xrt_device *xdev, char *dtb) +{ + return xrt_subdev_root_request(xdev, XRT_ROOT_CREATE_GROUP, dtb); +} +EXPORT_SYMBOL_GPL(xleaf_create_group); + +int xleaf_destroy_group(struct xrt_device *xdev, int instance) +{ + return xrt_subdev_root_request(xdev, XRT_ROOT_REMOVE_GROUP, (void *)(uintptr_t)instance); +} +EXPORT_SYMBOL_GPL(xleaf_destroy_group); + +int xleaf_wait_for_group_bringup(struct xrt_device *xdev) +{ + return xrt_subdev_root_request(xdev, XRT_ROOT_WAIT_GROUP_BRINGUP, NULL); +} +EXPORT_SYMBOL_GPL(xleaf_wait_for_group_bringup); + +static ssize_t +xrt_subdev_get_holders(struct xrt_subdev *sdev, char *buf, size_t len) +{ + const struct list_head *ptr; + struct xrt_subdev_holder *h; + ssize_t n = 0; + + list_for_each(ptr, &sdev->xs_holder_list) { + h = list_entry(ptr, struct xrt_subdev_holder, xsh_holder_list); + n += snprintf(buf + n, len - n, "%s:%d ", + dev_name(h->xsh_holder), kref_read(&h->xsh_kref)); + /* Truncation is fine here. Buffer content is only for debugging. */ + if (n >= (len - 1)) + break; + } + return n; +} + +void xrt_subdev_pool_init(struct device *dev, struct xrt_subdev_pool *spool) +{ + INIT_LIST_HEAD(&spool->xsp_dev_list); + spool->xsp_owner = dev; + mutex_init(&spool->xsp_lock); + spool->xsp_closing = false; +} + +static void xrt_subdev_free_holder(struct xrt_subdev_holder *holder) +{ + list_del(&holder->xsh_holder_list); + vfree(holder); +} + +static void xrt_subdev_pool_wait_for_holders(struct xrt_subdev_pool *spool, struct xrt_subdev *sdev) +{ + const struct list_head *ptr, *next; + char holders[128]; + struct xrt_subdev_holder *holder; + struct mutex *lk = &spool->xsp_lock; + + while (!list_empty(&sdev->xs_holder_list)) { + int rc; + + /* It's most likely a bug if we ever enters this loop. */ + xrt_subdev_get_holders(sdev, holders, sizeof(holders)); + xrt_err(sdev->xs_xdev, "awaits holders: %s", holders); + mutex_unlock(lk); + rc = wait_for_completion_killable(&sdev->xs_holder_comp); + mutex_lock(lk); + if (rc == -ERESTARTSYS) { + xrt_err(sdev->xs_xdev, "give up on waiting for holders, clean up now"); + list_for_each_safe(ptr, next, &sdev->xs_holder_list) { + holder = list_entry(ptr, struct xrt_subdev_holder, xsh_holder_list); + xrt_subdev_free_holder(holder); + } + } + } +} + +void xrt_subdev_pool_fini(struct xrt_subdev_pool *spool) +{ + struct list_head *dl = &spool->xsp_dev_list; + struct mutex *lk = &spool->xsp_lock; + + mutex_lock(lk); + if (spool->xsp_closing) { + mutex_unlock(lk); + return; + } + spool->xsp_closing = true; + mutex_unlock(lk); + + /* Remove subdev in the reverse order of added. */ + while (!list_empty(dl)) { + struct xrt_subdev *sdev = list_first_entry(dl, struct xrt_subdev, xs_dev_list); + + xrt_subdev_pool_wait_for_holders(spool, sdev); + list_del(&sdev->xs_dev_list); + xrt_subdev_destroy(sdev); + } +} + +static struct xrt_subdev_holder *xrt_subdev_find_holder(struct xrt_subdev *sdev, + struct device *holder_dev) +{ + struct list_head *hl = &sdev->xs_holder_list; + struct xrt_subdev_holder *holder; + const struct list_head *ptr; + + list_for_each(ptr, hl) { + holder = list_entry(ptr, struct xrt_subdev_holder, xsh_holder_list); + if (holder->xsh_holder == holder_dev) + return holder; + } + return NULL; +} + +static int xrt_subdev_hold(struct xrt_subdev *sdev, struct device *holder_dev) +{ + struct xrt_subdev_holder *holder = xrt_subdev_find_holder(sdev, holder_dev); + struct list_head *hl = &sdev->xs_holder_list; + + if (!holder) { + holder = vzalloc(sizeof(*holder)); + if (!holder) + return -ENOMEM; + holder->xsh_holder = holder_dev; + kref_init(&holder->xsh_kref); + list_add_tail(&holder->xsh_holder_list, hl); + } else { + kref_get(&holder->xsh_kref); + } + + return 0; +} + +static void xrt_subdev_free_holder_kref(struct kref *kref) +{ + struct xrt_subdev_holder *holder = container_of(kref, struct xrt_subdev_holder, xsh_kref); + + xrt_subdev_free_holder(holder); +} + +static int +xrt_subdev_release(struct xrt_subdev *sdev, struct device *holder_dev) +{ + struct xrt_subdev_holder *holder = xrt_subdev_find_holder(sdev, holder_dev); + struct list_head *hl = &sdev->xs_holder_list; + + if (!holder) { + dev_err(holder_dev, "can't release, %s did not hold %s", + dev_name(holder_dev), dev_name(DEV(sdev->xs_xdev))); + return -EINVAL; + } + kref_put(&holder->xsh_kref, xrt_subdev_free_holder_kref); + + /* kref_put above may remove holder from list. */ + if (list_empty(hl)) + complete(&sdev->xs_holder_comp); + return 0; +} + +int xrt_subdev_pool_add(struct xrt_subdev_pool *spool, enum xrt_subdev_id id, + xrt_subdev_root_cb_t pcb, void *pcb_arg, char *dtb) +{ + struct mutex *lk = &spool->xsp_lock; + struct list_head *dl = &spool->xsp_dev_list; + struct xrt_subdev *sdev; + int ret = 0; + + sdev = xrt_subdev_create(spool->xsp_owner, id, pcb, pcb_arg, dtb); + if (sdev) { + mutex_lock(lk); + if (spool->xsp_closing) { + /* No new subdev when pool is going away. */ + xrt_err(sdev->xs_xdev, "pool is closing"); + ret = -ENODEV; + } else { + list_add(&sdev->xs_dev_list, dl); + } + mutex_unlock(lk); + if (ret) + xrt_subdev_destroy(sdev); + } else { + ret = -EINVAL; + } + + ret = ret ? ret : sdev->xs_xdev->instance; + return ret; +} + +int xrt_subdev_pool_del(struct xrt_subdev_pool *spool, enum xrt_subdev_id id, int instance) +{ + const struct list_head *ptr; + struct mutex *lk = &spool->xsp_lock; + struct list_head *dl = &spool->xsp_dev_list; + struct xrt_subdev *sdev; + int ret = -ENOENT; + + mutex_lock(lk); + if (spool->xsp_closing) { + /* Pool is going away, all subdevs will be gone. */ + mutex_unlock(lk); + return 0; + } + list_for_each(ptr, dl) { + sdev = list_entry(ptr, struct xrt_subdev, xs_dev_list); + if (sdev->xs_id != id || sdev->xs_xdev->instance != instance) + continue; + xrt_subdev_pool_wait_for_holders(spool, sdev); + list_del(&sdev->xs_dev_list); + ret = 0; + break; + } + mutex_unlock(lk); + if (ret) + return ret; + + xrt_subdev_destroy(sdev); + return 0; +} + +static int xrt_subdev_pool_get_impl(struct xrt_subdev_pool *spool, xrt_subdev_match_t match, + void *arg, struct device *holder_dev, struct xrt_subdev **sdevp) +{ + struct xrt_device *xdev = (struct xrt_device *)arg; + struct list_head *dl = &spool->xsp_dev_list; + struct mutex *lk = &spool->xsp_lock; + struct xrt_subdev *sdev = NULL; + const struct list_head *ptr; + struct xrt_subdev *d = NULL; + int ret = -ENOENT; + + mutex_lock(lk); + + if (!xdev) { + if (match == XRT_SUBDEV_MATCH_PREV) { + sdev = list_empty(dl) ? NULL : + list_last_entry(dl, struct xrt_subdev, xs_dev_list); + } else if (match == XRT_SUBDEV_MATCH_NEXT) { + sdev = list_first_entry_or_null(dl, struct xrt_subdev, xs_dev_list); + } + } + + list_for_each(ptr, dl) { + d = list_entry(ptr, struct xrt_subdev, xs_dev_list); + if (match == XRT_SUBDEV_MATCH_PREV || match == XRT_SUBDEV_MATCH_NEXT) { + if (d->xs_xdev != xdev) + continue; + } else { + if (!match(d->xs_id, d->xs_xdev, arg)) + continue; + } + + if (match == XRT_SUBDEV_MATCH_PREV) + sdev = !list_is_first(ptr, dl) ? list_prev_entry(d, xs_dev_list) : NULL; + else if (match == XRT_SUBDEV_MATCH_NEXT) + sdev = !list_is_last(ptr, dl) ? list_next_entry(d, xs_dev_list) : NULL; + else + sdev = d; + } + + if (sdev) + ret = xrt_subdev_hold(sdev, holder_dev); + + mutex_unlock(lk); + + if (!ret) + *sdevp = sdev; + return ret; +} + +int xrt_subdev_pool_get(struct xrt_subdev_pool *spool, xrt_subdev_match_t match, void *arg, + struct device *holder_dev, struct xrt_device **xdevp) +{ + int rc; + struct xrt_subdev *sdev; + + rc = xrt_subdev_pool_get_impl(spool, match, arg, holder_dev, &sdev); + if (rc) { + if (rc != -ENOENT) + dev_err(holder_dev, "failed to hold device: %d", rc); + return rc; + } + + if (!IS_ROOT_DEV(holder_dev)) { + xrt_dbg(to_xrt_dev(holder_dev), "%s <<==== %s", + dev_name(holder_dev), dev_name(DEV(sdev->xs_xdev))); + } + + *xdevp = sdev->xs_xdev; + return 0; +} + +static int xrt_subdev_pool_put_impl(struct xrt_subdev_pool *spool, struct xrt_device *xdev, + struct device *holder_dev) +{ + const struct list_head *ptr; + struct mutex *lk = &spool->xsp_lock; + struct list_head *dl = &spool->xsp_dev_list; + struct xrt_subdev *sdev; + int ret = -ENOENT; + + mutex_lock(lk); + list_for_each(ptr, dl) { + sdev = list_entry(ptr, struct xrt_subdev, xs_dev_list); + if (sdev->xs_xdev != xdev) + continue; + ret = xrt_subdev_release(sdev, holder_dev); + break; + } + mutex_unlock(lk); + + return ret; +} + +int xrt_subdev_pool_put(struct xrt_subdev_pool *spool, struct xrt_device *xdev, + struct device *holder_dev) +{ + int ret = xrt_subdev_pool_put_impl(spool, xdev, holder_dev); + + if (ret) + return ret; + + if (!IS_ROOT_DEV(holder_dev)) { + xrt_dbg(to_xrt_dev(holder_dev), "%s <<==X== %s", + dev_name(holder_dev), dev_name(DEV(xdev))); + } + return 0; +} + +void xrt_subdev_pool_trigger_event(struct xrt_subdev_pool *spool, enum xrt_events e) +{ + struct xrt_device *tgt = NULL; + struct xrt_subdev *sdev = NULL; + struct xrt_event evt; + + while (!xrt_subdev_pool_get_impl(spool, XRT_SUBDEV_MATCH_NEXT, + tgt, spool->xsp_owner, &sdev)) { + tgt = sdev->xs_xdev; + evt.xe_evt = e; + evt.xe_subdev.xevt_subdev_id = sdev->xs_id; + evt.xe_subdev.xevt_subdev_instance = tgt->instance; + xrt_subdev_root_request(tgt, XRT_ROOT_EVENT_SYNC, &evt); + xrt_subdev_pool_put_impl(spool, tgt, spool->xsp_owner); + } +} + +void xrt_subdev_pool_handle_event(struct xrt_subdev_pool *spool, struct xrt_event *evt) +{ + struct xrt_device *tgt = NULL; + struct xrt_subdev *sdev = NULL; + + while (!xrt_subdev_pool_get_impl(spool, XRT_SUBDEV_MATCH_NEXT, + tgt, spool->xsp_owner, &sdev)) { + tgt = sdev->xs_xdev; + xleaf_call(tgt, XRT_XLEAF_EVENT, evt); + xrt_subdev_pool_put_impl(spool, tgt, spool->xsp_owner); + } +} + +ssize_t xrt_subdev_pool_get_holders(struct xrt_subdev_pool *spool, + struct xrt_device *xdev, char *buf, size_t len) +{ + const struct list_head *ptr; + struct mutex *lk = &spool->xsp_lock; + struct list_head *dl = &spool->xsp_dev_list; + struct xrt_subdev *sdev; + ssize_t ret = 0; + + mutex_lock(lk); + list_for_each(ptr, dl) { + sdev = list_entry(ptr, struct xrt_subdev, xs_dev_list); + if (sdev->xs_xdev != xdev) + continue; + ret = xrt_subdev_get_holders(sdev, buf, len); + break; + } + mutex_unlock(lk); + + return ret; +} +EXPORT_SYMBOL_GPL(xrt_subdev_pool_get_holders); + +int xleaf_broadcast_event(struct xrt_device *xdev, enum xrt_events evt, bool async) +{ + struct xrt_event e = { evt, }; + enum xrt_root_cmd cmd = async ? XRT_ROOT_EVENT_ASYNC : XRT_ROOT_EVENT_SYNC; + + WARN_ON(evt == XRT_EVENT_POST_CREATION || evt == XRT_EVENT_PRE_REMOVAL); + return xrt_subdev_root_request(xdev, cmd, &e); +} +EXPORT_SYMBOL_GPL(xleaf_broadcast_event); + +void xleaf_hot_reset(struct xrt_device *xdev) +{ + xrt_subdev_root_request(xdev, XRT_ROOT_HOT_RESET, NULL); +} +EXPORT_SYMBOL_GPL(xleaf_hot_reset); + +void xleaf_get_root_res(struct xrt_device *xdev, u32 region_id, struct resource **res) +{ + struct xrt_root_get_res arg = { 0 }; + + arg.xpigr_region_id = region_id; + xrt_subdev_root_request(xdev, XRT_ROOT_GET_RESOURCE, &arg); + *res = arg.xpigr_res; +} + +void xleaf_get_root_id(struct xrt_device *xdev, unsigned short *vendor, unsigned short *device, + unsigned short *subvendor, unsigned short *subdevice) +{ + struct xrt_root_get_id id = { 0 }; + + WARN_ON(!vendor && !device && !subvendor && !subdevice); + + xrt_subdev_root_request(xdev, XRT_ROOT_GET_ID, (void *)&id); + if (vendor) + *vendor = id.xpigi_vendor_id; + if (device) + *device = id.xpigi_device_id; + if (subvendor) + *subvendor = id.xpigi_sub_vendor_id; + if (subdevice) + *subdevice = id.xpigi_sub_device_id; +} + +struct device *xleaf_register_hwmon(struct xrt_device *xdev, const char *name, void *drvdata, + const struct attribute_group **grps) +{ + struct xrt_root_hwmon hm = { true, name, drvdata, grps, }; + + xrt_subdev_root_request(xdev, XRT_ROOT_HWMON, (void *)&hm); + return hm.xpih_hwmon_dev; +} + +void xleaf_unregister_hwmon(struct xrt_device *xdev, struct device *hwmon) +{ + struct xrt_root_hwmon hm = { false, }; + + hm.xpih_hwmon_dev = hwmon; + xrt_subdev_root_request(xdev, XRT_ROOT_HWMON, (void *)&hm); +} From patchwork Mon Jul 19 21:26:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B0A6C63793 for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CBB661175 for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1389158AbhGSVQu (ORCPT ); Mon, 19 Jul 2021 17:16:50 -0400 Received: from mail-dm6nam11on2055.outbound.protection.outlook.com ([40.107.223.55]:40345 "EHLO NAM11-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388259AbhGSUro (ORCPT ); Mon, 19 Jul 2021 16:47:44 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jaGh/SX2ffLmR9VTCyi5LUDad0CTvLE3eypaNM/aZk0k3OV0ZGZZEtuV/q6fktrH4JnX1wYpjF98FfSNlM7K0huUdBW9XbomPQWIS40P13Xr4aBx+koEm+yqr9t3gQ/mECGb+VNDNq5kk5R/TwkWcL/zr0qqnwLaymTYANOuL3t2rCsx6D/EAoBn/PVe0pFvUy3T6WB8iDJZRk7+RDdlObozWyGpqjVs6Zer/fzIFV6cs9WpcgNbfg1MYx/6n2dPowHfxDwwxqRW7FVlO/1xkzMbUl9i4Y7xAB71kRarEEFfGdVa3NYgxIbrYZFlGawCTVdCRSYwCJZaJf+Vuwsg3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qf3UZzc72eCQiDdDldm1ViikonO6NLeCIrav8se8DFI=; b=aNUNezTV8qRtoEpHIMaUoI25a2Pgk6DXhYOGUdM5WVRjlcTcQNujZ+vxmgKGFEg01hmfpz5fuhwSdpJ89U82IdVAnbQEhMdLSwPVgd3qagkIY4WLiFGvNgqISWSZ8MHMMKwDmX8yQOSGFLAS0xCNWg+SEISbAclbxlcTq2VjpS+FSKVlxUZYjt3wefDPWc3cWwk0ZzXSaiMwzFLSQiCyr/fFDZLAHpImtcEsqfIfVnmiFU/SDK4Zip+d5vR+2PlcYIFLt3a8RXLkeW3DRRGT0C5fjaFYg4AEWtsocBRWeE+FDoWLkQinfPxkXKQ6Hy+MhHovDRY17LUHw4t319hQbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qf3UZzc72eCQiDdDldm1ViikonO6NLeCIrav8se8DFI=; b=l1xx3qmG9RrxB2HdRON3nuSWAamAVTbOOr4jIv89kB5MyYxtL24Juh5fP/y1vazRHMrwxvLo8/UZmzuqlQQX23pwErH+1HWHGxeouRBG6enPKj6Bb+jgIoIRYYC5bzFbWb2gz2dxCG/bqfjg8Q0yiNzO0mETfU68tbjwDiIPQxY= Received: from DM5PR1401CA0016.namprd14.prod.outlook.com (2603:10b6:4:4a::26) by SN6PR02MB4894.namprd02.prod.outlook.com (2603:10b6:805:95::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 21:28:22 +0000 Received: from DM3NAM02FT058.eop-nam02.prod.protection.outlook.com (2603:10b6:4:4a:cafe::1) by DM5PR1401CA0016.outlook.office365.com (2603:10b6:4:4a::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:28:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT058.mail.protection.outlook.com (10.13.5.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:28:21 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:28:20 -0700 Received: from smtp.xilinx.com (172.19.127.95) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:28:20 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38282 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5aoG-0004z7-AH; Mon, 19 Jul 2021 14:28:20 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 1102C6020C2; Mon, 19 Jul 2021 14:26:32 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 09/14] fpga: xrt: management physical function driver (root) Date: Mon, 19 Jul 2021 14:26:23 -0700 Message-ID: <20210719212628.134129-10-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9acdee69-3f83-470f-07e4-08d94afc2268 X-MS-TrafficTypeDiagnostic: SN6PR02MB4894: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:326; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b5dW2rKTxekmMmvZxAw//ZdQhMyZU4N1YgsDHwjNwkp0jNxQBptLMF7JtjryJJFyFECNWg1KERUT4O0nV+F7DoNmAKSKdkYDyNCkXnt+uWovLDYczdgJxApIP+5NOywUR0dSu/hJPzZn78+TD+HBLulRLsGhPKGhV7i+iGn9W45ujN4ne/sLvwP9rcVjsqtsvKIGlS2UNthj0arlc1S9MXP7BT3h59j5iYTkHzIHLqQVSJhpci52OjkVMycKqSt3b8heR5xVDgmTrunvZ2dBnakpPgmCkXk+qltFiIhtpPJwELlZYsM4Our2gsONmABDYRNBqDABaY6Wurs/DLdgoUdr8eXDsKioLHAOi02AOkVrUaCzroZ6EseDaNwayVojFL3t86AuYnJLr5jE5ALzIjuL+t/4qz4Cj1TL1nCLPPhfYVSOhFwY93+/YvUQmZGp4ADkI60IASs8cg5fBYpGQxb1+HfF+RhXAx4cZnlr8gMEQijz23a+4jjK/iMbHOg79SieSUOqITD0SHDSOCGkaIvDhu1VzHWqKp38q7CrlSuY7mRD/BCNdd/5VPznqnXr4gcuhgniPRTM1CPHZbGB9ya0BdcPsc5L6A1vDusf/olsAWH0bqMHG1O74cAwQ5QyfE/fAfm5cI72iFd6UJ/t8lG2j0IGcdH7sF/4xB99QdKfJ5PaVCfRaKWoZSWEpCYAvHHV9stXQ1QmCgZff0zPlw== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(356005)(82310400003)(26005)(30864003)(83380400001)(7636003)(47076005)(36756003)(36860700001)(426003)(8676002)(44832011)(70586007)(5660300002)(2906002)(336012)(42186006)(4326008)(316002)(6266002)(186003)(2616005)(508600001)(6916009)(107886003)(54906003)(8936002)(6666004)(36906005)(70206006)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:28:21.8183 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9acdee69-3f83-470f-07e4-08d94afc2268 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT058.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR02MB4894 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The PCIE device driver which attaches to management function on Alveo devices. It instantiates one or more group drivers which, in turn, instantiate xrt drivers. The instantiation of group and xrt drivers is completely dtb driven. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/mgmt/root.c | 420 +++++++++++++++++++++++++++++++++++ 1 file changed, 420 insertions(+) create mode 100644 drivers/fpga/xrt/mgmt/root.c diff --git a/drivers/fpga/xrt/mgmt/root.c b/drivers/fpga/xrt/mgmt/root.c new file mode 100644 index 000000000000..9f3c806a9eaa --- /dev/null +++ b/drivers/fpga/xrt/mgmt/root.c @@ -0,0 +1,420 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx Alveo Management Function Driver + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: + * Cheng Zhen + */ + +#include +#include +#include +#include +#include + +#include "xroot.h" +#include "xmgmt.h" +#include "metadata.h" + +#define XMGMT_MODULE_NAME "xrt-mgmt" +#define XMGMT_DRIVER_VERSION "4.0.0" + +#define XMGMT_PDEV(xm) ((xm)->pdev) +#define XMGMT_DEV(xm) (&(XMGMT_PDEV(xm)->dev)) +#define xmgmt_err(xm, fmt, args...) \ + dev_err(XMGMT_DEV(xm), "%s: " fmt, __func__, ##args) +#define xmgmt_warn(xm, fmt, args...) \ + dev_warn(XMGMT_DEV(xm), "%s: " fmt, __func__, ##args) +#define xmgmt_info(xm, fmt, args...) \ + dev_info(XMGMT_DEV(xm), "%s: " fmt, __func__, ##args) +#define xmgmt_dbg(xm, fmt, args...) \ + dev_dbg(XMGMT_DEV(xm), "%s: " fmt, __func__, ##args) +#define XMGMT_DEV_ID(_pcidev) \ + ({ typeof(_pcidev) (pcidev) = (_pcidev); \ + ((pci_domain_nr((pcidev)->bus) << 16) | \ + PCI_DEVID((pcidev)->bus->number, 0)); }) +#define XRT_VSEC_ID 0x20 +#define XRT_MAX_READRQ 512 + +static struct class *xmgmt_class; + +/* PCI Device IDs */ +/* + * Golden image is preloaded on the device when it is shipped to customer. + * Then, customer can load other shells (from Xilinx or some other vendor). + * If something goes wrong with the shell, customer can always go back to + * golden and start over again. + */ +#define PCI_DEVICE_ID_U50_GOLDEN 0xD020 +#define PCI_DEVICE_ID_U50 0x5020 +static const struct pci_device_id xmgmt_pci_ids[] = { + { PCI_DEVICE(PCI_VENDOR_ID_XILINX, PCI_DEVICE_ID_U50_GOLDEN), }, /* Alveo U50 (golden) */ + { PCI_DEVICE(PCI_VENDOR_ID_XILINX, PCI_DEVICE_ID_U50), }, /* Alveo U50 */ + { 0, } +}; + +struct xmgmt { + struct pci_dev *pdev; + void *root; + + bool ready; +}; + +static int xmgmt_config_pci(struct xmgmt *xm) +{ + struct pci_dev *pdev = XMGMT_PDEV(xm); + int rc; + + rc = pcim_enable_device(pdev); + if (rc < 0) { + xmgmt_err(xm, "failed to enable device: %d", rc); + return rc; + } + + rc = pci_enable_pcie_error_reporting(pdev); + if (rc) + xmgmt_warn(xm, "failed to enable AER: %d", rc); + + pci_set_master(pdev); + + rc = pcie_get_readrq(pdev); + if (rc > XRT_MAX_READRQ) + pcie_set_readrq(pdev, XRT_MAX_READRQ); + return 0; +} + +static int xmgmt_match_slot_and_save(struct device *dev, void *data) +{ + struct xmgmt *xm = data; + struct pci_dev *pdev = to_pci_dev(dev); + + if (XMGMT_DEV_ID(pdev) == XMGMT_DEV_ID(xm->pdev)) { + pci_cfg_access_lock(pdev); + pci_save_state(pdev); + } + + return 0; +} + +static void xmgmt_pci_save_config_all(struct xmgmt *xm) +{ + bus_for_each_dev(&pci_bus_type, NULL, xm, xmgmt_match_slot_and_save); +} + +static int xmgmt_match_slot_and_restore(struct device *dev, void *data) +{ + struct xmgmt *xm = data; + struct pci_dev *pdev = to_pci_dev(dev); + + if (XMGMT_DEV_ID(pdev) == XMGMT_DEV_ID(xm->pdev)) { + pci_restore_state(pdev); + pci_cfg_access_unlock(pdev); + } + + return 0; +} + +static void xmgmt_pci_restore_config_all(struct xmgmt *xm) +{ + bus_for_each_dev(&pci_bus_type, NULL, xm, xmgmt_match_slot_and_restore); +} + +static void xmgmt_root_hot_reset(struct device *dev) +{ + struct pci_dev *pdev = to_pci_dev(dev); + struct pci_bus *bus; + u16 pci_cmd, devctl; + struct xmgmt *xm; + u8 pci_bctl; + int i, ret; + + xm = pci_get_drvdata(pdev); + xmgmt_info(xm, "hot reset start"); + xmgmt_pci_save_config_all(xm); + pci_disable_device(pdev); + bus = pdev->bus; + + /* + * When flipping the SBR bit, device can fall off the bus. This is + * usually no problem at all so long as drivers are working properly + * after SBR. However, some systems complain bitterly when the device + * falls off the bus. + * The quick solution is to temporarily disable the SERR reporting of + * switch port during SBR. + */ + + pci_read_config_word(bus->self, PCI_COMMAND, &pci_cmd); + pci_write_config_word(bus->self, PCI_COMMAND, (pci_cmd & ~PCI_COMMAND_SERR)); + pcie_capability_read_word(bus->self, PCI_EXP_DEVCTL, &devctl); + pcie_capability_write_word(bus->self, PCI_EXP_DEVCTL, (devctl & ~PCI_EXP_DEVCTL_FERE)); + pci_read_config_byte(bus->self, PCI_BRIDGE_CONTROL, &pci_bctl); + pci_write_config_byte(bus->self, PCI_BRIDGE_CONTROL, pci_bctl | PCI_BRIDGE_CTL_BUS_RESET); + msleep(100); + pci_write_config_byte(bus->self, PCI_BRIDGE_CONTROL, pci_bctl); + ssleep(1); + + pcie_capability_write_word(bus->self, PCI_EXP_DEVCTL, devctl); + pci_write_config_word(bus->self, PCI_COMMAND, pci_cmd); + + ret = pci_enable_device(pdev); + if (ret) + xmgmt_err(xm, "failed to enable device, ret %d", ret); + + for (i = 0; i < 300; i++) { + pci_read_config_word(pdev, PCI_COMMAND, &pci_cmd); + if (pci_cmd != 0xffff) + break; + msleep(20); + } + if (i == 300) + xmgmt_err(xm, "timed out waiting for device to be online after reset"); + + xmgmt_info(xm, "waiting for %d ms", i * 20); + xmgmt_pci_restore_config_all(xm); + xmgmt_config_pci(xm); +} + +static int xmgmt_add_vsec_node(struct xmgmt *xm, char *dtb) +{ + struct pci_dev *pdev = XMGMT_PDEV(xm); + struct xrt_md_endpoint ep = { 0 }; + struct device *dev = DEV(pdev); + u32 off_low, off_high, header; + int cap = 0, ret = 0; + __be32 vsec_bar; + __be64 vsec_off; + + while ((cap = pci_find_next_ext_capability(pdev, cap, PCI_EXT_CAP_ID_VNDR))) { + pci_read_config_dword(pdev, cap + PCI_VNDR_HEADER, &header); + if (PCI_VNDR_HEADER_ID(header) == XRT_VSEC_ID) + break; + } + if (!cap) { + xmgmt_info(xm, "No Vendor Specific Capability."); + return -ENOENT; + } + + if (pci_read_config_dword(pdev, cap + 8, &off_low) || + pci_read_config_dword(pdev, cap + 12, &off_high)) { + xmgmt_err(xm, "pci_read vendor specific failed."); + return -EINVAL; + } + + ep.ep_name = XRT_MD_NODE_VSEC; + ret = xrt_md_add_endpoint(dev, dtb, &ep); + if (ret) { + xmgmt_err(xm, "add vsec metadata failed, ret %d", ret); + goto failed; + } + + vsec_bar = cpu_to_be32(off_low & 0xf); + ret = xrt_md_set_prop(dev, dtb, XRT_MD_NODE_VSEC, NULL, + XRT_MD_PROP_BAR_IDX, &vsec_bar, sizeof(vsec_bar)); + if (ret) { + xmgmt_err(xm, "add vsec bar idx failed, ret %d", ret); + goto failed; + } + + vsec_off = cpu_to_be64(((u64)off_high << 32) | (off_low & ~0xfU)); + ret = xrt_md_set_prop(dev, dtb, XRT_MD_NODE_VSEC, NULL, + XRT_MD_PROP_OFFSET, &vsec_off, sizeof(vsec_off)); + if (ret) { + xmgmt_err(xm, "add vsec offset failed, ret %d", ret); + goto failed; + } + +failed: + return ret; +} + +static int xmgmt_create_root_metadata(struct xmgmt *xm, char **root_dtb) +{ + char *dtb = NULL; + int ret; + + ret = xrt_md_create(XMGMT_DEV(xm), &dtb); + if (ret) { + xmgmt_err(xm, "create metadata failed, ret %d", ret); + goto failed; + } + + ret = xmgmt_add_vsec_node(xm, dtb); + if (ret == -ENOENT) { + /* + * We may be dealing with a MFG board. + * Try vsec-golden which will bring up all hard-coded leaves + * at hard-coded offsets. + */ + ret = xroot_add_simple_node(xm->root, dtb, XRT_MD_NODE_VSEC_GOLDEN); + } else if (ret == 0) { + ret = xroot_add_simple_node(xm->root, dtb, XRT_MD_NODE_MGMT_MAIN); + } + if (ret) + goto failed; + + *root_dtb = dtb; + return 0; + +failed: + vfree(dtb); + return ret; +} + +static ssize_t ready_show(struct device *dev, + struct device_attribute *da, + char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + struct xmgmt *xm = pci_get_drvdata(pdev); + + return sprintf(buf, "%d\n", xm->ready); +} +static DEVICE_ATTR_RO(ready); + +static struct attribute *xmgmt_root_attrs[] = { + &dev_attr_ready.attr, + NULL +}; + +static struct attribute_group xmgmt_root_attr_group = { + .attrs = xmgmt_root_attrs, +}; + +static void xmgmt_root_get_id(struct device *dev, struct xrt_root_get_id *rid) +{ + struct pci_dev *pdev = to_pci_dev(dev); + + rid->xpigi_vendor_id = pdev->vendor; + rid->xpigi_device_id = pdev->device; + rid->xpigi_sub_vendor_id = pdev->subsystem_vendor; + rid->xpigi_sub_device_id = pdev->subsystem_device; +} + +static int xmgmt_root_get_resource(struct device *dev, struct xrt_root_get_res *res) +{ + struct pci_dev *pdev = to_pci_dev(dev); + struct xmgmt *xm; + + xm = pci_get_drvdata(pdev); + if (res->xpigr_region_id > PCI_STD_RESOURCE_END) { + xmgmt_err(xm, "Invalid bar idx %d", res->xpigr_region_id); + return -EINVAL; + } + + res->xpigr_res = &pdev->resource[res->xpigr_region_id]; + return 0; +} + +static struct xroot_physical_function_callback xmgmt_xroot_pf_cb = { + .xpc_get_id = xmgmt_root_get_id, + .xpc_get_resource = xmgmt_root_get_resource, + .xpc_hot_reset = xmgmt_root_hot_reset, +}; + +static int xmgmt_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + int ret; + struct device *dev = &pdev->dev; + struct xmgmt *xm = devm_kzalloc(dev, sizeof(*xm), GFP_KERNEL); + char *dtb = NULL; + + if (!xm) + return -ENOMEM; + xm->pdev = pdev; + pci_set_drvdata(pdev, xm); + + ret = xmgmt_config_pci(xm); + if (ret) + goto failed; + + ret = xroot_probe(&pdev->dev, &xmgmt_xroot_pf_cb, &xm->root); + if (ret) + goto failed; + + ret = xmgmt_create_root_metadata(xm, &dtb); + if (ret) + goto failed_metadata; + + ret = xroot_create_group(xm->root, dtb); + vfree(dtb); + if (ret) + xmgmt_err(xm, "failed to create root group: %d", ret); + + if (!xroot_wait_for_bringup(xm->root)) + xmgmt_err(xm, "failed to bringup all groups"); + else + xm->ready = true; + + ret = sysfs_create_group(&pdev->dev.kobj, &xmgmt_root_attr_group); + if (ret) { + /* Warning instead of failing the probe. */ + xmgmt_warn(xm, "create xmgmt root attrs failed: %d", ret); + } + + xroot_broadcast(xm->root, XRT_EVENT_POST_CREATION); + xmgmt_info(xm, "%s started successfully", XMGMT_MODULE_NAME); + return 0; + +failed_metadata: + xroot_remove(xm->root); +failed: + pci_set_drvdata(pdev, NULL); + return ret; +} + +static void xmgmt_remove(struct pci_dev *pdev) +{ + struct xmgmt *xm = pci_get_drvdata(pdev); + + xroot_broadcast(xm->root, XRT_EVENT_PRE_REMOVAL); + sysfs_remove_group(&pdev->dev.kobj, &xmgmt_root_attr_group); + xroot_remove(xm->root); + pci_disable_pcie_error_reporting(xm->pdev); + xmgmt_info(xm, "%s cleaned up successfully", XMGMT_MODULE_NAME); +} + +static struct pci_driver xmgmt_driver = { + .name = XMGMT_MODULE_NAME, + .id_table = xmgmt_pci_ids, + .probe = xmgmt_probe, + .remove = xmgmt_remove, +}; + +static int __init xmgmt_init(void) +{ + int res = 0; + + res = xmgmt_register_leaf(); + if (res) + return res; + + xmgmt_class = class_create(THIS_MODULE, XMGMT_MODULE_NAME); + if (IS_ERR(xmgmt_class)) + return PTR_ERR(xmgmt_class); + + res = pci_register_driver(&xmgmt_driver); + if (res) { + class_destroy(xmgmt_class); + return res; + } + + return 0; +} + +static __exit void xmgmt_exit(void) +{ + pci_unregister_driver(&xmgmt_driver); + class_destroy(xmgmt_class); + xmgmt_unregister_leaf(); +} + +module_init(xmgmt_init); +module_exit(xmgmt_exit); + +MODULE_DEVICE_TABLE(pci, xmgmt_pci_ids); +MODULE_VERSION(XMGMT_DRIVER_VERSION); +MODULE_AUTHOR("XRT Team "); +MODULE_DESCRIPTION("Xilinx Alveo management function driver"); +MODULE_LICENSE("GPL v2"); From patchwork Mon Jul 19 21:26:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B74BFC6379D for ; Mon, 19 Jul 2021 22:00:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C27CF61002 for ; Mon, 19 Jul 2021 22:00:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1389166AbhGSVRJ (ORCPT ); Mon, 19 Jul 2021 17:17:09 -0400 Received: from mail-dm6nam10on2050.outbound.protection.outlook.com ([40.107.93.50]:5345 "EHLO NAM10-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388267AbhGSUsM (ORCPT ); Mon, 19 Jul 2021 16:48:12 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AkI6Sgn8+PzgPgiULB8U26WKpdWAAQmzfSiMlXkzgtMTRaevRuZ3KgU7iCUANeQHg3JXAB4IfHr2hNYvE3fKYpmPBzZ9dqq9RddeemjMc9Mdzh+3bAHJu8OQFHqJuzYofnrUs2MpDChiyAlJxGw77/0y8xfi0IysOiea9HANIeTyJOyos1KVJGlYjtlXWt67VzccDgmx6HZqjIe/Fv4dnQZiUamIGgWS927oVxfvjpbOnSnA4GHaorNmeO065OaYDx4qP9RsBg9iVpcxa2EbqQvMf2PsA8DFYpDw0M0Weg/qBBzCFHpA0hNaStWEYP+YS1hFUal6pksQ9P+b5HSssQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iUl7AflccZxU4PvOj7Nplt9qWvSh3441SZg9SXDXutA=; b=WL+VeH2H7EoXjPC+2P3pr92wYW2KsGaopke4DEfqyvjf82fK48teugwXBZMVftTfaDzdB0Zra8v7xS7VMSI1cOQQCBDwTIkTFNgkLXLZNsRnGh4mZfZld0kAiMraSVmf1zE8BrLJMq1AqUwLH8IKSrWP4HiG23sx/hSWuxar6S+tqY/TybNJ6lT5CdD1cXr4YTyqejiQHc1yf5aOKprwSxVAVDvnR94C9AJqT22okjfRBFtsJWxhImw/R4FH2wjNUegUlYITj5YEGXrgkSaBvcC3xJ/nSLd+uQDey9wHFrJD+DVPhzwzqNwNCthdW5KtfBdUdfL4DQICn4Pd3TH87A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iUl7AflccZxU4PvOj7Nplt9qWvSh3441SZg9SXDXutA=; b=TSicu6JKWnThHARuwXVqlnFAPYh6ZmwYXzohtO3GsqEVwmds72GKqb0PdHDPrCemwFHb6udy3c/qyBdY18/3fE3S6hG7b1oyq2oqT4aZXdOJmA70hERIAVDjHHdrSpd7Nf+WPxly3fuqVLmai1gtRftNRsmZipaMrCTNdzWI1t0= Received: from DS7PR03CA0047.namprd03.prod.outlook.com (2603:10b6:5:3b5::22) by SJ0PR02MB8628.namprd02.prod.outlook.com (2603:10b6:a03:3fd::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 21:28:46 +0000 Received: from DM3NAM02FT034.eop-nam02.prod.protection.outlook.com (2603:10b6:5:3b5:cafe::df) by DS7PR03CA0047.outlook.office365.com (2603:10b6:5:3b5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:28:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT034.mail.protection.outlook.com (10.13.4.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:28:46 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:28:44 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:28:44 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38292 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5aoe-000FHx-DF; Mon, 19 Jul 2021 14:28:44 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 4015F6020C4; Mon, 19 Jul 2021 14:26:32 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 11/14] fpga: xrt: fpga-mgr and region implementation for xclbin download Date: Mon, 19 Jul 2021 14:26:25 -0700 Message-ID: <20210719212628.134129-12-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1fb4f371-ed31-4652-2262-08d94afc30e5 X-MS-TrafficTypeDiagnostic: SJ0PR02MB8628: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:608; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fcehSqoovKCpFyo5YggmyUNkxl3QfwFg5OUHih6pIyVgmxyFcFYq4g2gsvEpvNgmPbZWYZsIufpyU3/kdJrWdW1hsGM8ZI1ST9x2uXszGrYvMT7CHtOO5hlcPIRsd3UYMkH5kf98F6Na/AdXFghHRz9pi4tX7H+8Zw4vT+vJ8/ThQkjnidzvNqMtAIxx2ZqjaxDdOPWa4u0ILT0lMMd7dsb9Wcg6JDP192MeSFL2zIDS1RL5dpSpYrD5FkK9TQbUdGAh6BCOsH9o8dRKh3mqfeCpoX6ny8LkiELE+o5gNXL2KI1Zc/52LuK8u5ktPguKd3D2zs0uK0dVr+m2g1Odyha75ZwQPAf+3OF7rYmlm10Kipbqi1ldkXo6XZ2AdhzJVBYvd9JxBJhsHp7EIr253e0ljah/y9WVQW8dVptIlXaV77ilVQJam7hGp/nkoZDVHcwTjcEZKI+YtVtR5WH5xp4wDPIA0TT4Z1FxrMWqAWHtME1amUcJvBwYtPYg65ZqCTmCdTi2CTgW1Zh8BOGIz3jEtAHeBT6378Xt4lx72bb0AXhDoploKBi9nDZT7UStzI3Xgs39kdeACibWbQIIsGbXmbC7XL7+XT+ByRuCn7q+FkrO74Waov66M1sR6gAJf9vhAkV3bq8BCmNOj/KKzZOplDf2OCHEBtHKj+peRSd/WiFeR75r0ezEBo6+TUPbXP7uCu+Gm6O9hnw0RCM0vA== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(396003)(346002)(39860400002)(136003)(376002)(46966006)(36840700001)(7636003)(36860700001)(83380400001)(426003)(8676002)(336012)(186003)(70206006)(30864003)(5660300002)(44832011)(356005)(70586007)(82740400003)(36906005)(36756003)(47076005)(1076003)(26005)(107886003)(478600001)(2616005)(42186006)(316002)(6666004)(2906002)(6916009)(4326008)(82310400003)(54906003)(6266002)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:28:46.1273 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1fb4f371-ed31-4652-2262-08d94afc30e5 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT034.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR02MB8628 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org fpga-mgr and region implementation for xclbin download which will be called from main xrt driver Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/fpga/xrt/mgmt/xmgmt-main-region.c | 483 ++++++++++++++++++++++ drivers/fpga/xrt/mgmt/xrt-mgr.c | 190 +++++++++ drivers/fpga/xrt/mgmt/xrt-mgr.h | 16 + 3 files changed, 689 insertions(+) create mode 100644 drivers/fpga/xrt/mgmt/xmgmt-main-region.c create mode 100644 drivers/fpga/xrt/mgmt/xrt-mgr.c create mode 100644 drivers/fpga/xrt/mgmt/xrt-mgr.h diff --git a/drivers/fpga/xrt/mgmt/xmgmt-main-region.c b/drivers/fpga/xrt/mgmt/xmgmt-main-region.c new file mode 100644 index 000000000000..6e6a16b13258 --- /dev/null +++ b/drivers/fpga/xrt/mgmt/xmgmt-main-region.c @@ -0,0 +1,483 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * FPGA Region Support for Xilinx Alveo + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: Lizhi.Hou@xilinx.com + */ + +#include +#include +#include +#include +#include "metadata.h" +#include "xleaf.h" +#include "xleaf/axigate.h" +#include "xclbin-helper.h" +#include "xmgmt.h" + +struct xmgmt_bridge { + struct xrt_device *xdev; + const char *bridge_name; +}; + +struct xmgmt_region { + struct xrt_device *xdev; + struct fpga_region *region; + struct fpga_compat_id compat_id; + uuid_t interface_uuid; + struct fpga_bridge *bridge; + int group_instance; + uuid_t depend_uuid; + struct list_head list; +}; + +struct xmgmt_region_match_arg { + struct xrt_device *xdev; + uuid_t *uuids; + u32 uuid_num; +}; + +static int xmgmt_br_enable_set(struct fpga_bridge *bridge, bool enable) +{ + struct xmgmt_bridge *br_data = (struct xmgmt_bridge *)bridge->priv; + struct xrt_device *axigate_leaf; + int rc; + + axigate_leaf = xleaf_get_leaf_by_epname(br_data->xdev, br_data->bridge_name); + if (!axigate_leaf) { + xrt_err(br_data->xdev, "failed to get leaf %s", + br_data->bridge_name); + return -ENOENT; + } + + if (enable) + rc = xleaf_call(axigate_leaf, XRT_AXIGATE_OPEN, NULL); + else + rc = xleaf_call(axigate_leaf, XRT_AXIGATE_CLOSE, NULL); + + if (rc) { + xrt_err(br_data->xdev, "failed to %s gate %s, rc %d", + (enable ? "free" : "freeze"), br_data->bridge_name, + rc); + } + + xleaf_put_leaf(br_data->xdev, axigate_leaf); + + return rc; +} + +static const struct fpga_bridge_ops xmgmt_bridge_ops = { + .enable_set = xmgmt_br_enable_set +}; + +static void xmgmt_destroy_bridge(struct fpga_bridge *br) +{ + struct xmgmt_bridge *br_data = br->priv; + + if (!br_data) + return; + + xrt_info(br_data->xdev, "destroy fpga bridge %s", br_data->bridge_name); + fpga_bridge_unregister(br); + + devm_kfree(DEV(br_data->xdev), br_data); + + fpga_bridge_free(br); +} + +static struct fpga_bridge *xmgmt_create_bridge(struct xrt_device *xdev, + char *dtb) +{ + struct fpga_bridge *br = NULL; + struct xmgmt_bridge *br_data; + const char *gate; + int rc; + + br_data = devm_kzalloc(DEV(xdev), sizeof(*br_data), GFP_KERNEL); + if (!br_data) + return NULL; + br_data->xdev = xdev; + + br_data->bridge_name = XRT_MD_NODE_GATE_ULP; + rc = xrt_md_find_endpoint(&xdev->dev, dtb, XRT_MD_NODE_GATE_ULP, + NULL, &gate); + if (rc) { + br_data->bridge_name = XRT_MD_NODE_GATE_PLP; + rc = xrt_md_find_endpoint(&xdev->dev, dtb, XRT_MD_NODE_GATE_PLP, + NULL, &gate); + } + if (rc) { + xrt_err(xdev, "failed to get axigate, rc %d", rc); + goto failed; + } + + br = fpga_bridge_create(DEV(xdev), br_data->bridge_name, + &xmgmt_bridge_ops, br_data); + if (!br) { + xrt_err(xdev, "failed to create bridge"); + goto failed; + } + + rc = fpga_bridge_register(br); + if (rc) { + xrt_err(xdev, "failed to register bridge, rc %d", rc); + goto failed; + } + + xrt_info(xdev, "created fpga bridge %s", br_data->bridge_name); + + return br; + +failed: + if (br) + fpga_bridge_free(br); + if (br_data) + devm_kfree(DEV(xdev), br_data); + + return NULL; +} + +static void xmgmt_destroy_region(struct fpga_region *region) +{ + struct xmgmt_region *r_data = region->priv; + + xrt_info(r_data->xdev, "destroy fpga region %llx.%llx", + region->compat_id->id_h, region->compat_id->id_l); + + fpga_region_unregister(region); + + if (r_data->group_instance > 0) + xleaf_destroy_group(r_data->xdev, r_data->group_instance); + + if (r_data->bridge) + xmgmt_destroy_bridge(r_data->bridge); + + if (r_data->region->info) { + fpga_image_info_free(r_data->region->info); + r_data->region->info = NULL; + } + + fpga_region_free(region); + + devm_kfree(DEV(r_data->xdev), r_data); +} + +static int xmgmt_region_match(struct device *dev, const void *data) +{ + const struct xmgmt_region_match_arg *arg = data; + const struct fpga_region *match_region; + uuid_t compat_uuid; + int i; + + if (dev->parent != &arg->xdev->dev) + return false; + + match_region = to_fpga_region(dev); + /* + * The device tree provides both parent and child uuids for an + * xclbin in one array. Here we try both uuids to see if it matches + * with target region's compat_id. Strictly speaking we should + * only match xclbin's parent uuid with target region's compat_id + * but given the uuids by design are unique comparing with both + * does not hurt. + */ + import_uuid(&compat_uuid, (const char *)match_region->compat_id); + for (i = 0; i < arg->uuid_num; i++) { + if (uuid_equal(&compat_uuid, &arg->uuids[i])) + return true; + } + + return false; +} + +static int xmgmt_region_match_base(struct device *dev, const void *data) +{ + const struct xmgmt_region_match_arg *arg = data; + const struct fpga_region *match_region; + const struct xmgmt_region *r_data; + + if (dev->parent != &arg->xdev->dev) + return false; + + match_region = to_fpga_region(dev); + r_data = match_region->priv; + if (uuid_is_null(&r_data->depend_uuid)) + return true; + + return false; +} + +static int xmgmt_region_match_by_uuid(struct device *dev, const void *data) +{ + const struct xmgmt_region_match_arg *arg = data; + const struct fpga_region *match_region; + const struct xmgmt_region *r_data; + + if (dev->parent != &arg->xdev->dev) + return false; + + if (arg->uuid_num != 1) + return false; + + match_region = to_fpga_region(dev); + r_data = match_region->priv; + if (uuid_equal(&r_data->depend_uuid, arg->uuids)) + return true; + + return false; +} + +static void xmgmt_region_cleanup(struct fpga_region *region) +{ + struct xmgmt_region *r_data = region->priv, *pdata, *temp; + struct xrt_device *xdev = r_data->xdev; + struct xmgmt_region_match_arg arg = { 0 }; + struct fpga_region *match_region = NULL; + struct device *start_dev = NULL; + LIST_HEAD(free_list); + uuid_t compat_uuid; + + list_add_tail(&r_data->list, &free_list); + arg.xdev = xdev; + arg.uuid_num = 1; + arg.uuids = &compat_uuid; + + /* find all regions depending on this region */ + list_for_each_entry_safe(pdata, temp, &free_list, list) { + import_uuid(arg.uuids, (const char *)pdata->region->compat_id); + start_dev = NULL; + while ((match_region = fpga_region_class_find(start_dev, &arg, + xmgmt_region_match_by_uuid))) { + pdata = match_region->priv; + list_add_tail(&pdata->list, &free_list); + start_dev = &match_region->dev; + put_device(&match_region->dev); + } + } + + list_del(&r_data->list); + + list_for_each_entry_safe_reverse(pdata, temp, &free_list, list) + xmgmt_destroy_region(pdata->region); + + if (r_data->group_instance > 0) { + xleaf_destroy_group(xdev, r_data->group_instance); + r_data->group_instance = -1; + } + if (r_data->region->info) { + fpga_image_info_free(r_data->region->info); + r_data->region->info = NULL; + } +} + +void xmgmt_region_cleanup_all(struct xrt_device *xdev) +{ + struct xmgmt_region_match_arg arg = { 0 }; + struct fpga_region *base_region; + + arg.xdev = xdev; + + while ((base_region = fpga_region_class_find(NULL, &arg, xmgmt_region_match_base))) { + put_device(&base_region->dev); + + xmgmt_region_cleanup(base_region); + xmgmt_destroy_region(base_region); + } +} + +/* + * Program a region with a xclbin image. Bring up the subdevs and the + * group object to contain the subdevs. + */ +static int xmgmt_region_program(struct fpga_region *region, const void *xclbin, char *dtb) +{ + const struct axlf *xclbin_obj = xclbin; + struct fpga_image_info *info; + struct xrt_device *xdev; + struct xmgmt_region *r_data; + int rc; + + r_data = region->priv; + xdev = r_data->xdev; + + info = fpga_image_info_alloc(&xdev->dev); + if (!info) + return -ENOMEM; + + info->buf = xclbin; + info->count = xclbin_obj->header.length; + info->flags |= FPGA_MGR_PARTIAL_RECONFIG; + region->info = info; + rc = fpga_region_program_fpga(region); + if (rc) { + xrt_err(xdev, "programming xclbin failed, rc %d", rc); + return rc; + } + + /* free bridges to allow reprogram */ + if (region->get_bridges) + fpga_bridges_put(®ion->bridge_list); + + /* + * Next bringup the subdevs for this region which will be managed by + * its own group object. + */ + r_data->group_instance = xleaf_create_group(xdev, dtb); + if (r_data->group_instance < 0) { + xrt_err(xdev, "failed to create group, rc %d", + r_data->group_instance); + rc = r_data->group_instance; + return rc; + } + + rc = xleaf_wait_for_group_bringup(xdev); + if (rc) + xrt_err(xdev, "group bringup failed, rc %d", rc); + return rc; +} + +static int xmgmt_get_bridges(struct fpga_region *region) +{ + struct xmgmt_region *r_data = region->priv; + struct device *dev = &r_data->xdev->dev; + + return fpga_bridge_get_to_list(dev, region->info, ®ion->bridge_list); +} + +/* + * Program/create FPGA regions based on input xclbin file. + * 1. Identify a matching existing region for this xclbin + * 2. Tear down any previous objects for the found region + * 3. Program this region with input xclbin + * 4. Iterate over this region's interface uuids to determine if it defines any + * child region. Create fpga_region for the child region. + */ +int xmgmt_process_xclbin(struct xrt_device *xdev, + struct fpga_manager *fmgr, + const struct axlf *xclbin, + enum provider_kind kind) +{ + struct fpga_region *region, *compat_region = NULL; + struct xmgmt_region_match_arg arg = { 0 }; + struct xmgmt_region *r_data; + uuid_t compat_uuid; + char *dtb = NULL; + int rc, i; + + rc = xrt_xclbin_get_metadata(DEV(xdev), xclbin, &dtb); + if (rc) { + xrt_err(xdev, "failed to get dtb: %d", rc); + goto failed; + } + + rc = xrt_md_get_interface_uuids(DEV(xdev), dtb, 0, NULL); + if (rc < 0) { + xrt_err(xdev, "failed to get intf uuid"); + rc = -EINVAL; + goto failed; + } + arg.uuid_num = rc; + arg.uuids = kcalloc(arg.uuid_num, sizeof(uuid_t), GFP_KERNEL); + if (!arg.uuids) { + rc = -ENOMEM; + goto failed; + } + arg.xdev = xdev; + + rc = xrt_md_get_interface_uuids(DEV(xdev), dtb, arg.uuid_num, arg.uuids); + if (rc != arg.uuid_num) { + xrt_err(xdev, "only get %d uuids, expect %d", rc, arg.uuid_num); + rc = -EINVAL; + goto failed; + } + + /* if this is not base firmware, search for a compatible region */ + if (kind != XMGMT_BLP) { + compat_region = fpga_region_class_find(NULL, &arg, xmgmt_region_match); + if (!compat_region) { + xrt_err(xdev, "failed to get compatible region"); + rc = -ENOENT; + goto failed; + } + + xmgmt_region_cleanup(compat_region); + + rc = xmgmt_region_program(compat_region, xclbin, dtb); + if (rc) { + xrt_err(xdev, "failed to program region"); + goto failed; + } + } + + if (compat_region) + import_uuid(&compat_uuid, (const char *)compat_region->compat_id); + + /* create all the new regions contained in this xclbin */ + for (i = 0; i < arg.uuid_num; i++) { + if (compat_region && uuid_equal(&compat_uuid, &arg.uuids[i])) { + /* region for this interface already exists */ + continue; + } + + region = fpga_region_create(DEV(xdev), fmgr, xmgmt_get_bridges); + if (!region) { + xrt_err(xdev, "failed to create fpga region"); + rc = -EFAULT; + goto failed; + } + r_data = devm_kzalloc(DEV(xdev), sizeof(*r_data), GFP_KERNEL); + if (!r_data) { + rc = -ENOMEM; + fpga_region_free(region); + goto failed; + } + r_data->xdev = xdev; + r_data->region = region; + r_data->group_instance = -1; + uuid_copy(&r_data->interface_uuid, &arg.uuids[i]); + if (compat_region) + import_uuid(&r_data->depend_uuid, (const char *)compat_region->compat_id); + r_data->bridge = xmgmt_create_bridge(xdev, dtb); + if (!r_data->bridge) { + xrt_err(xdev, "failed to create fpga bridge"); + rc = -EFAULT; + devm_kfree(DEV(xdev), r_data); + fpga_region_free(region); + goto failed; + } + + region->compat_id = &r_data->compat_id; + export_uuid((char *)region->compat_id, &r_data->interface_uuid); + region->priv = r_data; + + rc = fpga_region_register(region); + if (rc) { + xrt_err(xdev, "failed to register fpga region"); + xmgmt_destroy_bridge(r_data->bridge); + fpga_region_free(region); + devm_kfree(DEV(xdev), r_data); + goto failed; + } + + xrt_info(xdev, "created fpga region %llx.%llx", + region->compat_id->id_h, region->compat_id->id_l); + } + + if (compat_region) + put_device(&compat_region->dev); + vfree(dtb); + kfree(arg.uuids); + return 0; + +failed: + if (compat_region) { + put_device(&compat_region->dev); + xmgmt_region_cleanup(compat_region); + } + + vfree(dtb); + kfree(arg.uuids); + return rc; +} diff --git a/drivers/fpga/xrt/mgmt/xrt-mgr.c b/drivers/fpga/xrt/mgmt/xrt-mgr.c new file mode 100644 index 000000000000..ab253b516e8d --- /dev/null +++ b/drivers/fpga/xrt/mgmt/xrt-mgr.c @@ -0,0 +1,190 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * FPGA Manager Support for Xilinx Alveo + * + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: Sonal.Santan@xilinx.com + */ + +#include +#include +#include +#include +#include + +#include "xclbin-helper.h" +#include "xleaf.h" +#include "xrt-mgr.h" +#include "xleaf/axigate.h" +#include "xleaf/icap.h" +#include "xmgmt.h" + +struct xfpga_class { + struct xrt_device *xdev; + char name[64]; +}; + +/* + * xclbin download plumbing -- find the download subsystem, ICAP and + * pass the xclbin for heavy lifting + */ +static int xmgmt_download_bitstream(struct xrt_device *xdev, + const struct axlf *xclbin) + +{ + struct xclbin_bit_head_info bit_header = { 0 }; + struct xrt_device *icap_leaf = NULL; + struct xrt_icap_wr arg; + char *bitstream = NULL; + u64 bit_len; + int ret; + + ret = xrt_xclbin_get_section(DEV(xdev), xclbin, BITSTREAM, (void **)&bitstream, &bit_len); + if (ret) { + xrt_err(xdev, "bitstream not found"); + return -ENOENT; + } + ret = xrt_xclbin_parse_bitstream_header(DEV(xdev), bitstream, + XCLBIN_HWICAP_BITFILE_BUF_SZ, + &bit_header); + if (ret) { + ret = -EINVAL; + xrt_err(xdev, "invalid bitstream header"); + goto fail; + } + if (bit_header.header_length + bit_header.bitstream_length > bit_len) { + ret = -EINVAL; + xrt_err(xdev, "invalid bitstream length. header %d, bitstream %d, section len %lld", + bit_header.header_length, bit_header.bitstream_length, bit_len); + goto fail; + } + + icap_leaf = xleaf_get_leaf_by_id(xdev, XRT_SUBDEV_ICAP, XRT_INVALID_DEVICE_INST); + if (!icap_leaf) { + ret = -ENODEV; + xrt_err(xdev, "icap does not exist"); + goto fail; + } + arg.xiiw_bit_data = bitstream + bit_header.header_length; + arg.xiiw_data_len = bit_header.bitstream_length; + ret = xleaf_call(icap_leaf, XRT_ICAP_WRITE, &arg); + if (ret) { + xrt_err(xdev, "write bitstream failed, ret = %d", ret); + xleaf_put_leaf(xdev, icap_leaf); + goto fail; + } + + xleaf_put_leaf(xdev, icap_leaf); + vfree(bitstream); + + return 0; + +fail: + vfree(bitstream); + + return ret; +} + +/* + * There is no HW prep work we do here since we need the full + * xclbin for its sanity check. + */ +static int xmgmt_pr_write_init(struct fpga_manager *mgr, + struct fpga_image_info *info, + const char *buf, size_t count) +{ + const struct axlf *bin = (const struct axlf *)buf; + struct xfpga_class *obj = mgr->priv; + + if (!(info->flags & FPGA_MGR_PARTIAL_RECONFIG)) { + xrt_info(obj->xdev, "%s only supports partial reconfiguration\n", obj->name); + return -EINVAL; + } + + if (count < sizeof(struct axlf)) + return -EINVAL; + + if (count > bin->header.length) + return -EINVAL; + + xrt_info(obj->xdev, "Prepare download of xclbin %pUb of length %lld B", + &bin->header.uuid, bin->header.length); + + return 0; +} + +/* + * The implementation requires full xclbin image before we can start + * programming the hardware via ICAP subsystem. The full image is required + * for checking the validity of xclbin and walking the sections to + * discover the bitstream. + */ +static int xmgmt_pr_write(struct fpga_manager *mgr, + const char *buf, size_t count) +{ + const struct axlf *bin = (const struct axlf *)buf; + struct xfpga_class *obj = mgr->priv; + + if (bin->header.length != count) + return -EINVAL; + + return xmgmt_download_bitstream((void *)obj->xdev, bin); +} + +static int xmgmt_pr_write_complete(struct fpga_manager *mgr, + struct fpga_image_info *info) +{ + const struct axlf *bin = (const struct axlf *)info->buf; + struct xfpga_class *obj = mgr->priv; + + xrt_info(obj->xdev, "Finished download of xclbin %pUb", + &bin->header.uuid); + return 0; +} + +static enum fpga_mgr_states xmgmt_pr_state(struct fpga_manager *mgr) +{ + return FPGA_MGR_STATE_UNKNOWN; +} + +static const struct fpga_manager_ops xmgmt_pr_ops = { + .initial_header_size = sizeof(struct axlf), + .write_init = xmgmt_pr_write_init, + .write = xmgmt_pr_write, + .write_complete = xmgmt_pr_write_complete, + .state = xmgmt_pr_state, +}; + +struct fpga_manager *xmgmt_fmgr_probe(struct xrt_device *xdev) +{ + struct xfpga_class *obj = devm_kzalloc(DEV(xdev), sizeof(struct xfpga_class), + GFP_KERNEL); + struct fpga_manager *fmgr = NULL; + int ret = 0; + + if (!obj) + return ERR_PTR(-ENOMEM); + + snprintf(obj->name, sizeof(obj->name), "Xilinx Alveo FPGA Manager"); + obj->xdev = xdev; + fmgr = fpga_mgr_create(&xdev->dev, + obj->name, + &xmgmt_pr_ops, + obj); + if (!fmgr) + return ERR_PTR(-ENOMEM); + + ret = fpga_mgr_register(fmgr); + if (ret) { + fpga_mgr_free(fmgr); + return ERR_PTR(ret); + } + return fmgr; +} + +int xmgmt_fmgr_remove(struct fpga_manager *fmgr) +{ + fpga_mgr_unregister(fmgr); + return 0; +} diff --git a/drivers/fpga/xrt/mgmt/xrt-mgr.h b/drivers/fpga/xrt/mgmt/xrt-mgr.h new file mode 100644 index 000000000000..a3d1ab1c34f0 --- /dev/null +++ b/drivers/fpga/xrt/mgmt/xrt-mgr.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2021 Xilinx, Inc. + * + * Authors: Sonal.Santan@xilinx.com + */ + +#ifndef _XRT_MGR_H_ +#define _XRT_MGR_H_ + +#include + +struct fpga_manager *xmgmt_fmgr_probe(struct xrt_device *xdev); +int xmgmt_fmgr_remove(struct fpga_manager *fmgr); + +#endif /* _XRT_MGR_H_ */ From patchwork Mon Jul 19 21:26:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lizhi Hou X-Patchwork-Id: 480109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1919C64981 for ; Mon, 19 Jul 2021 22:00:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5CD936127C for ; Mon, 19 Jul 2021 22:00:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1389219AbhGSVSM (ORCPT ); Mon, 19 Jul 2021 17:18:12 -0400 Received: from mail-bn8nam11on2071.outbound.protection.outlook.com ([40.107.236.71]:65377 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1388281AbhGSUsq (ORCPT ); Mon, 19 Jul 2021 16:48:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KdxGSj8NjNqMBRn/9cg0FdYtAIciU5iPharMOSiITBg7jP9V+jveMQ7lXWThJ9DynyztI08AuG/ayunMqkCSJ3+XwFhx5ALG/YOTBpKJ6wPSAsU7McTrKC0ftkyNPSEtjsSfC81stPTqKTRDog0nbISFtU4dKcW+ke7e4u+cPB6flaeHJkl17zWhozT9pLYrHydGnNh87u3T36b9SFzERS6SnvNFu70QXPIqkv+i3LKaDbZ7Zez1tY0c60gEs6XCHWssDPso6lkLwlSgQuB6GilXn71S8dYp9ydTGag7hZPGa1rx9lO6yPqd5AASGtXB4kEazLSv+FeUdF+6kwPYiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bAdSl5BtPpX3N6B7GVv+60kBFM5C6Yx6nNmTgjtL26k=; b=QRi5x/e6IeVMcEv0w+l5QmF9HRa8bu1BqwOR5MRhh2Ck2pn3RKOKZ4Gipxca/qBgt9nJ6vG27KVY/S+ypDcZKMxV6fKT3EqBi1JJONAyfnxx1abnQhUItS5CsMfMws8SWRQNk85UyQJuSu5PVNVe0iTMTI/TVBejjDxSCyIZB2X8HAsqTInz/l3yJ0s4rl8I/gb4Nw/Xosd/LELN1UFEFt7fLBZ+phlKpFPlIsM5AVvU/GQC7Oql5bOvQetKD49+fbqtdJ6AlkIMhtl/5dsvkIWuUCKOc/c+cMDfuaKwYQIdu+b4hPBQbhzPqwJVCOgiSU0IP/HDijVXGWBxWkYc7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bAdSl5BtPpX3N6B7GVv+60kBFM5C6Yx6nNmTgjtL26k=; b=sbRaa5rpGd+Kr0TDC8kRFLpt5SlQHHAC+NRCZG/qX7q3DAvCioXQ7SeXpvtOe6/ubwNCCfbYt7Q55xCcRgquYuokHIWbL3YOS5wdqrmeECaSFRBRa2MLeoEy/w4Yo68I+yvlHs0JC2FwBzBGnrNDHJoBTicWTOml+lpVU8Gwvvc= Received: from DM3PR12CA0128.namprd12.prod.outlook.com (2603:10b6:0:51::24) by BYAPR02MB4710.namprd02.prod.outlook.com (2603:10b6:a03:52::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 21:29:21 +0000 Received: from DM3NAM02FT061.eop-nam02.prod.protection.outlook.com (2603:10b6:0:51:cafe::e3) by DM3PR12CA0128.outlook.office365.com (2603:10b6:0:51::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:29:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none; kernel.org; dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by DM3NAM02FT061.mail.protection.outlook.com (10.13.4.230) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 21:29:21 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 19 Jul 2021 14:29:20 -0700 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.2176.2 via Frontend Transport; Mon, 19 Jul 2021 14:29:20 -0700 Envelope-to: mdf@kernel.org, robh@kernel.org, trix@redhat.com, devicetree@vger.kernel.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Received: from [172.19.73.109] (port=38300 helo=xsj-xw9400.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1m5apE-000FL6-He; Mon, 19 Jul 2021 14:29:20 -0700 Received: by xsj-xw9400.xilinx.com (Postfix, from userid 21952) id 7D35F6020C7; Mon, 19 Jul 2021 14:26:32 -0700 (PDT) From: Lizhi Hou To: CC: Lizhi Hou , , , , , , , , , , , Max Zhen Subject: [PATCH V8 XRT Alveo 14/14] fpga: xrt: Kconfig and Makefile updates for XRT drivers Date: Mon, 19 Jul 2021 14:26:28 -0700 Message-ID: <20210719212628.134129-15-lizhi.hou@xilinx.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210719212628.134129-1-lizhi.hou@xilinx.com> References: <20210719212628.134129-1-lizhi.hou@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ea6fcbe1-4929-42cb-a482-08d94afc45d3 X-MS-TrafficTypeDiagnostic: BYAPR02MB4710: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vMQ/8DGk8laKv/vCv5TqHCz4wZTHuTmhIQK/Wm5Yr8/vvdo9EBDKhr+7lr+Q4GTzoeQpZhbwfiPHu5z0g7W/5qQSSwS/v+coSA/n3hddcP3UH2Y/bHq+03hmw78OSz8u1RaTfXCvN+2zyWSNdw2BomUHVHwhhYFtULdobLoA8ulXHKk5eFxX/2sLj15T5mbL4TLLKalgD7Nk5EZKLbTnTgJYr7QIHIs5EKOLABbYkJ16igo88N6eLVJX8Xel8jUcIhEeglFO5tkgZOALRFvX+ZUJdoRALLZimrwe2L+05+CJIdx8mCnRWrPizXp4xejyfip/xRT0OBbdrD0v2izOjhGKhZxkSmUasiGKU3lIiUX5wJLyXWezETzj0pJ/kd3FMhNvHd2Czz7V4MwF8F9zlFLyS9Ym2ZKmFIi+TJjUdZLAqxhO7nQOIpYlzwliHrD3d3mYBNKsixrwoXBRB5AOyVJfycIz//DFqycHU9BLTiEpypf4FIVDFo50jlUDyXF9pgq4Re3UjLV8rxx1SMHtOFX4DpIFM4hlA44ZjMm7l4OzPR9ZwJeKJWo9BiRiH5DVcZw0QkyqPxva/8kxRIxLymop7AO8ikbEJdq0Az3cFkjo+EogmAGcVvdQR1UqirKL2w/x2GuWWtDyFiLrQkFUVriBw+BoEwo6w+vOw84jQYJeHSAllIzg4jWCfU45cMw3VGMk5alL0mzIcnAmc7G9Jw== X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(26005)(356005)(54906003)(186003)(70206006)(82310400003)(7636003)(8676002)(2616005)(336012)(8936002)(83380400001)(426003)(2906002)(44832011)(316002)(36906005)(15650500001)(70586007)(42186006)(1076003)(47076005)(6266002)(508600001)(5660300002)(6666004)(6916009)(107886003)(36756003)(4326008)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 21:29:21.2428 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ea6fcbe1-4929-42cb-a482-08d94afc45d3 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT061.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR02MB4710 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Update fpga Kconfig/Makefile and add Kconfig/Makefile for new drivers. Signed-off-by: Sonal Santan Signed-off-by: Max Zhen Signed-off-by: Lizhi Hou Reviewed-by: Tom Rix --- drivers/Makefile | 1 + drivers/fpga/Kconfig | 2 ++ drivers/fpga/Makefile | 5 +++++ drivers/fpga/xrt/Kconfig | 8 ++++++++ drivers/fpga/xrt/lib/Kconfig | 17 +++++++++++++++++ drivers/fpga/xrt/lib/Makefile | 24 ++++++++++++++++++++++++ drivers/fpga/xrt/metadata/Kconfig | 12 ++++++++++++ drivers/fpga/xrt/metadata/Makefile | 16 ++++++++++++++++ drivers/fpga/xrt/mgmt/Kconfig | 15 +++++++++++++++ drivers/fpga/xrt/mgmt/Makefile | 19 +++++++++++++++++++ 10 files changed, 119 insertions(+) create mode 100644 drivers/fpga/xrt/Kconfig create mode 100644 drivers/fpga/xrt/lib/Kconfig create mode 100644 drivers/fpga/xrt/lib/Makefile create mode 100644 drivers/fpga/xrt/metadata/Kconfig create mode 100644 drivers/fpga/xrt/metadata/Makefile create mode 100644 drivers/fpga/xrt/mgmt/Kconfig create mode 100644 drivers/fpga/xrt/mgmt/Makefile diff --git a/drivers/Makefile b/drivers/Makefile index 27c018bdf4de..64fba9d3adb9 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -180,6 +180,7 @@ obj-$(CONFIG_STM) += hwtracing/stm/ obj-$(CONFIG_ANDROID) += android/ obj-$(CONFIG_NVMEM) += nvmem/ obj-$(CONFIG_FPGA) += fpga/ +obj-$(CONFIG_FPGA_XRT_METADATA) += fpga/ obj-$(CONFIG_FSI) += fsi/ obj-$(CONFIG_TEE) += tee/ obj-$(CONFIG_MULTIPLEXER) += mux/ diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig index 8cd454ee20c0..526447770cab 100644 --- a/drivers/fpga/Kconfig +++ b/drivers/fpga/Kconfig @@ -234,4 +234,6 @@ config FPGA_MGR_ZYNQMP_FPGA to configure the programmable logic(PL) through PS on ZynqMP SoC. +source "drivers/fpga/xrt/Kconfig" + endif # FPGA diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile index 18dc9885883a..4b887bf95cb3 100644 --- a/drivers/fpga/Makefile +++ b/drivers/fpga/Makefile @@ -48,3 +48,8 @@ obj-$(CONFIG_FPGA_DFL_NIOS_INTEL_PAC_N3000) += dfl-n3000-nios.o # Drivers for FPGAs which implement DFL obj-$(CONFIG_FPGA_DFL_PCI) += dfl-pci.o + +# XRT drivers for Alveo +obj-$(CONFIG_FPGA_XRT_METADATA) += xrt/metadata/ +obj-$(CONFIG_FPGA_XRT_LIB) += xrt/lib/ +obj-$(CONFIG_FPGA_XRT_XMGMT) += xrt/mgmt/ diff --git a/drivers/fpga/xrt/Kconfig b/drivers/fpga/xrt/Kconfig new file mode 100644 index 000000000000..0e2c59589ddd --- /dev/null +++ b/drivers/fpga/xrt/Kconfig @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Xilinx Alveo FPGA device configuration +# + +source "drivers/fpga/xrt/metadata/Kconfig" +source "drivers/fpga/xrt/lib/Kconfig" +source "drivers/fpga/xrt/mgmt/Kconfig" diff --git a/drivers/fpga/xrt/lib/Kconfig b/drivers/fpga/xrt/lib/Kconfig new file mode 100644 index 000000000000..935369fad570 --- /dev/null +++ b/drivers/fpga/xrt/lib/Kconfig @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# XRT Alveo FPGA device configuration +# + +config FPGA_XRT_LIB + tristate "XRT Alveo Driver Library" + depends on HWMON && PCI && HAS_IOMEM + select FPGA_XRT_METADATA + select REGMAP_MMIO + help + Select this option to enable Xilinx XRT Alveo driver library. This + library is core infrastructure of XRT Alveo FPGA drivers which + provides functions for working with device nodes, iteration and + lookup of platform devices, common interfaces for platform devices, + plumbing of function call and ioctls between platform devices and + parent partitions. diff --git a/drivers/fpga/xrt/lib/Makefile b/drivers/fpga/xrt/lib/Makefile new file mode 100644 index 000000000000..55cd6063a324 --- /dev/null +++ b/drivers/fpga/xrt/lib/Makefile @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved. +# +# Authors: Sonal.Santan@xilinx.com +# + +FULL_XRT_PATH=$(srctree)/$(src)/.. +FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt + +obj-$(CONFIG_FPGA_XRT_LIB) += xrt-lib.o + +xrt-lib-objs := \ + lib-drv.o \ + xroot.o \ + xclbin.o \ + subdev.o \ + cdev.o \ + group.o \ + xleaf/axigate.o \ + xleaf/icap.o + +ccflags-y := -I$(FULL_XRT_PATH)/include \ + -I$(FULL_DTC_PATH) diff --git a/drivers/fpga/xrt/metadata/Kconfig b/drivers/fpga/xrt/metadata/Kconfig new file mode 100644 index 000000000000..129adda47e94 --- /dev/null +++ b/drivers/fpga/xrt/metadata/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# XRT Alveo FPGA device configuration +# + +config FPGA_XRT_METADATA + bool "XRT Alveo Driver Metadata Parser" + select LIBFDT + help + This option provides helper functions to parse Xilinx Alveo FPGA + firmware metadata. The metadata is in device tree format and the + XRT driver uses it to discover the HW subsystems behind PCIe BAR. diff --git a/drivers/fpga/xrt/metadata/Makefile b/drivers/fpga/xrt/metadata/Makefile new file mode 100644 index 000000000000..14f65ef1595c --- /dev/null +++ b/drivers/fpga/xrt/metadata/Makefile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved. +# +# Authors: Sonal.Santan@xilinx.com +# + +FULL_XRT_PATH=$(srctree)/$(src)/.. +FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt + +obj-$(CONFIG_FPGA_XRT_METADATA) += xrt-md.o + +xrt-md-objs := metadata.o + +ccflags-y := -I$(FULL_XRT_PATH)/include \ + -I$(FULL_DTC_PATH) diff --git a/drivers/fpga/xrt/mgmt/Kconfig b/drivers/fpga/xrt/mgmt/Kconfig new file mode 100644 index 000000000000..31e9e19fffb8 --- /dev/null +++ b/drivers/fpga/xrt/mgmt/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Xilinx XRT FPGA device configuration +# + +config FPGA_XRT_XMGMT + tristate "Xilinx Alveo Management Driver" + depends on FPGA_XRT_LIB + select FPGA_XRT_METADATA + select FPGA_BRIDGE + select FPGA_REGION + help + Select this option to enable XRT PCIe driver for Xilinx Alveo FPGA. + This driver provides interfaces for userspace application to access + Alveo FPGA device. diff --git a/drivers/fpga/xrt/mgmt/Makefile b/drivers/fpga/xrt/mgmt/Makefile new file mode 100644 index 000000000000..16644571b673 --- /dev/null +++ b/drivers/fpga/xrt/mgmt/Makefile @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved. +# +# Authors: Sonal.Santan@xilinx.com +# + +FULL_XRT_PATH=$(srctree)/$(src)/.. +FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt + +obj-$(CONFIG_FPGA_XRT_XMGMT) += xrt-mgmt.o + +xrt-mgmt-objs := root.o \ + xmgmt-main.o \ + xrt-mgr.o \ + xmgmt-main-region.o + +ccflags-y := -I$(FULL_XRT_PATH)/include \ + -I$(FULL_DTC_PATH)