From patchwork Fri May 7 22:51:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 432167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35889C433ED for ; Fri, 7 May 2021 22:51:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0049A61003 for ; Fri, 7 May 2021 22:51:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbhEGWwb (ORCPT ); Fri, 7 May 2021 18:52:31 -0400 Received: from mga09.intel.com ([134.134.136.24]:45212 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230034AbhEGWw3 (ORCPT ); Fri, 7 May 2021 18:52:29 -0400 IronPort-SDR: uSEcIhATXqVVQCK/hZaYEWHFxruinh1KTyOXktWIYiF9HYXjnJ54UOuZOK3/cIxZp9jgWhJ4nX 8+IiCXP9VpnQ== X-IronPort-AV: E=McAfee;i="6200,9189,9977"; a="198909980" X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="198909980" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:26 -0700 IronPort-SDR: Xl5evuaRmfSOw4vbxxyO6VrEGAFHyETRh1OHKqQWn1HFaLg5hPb1tyQYpbX1K1Vp1+HlTBEDGj j40PYT/MX9xw== X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="431695500" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.25]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:25 -0700 Subject: [PATCH 2/8] cxl/mem: Introduce 'struct cxl_regs' for "composable" CXL devices From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org Date: Fri, 07 May 2021 15:51:25 -0700 Message-ID: <162042788555.1202325.7566057227479911213.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> References: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org CXL MMIO register blocks are organized by device type and capabilities. There are Component registers, Device registers (yes, an ambiguous name), and Memory Device registers (a specific extension of Device registers). It is possible for a given device instance (endpoint or port) to implement register sets from multiple of the above categories. The driver code that enumerates and maps the registers is type specific so it is useful to have a dedicated type and helpers for each block type. At the same time, once the registers are mapped the origin type does not matter. It is overly pedantic to reference the register block type in code that is using the registers. In preparation for the endpoint driver to incorporate Component registers into its MMIO operations reorganize the registers to allow typed enumeration + mapping, but anonymous usage. With the end state of 'struct cxl_regs' to be: struct cxl_regs { union { struct { CXL_DEVICE_REGS(); }; struct cxl_device_regs device_regs; }; union { struct { CXL_COMPONENT_REGS(); }; struct cxl_component_regs component_regs; }; }; With this arrangement the driver can share component init code with ports, but when using the registers it can directly reference the component register block type by name without the 'component_regs' prefix. So, map + enumerate can be shared across drivers of different CXL classes e.g.: void cxl_setup_device_regs(struct device *dev, void __iomem *base, struct cxl_device_regs *regs); void cxl_setup_component_regs(struct device *dev, void __iomem *base, struct cxl_component_regs *regs); ...while inline usage in the driver need not indicate where the registers came from: readl(cxlm->regs.mbox + MBOX_OFFSET); readl(cxlm->regs.hdm + HDM_OFFSET); ...instead of: readl(cxlm->regs.device_regs.mbox + MBOX_OFFSET); readl(cxlm->regs.component_regs.hdm + HDM_OFFSET); This complexity of the definition in .h yields improvement in code readability in .c while maintaining type-safety for organization of setup code. It prepares the implementation to maintain organization in the face of CXL devices that compose register interfaces consisting of multiple types. Given that this new container is named 'regs' rename the common register base pointer @base, and fixup the kernel-doc for the missing @cxlmd description. Reviewed-by: Ben Widawsky Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- drivers/cxl/cxl.h | 32 ++++++++++++++++++++++++++++++++ drivers/cxl/mem.c | 44 ++++++++++++++++++++++++-------------------- drivers/cxl/mem.h | 13 +++++-------- 3 files changed, 61 insertions(+), 28 deletions(-) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 2e3bdacb32e7..1f3434f89ef2 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -34,5 +34,37 @@ #define CXLDEV_MBOX_BG_CMD_STATUS_OFFSET 0x18 #define CXLDEV_MBOX_PAYLOAD_OFFSET 0x20 +/* + * CXL_DEVICE_REGS - Common set of CXL Device register block base pointers + * @status: CXL 2.0 8.2.8.3 Device Status Registers + * @mbox: CXL 2.0 8.2.8.4 Mailbox Registers + * @memdev: CXL 2.0 8.2.8.5 Memory Device Registers + */ +#define CXL_DEVICE_REGS() \ + void __iomem *status; \ + void __iomem *mbox; \ + void __iomem *memdev + +/* See note for 'struct cxl_regs' for the rationale of this organization */ +struct cxl_device_regs { + CXL_DEVICE_REGS(); +}; + +/* + * Note, the anonymous union organization allows for per + * register-block-type helper routines, without requiring block-type + * agnostic code to include the prefix. I.e. + * cxl_setup_device_regs(&cxlm->regs.dev) vs readl(cxlm->regs.mbox). + * The specificity reads naturally from left-to-right. + */ +struct cxl_regs { + union { + struct { + CXL_DEVICE_REGS(); + }; + struct cxl_device_regs device_regs; + }; +}; + extern struct bus_type cxl_bus_type; #endif /* __CXL_H__ */ diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 45871ef65152..6951243d128e 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -31,7 +31,7 @@ */ #define cxl_doorbell_busy(cxlm) \ - (readl((cxlm)->mbox_regs + CXLDEV_MBOX_CTRL_OFFSET) & \ + (readl((cxlm)->regs.mbox + CXLDEV_MBOX_CTRL_OFFSET) & \ CXLDEV_MBOX_CTRL_DOORBELL) /* CXL 2.0 - 8.2.8.4 */ @@ -271,7 +271,7 @@ static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm, static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, struct mbox_cmd *mbox_cmd) { - void __iomem *payload = cxlm->mbox_regs + CXLDEV_MBOX_PAYLOAD_OFFSET; + void __iomem *payload = cxlm->regs.mbox + CXLDEV_MBOX_PAYLOAD_OFFSET; u64 cmd_reg, status_reg; size_t out_len; int rc; @@ -314,12 +314,12 @@ static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, } /* #2, #3 */ - writeq(cmd_reg, cxlm->mbox_regs + CXLDEV_MBOX_CMD_OFFSET); + writeq(cmd_reg, cxlm->regs.mbox + CXLDEV_MBOX_CMD_OFFSET); /* #4 */ dev_dbg(&cxlm->pdev->dev, "Sending command\n"); writel(CXLDEV_MBOX_CTRL_DOORBELL, - cxlm->mbox_regs + CXLDEV_MBOX_CTRL_OFFSET); + cxlm->regs.mbox + CXLDEV_MBOX_CTRL_OFFSET); /* #5 */ rc = cxl_mem_wait_for_doorbell(cxlm); @@ -329,7 +329,7 @@ static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, } /* #6 */ - status_reg = readq(cxlm->mbox_regs + CXLDEV_MBOX_STATUS_OFFSET); + status_reg = readq(cxlm->regs.mbox + CXLDEV_MBOX_STATUS_OFFSET); mbox_cmd->return_code = FIELD_GET(CXLDEV_MBOX_STATUS_RET_CODE_MASK, status_reg); @@ -339,7 +339,7 @@ static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, } /* #7 */ - cmd_reg = readq(cxlm->mbox_regs + CXLDEV_MBOX_CMD_OFFSET); + cmd_reg = readq(cxlm->regs.mbox + CXLDEV_MBOX_CMD_OFFSET); out_len = FIELD_GET(CXLDEV_MBOX_CMD_PAYLOAD_LENGTH_MASK, cmd_reg); /* #8 */ @@ -400,7 +400,7 @@ static int cxl_mem_mbox_get(struct cxl_mem *cxlm) goto out; } - md_status = readq(cxlm->memdev_regs + CXLMDEV_STATUS_OFFSET); + md_status = readq(cxlm->regs.memdev + CXLMDEV_STATUS_OFFSET); if (!(md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status))) { dev_err(dev, "mbox: reported doorbell ready, but not mbox ready\n"); rc = -EBUSY; @@ -868,7 +868,7 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) int cap, cap_count; u64 cap_array; - cap_array = readq(cxlm->regs + CXLDEV_CAP_ARRAY_OFFSET); + cap_array = readq(cxlm->base + CXLDEV_CAP_ARRAY_OFFSET); if (FIELD_GET(CXLDEV_CAP_ARRAY_ID_MASK, cap_array) != CXLDEV_CAP_ARRAY_CAP_ID) return -ENODEV; @@ -881,25 +881,25 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) u16 cap_id; cap_id = FIELD_GET(CXLDEV_CAP_HDR_CAP_ID_MASK, - readl(cxlm->regs + cap * 0x10)); - offset = readl(cxlm->regs + cap * 0x10 + 0x4); - register_block = cxlm->regs + offset; + readl(cxlm->base + cap * 0x10)); + offset = readl(cxlm->base + cap * 0x10 + 0x4); + register_block = cxlm->base + offset; switch (cap_id) { case CXLDEV_CAP_CAP_ID_DEVICE_STATUS: dev_dbg(dev, "found Status capability (0x%x)\n", offset); - cxlm->status_regs = register_block; + cxlm->regs.status = register_block; break; case CXLDEV_CAP_CAP_ID_PRIMARY_MAILBOX: dev_dbg(dev, "found Mailbox capability (0x%x)\n", offset); - cxlm->mbox_regs = register_block; + cxlm->regs.mbox = register_block; break; case CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX: dev_dbg(dev, "found Secondary Mailbox capability (0x%x)\n", offset); break; case CXLDEV_CAP_CAP_ID_MEMDEV: dev_dbg(dev, "found Memory Device capability (0x%x)\n", offset); - cxlm->memdev_regs = register_block; + cxlm->regs.memdev = register_block; break; default: dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset); @@ -907,11 +907,11 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) } } - if (!cxlm->status_regs || !cxlm->mbox_regs || !cxlm->memdev_regs) { + if (!cxlm->regs.status || !cxlm->regs.mbox || !cxlm->regs.memdev) { dev_err(dev, "registers not found: %s%s%s\n", - !cxlm->status_regs ? "status " : "", - !cxlm->mbox_regs ? "mbox " : "", - !cxlm->memdev_regs ? "memdev" : ""); + !cxlm->regs.status ? "status " : "", + !cxlm->regs.mbox ? "mbox " : "", + !cxlm->regs.memdev ? "memdev" : ""); return -ENXIO; } @@ -920,7 +920,7 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm) { - const int cap = readl(cxlm->mbox_regs + CXLDEV_MBOX_CAPS_OFFSET); + const int cap = readl(cxlm->regs.mbox + CXLDEV_MBOX_CAPS_OFFSET); cxlm->payload_size = 1 << FIELD_GET(CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK, cap); @@ -980,7 +980,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, mutex_init(&cxlm->mbox_mutex); cxlm->pdev = pdev; - cxlm->regs = regs + offset; + cxlm->base = regs + offset; cxlm->enabled_cmds = devm_kmalloc_array(dev, BITS_TO_LONGS(cxl_cmd_count), sizeof(unsigned long), @@ -1495,6 +1495,10 @@ static __init int cxl_mem_init(void) dev_t devt; int rc; + /* Double check the anonymous union trickery in struct cxl_regs */ + BUILD_BUG_ON(offsetof(struct cxl_regs, memdev) != + offsetof(struct cxl_regs, device_regs.memdev)); + rc = alloc_chrdev_region(&devt, 0, CXL_MEM_MAX_DEVS, "cxl"); if (rc) return rc; diff --git a/drivers/cxl/mem.h b/drivers/cxl/mem.h index 616bb6fd721f..f4d89e816a89 100644 --- a/drivers/cxl/mem.h +++ b/drivers/cxl/mem.h @@ -53,10 +53,9 @@ struct cxl_memdev { /** * struct cxl_mem - A CXL memory device * @pdev: The PCI device associated with this CXL device. - * @regs: IO mappings to the device's MMIO - * @status_regs: CXL 2.0 8.2.8.3 Device Status Registers - * @mbox_regs: CXL 2.0 8.2.8.4 Mailbox Registers - * @memdev_regs: CXL 2.0 8.2.8.5 Memory Device Registers + * @base: IO mappings to the device's MMIO + * @cxlmd: Logical memory device chardev / interface + * @regs: Parsed register blocks * @payload_size: Size of space for payload * (CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: Mutex to synchronize mailbox access. @@ -67,12 +66,10 @@ struct cxl_memdev { */ struct cxl_mem { struct pci_dev *pdev; - void __iomem *regs; + void __iomem *base; struct cxl_memdev *cxlmd; - void __iomem *status_regs; - void __iomem *mbox_regs; - void __iomem *memdev_regs; + struct cxl_regs regs; size_t payload_size; struct mutex mbox_mutex; /* Protects device mailbox and firmware */ From patchwork Fri May 7 22:51:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 432166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD340C433B4 for ; Fri, 7 May 2021 22:51:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 893A160249 for ; Fri, 7 May 2021 22:51:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230102AbhEGWwo (ORCPT ); Fri, 7 May 2021 18:52:44 -0400 Received: from mga04.intel.com ([192.55.52.120]:57716 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230095AbhEGWwn (ORCPT ); Fri, 7 May 2021 18:52:43 -0400 IronPort-SDR: Rr9evBXOn4wgDrWfsmi4aGCXhxGt3JHHN2F95pmWaTEgdSwe9EGQSgI/5fepsRwLN49ky18c4e mRdioy5pW08A== X-IronPort-AV: E=McAfee;i="6200,9189,9977"; a="196817065" X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="196817065" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:42 -0700 IronPort-SDR: 8bMoU5ONJOmRYPfw/J7MunRVJ7jpoq2dZhokOn/dXYjw2uHSf/q6GeqtC3XYqRU1geFAN3Ewta zfA9bu+dByeQ== X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="533857168" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.25]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:42 -0700 Subject: [PATCH 4/8] cxl/core: Refactor CXL register lookup for bridge reuse From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , Jonathan Cameron , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org Date: Fri, 07 May 2021 15:51:42 -0700 Message-ID: <162042790221.1202325.17592119380528540070.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> References: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org While CXL Memory Device endpoints locate the CXL MMIO registers in a PCI BAR, CXL root bridges have their MMIO base address described by platform firmware. Refactor the existing register lookup into a generic facility for endpoints and bridges to share. Reviewed-by: Ben Widawsky Reviewed-by: Jonathan Cameron Signed-off-by: Dan Williams --- drivers/cxl/core.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 +++ drivers/cxl/mem.c | 50 ++++++---------------------------------------- 3 files changed, 65 insertions(+), 44 deletions(-) diff --git a/drivers/cxl/core.c b/drivers/cxl/core.c index 7f8d2034038a..d94c4c2dee8f 100644 --- a/drivers/cxl/core.c +++ b/drivers/cxl/core.c @@ -2,6 +2,7 @@ /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ #include #include +#include "cxl.h" /** * DOC: cxl core @@ -10,6 +11,61 @@ * point for cross-device interleave coordination through cxl ports. */ +/** + * cxl_setup_device_regs() - Detect CXL Device register blocks + * @dev: Host device of the @base mapping + * @base: Mapping of CXL 2.0 8.2.8 CXL Device Register Interface + * @regs: Base pointers for device register blocks (see CXL_DEVICE_REGS()) + */ +void cxl_setup_device_regs(struct device *dev, void __iomem *base, + struct cxl_device_regs *regs) +{ + int cap, cap_count; + u64 cap_array; + + *regs = (struct cxl_device_regs) { 0 }; + + cap_array = readq(base + CXLDEV_CAP_ARRAY_OFFSET); + if (FIELD_GET(CXLDEV_CAP_ARRAY_ID_MASK, cap_array) != + CXLDEV_CAP_ARRAY_CAP_ID) + return; + + cap_count = FIELD_GET(CXLDEV_CAP_ARRAY_COUNT_MASK, cap_array); + + for (cap = 1; cap <= cap_count; cap++) { + void __iomem *register_block; + u32 offset; + u16 cap_id; + + cap_id = FIELD_GET(CXLDEV_CAP_HDR_CAP_ID_MASK, + readl(base + cap * 0x10)); + offset = readl(base + cap * 0x10 + 0x4); + register_block = base + offset; + + switch (cap_id) { + case CXLDEV_CAP_CAP_ID_DEVICE_STATUS: + dev_dbg(dev, "found Status capability (0x%x)\n", offset); + regs->status = register_block; + break; + case CXLDEV_CAP_CAP_ID_PRIMARY_MAILBOX: + dev_dbg(dev, "found Mailbox capability (0x%x)\n", offset); + regs->mbox = register_block; + break; + case CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX: + dev_dbg(dev, "found Secondary Mailbox capability (0x%x)\n", offset); + break; + case CXLDEV_CAP_CAP_ID_MEMDEV: + dev_dbg(dev, "found Memory Device capability (0x%x)\n", offset); + regs->memdev = register_block; + break; + default: + dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset); + break; + } + } +} +EXPORT_SYMBOL_GPL(cxl_setup_device_regs); + struct bus_type cxl_bus_type = { .name = "cxl", }; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1f3434f89ef2..d49e0cb679fa 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -66,5 +66,8 @@ struct cxl_regs { }; }; +void cxl_setup_device_regs(struct device *dev, void __iomem *base, + struct cxl_device_regs *regs); + extern struct bus_type cxl_bus_type; #endif /* __CXL_H__ */ diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 6951243d128e..ee55abfa147e 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -865,53 +865,15 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode, static int cxl_mem_setup_regs(struct cxl_mem *cxlm) { struct device *dev = &cxlm->pdev->dev; - int cap, cap_count; - u64 cap_array; + struct cxl_regs *regs = &cxlm->regs; - cap_array = readq(cxlm->base + CXLDEV_CAP_ARRAY_OFFSET); - if (FIELD_GET(CXLDEV_CAP_ARRAY_ID_MASK, cap_array) != - CXLDEV_CAP_ARRAY_CAP_ID) - return -ENODEV; - - cap_count = FIELD_GET(CXLDEV_CAP_ARRAY_COUNT_MASK, cap_array); - - for (cap = 1; cap <= cap_count; cap++) { - void __iomem *register_block; - u32 offset; - u16 cap_id; - - cap_id = FIELD_GET(CXLDEV_CAP_HDR_CAP_ID_MASK, - readl(cxlm->base + cap * 0x10)); - offset = readl(cxlm->base + cap * 0x10 + 0x4); - register_block = cxlm->base + offset; - - switch (cap_id) { - case CXLDEV_CAP_CAP_ID_DEVICE_STATUS: - dev_dbg(dev, "found Status capability (0x%x)\n", offset); - cxlm->regs.status = register_block; - break; - case CXLDEV_CAP_CAP_ID_PRIMARY_MAILBOX: - dev_dbg(dev, "found Mailbox capability (0x%x)\n", offset); - cxlm->regs.mbox = register_block; - break; - case CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX: - dev_dbg(dev, "found Secondary Mailbox capability (0x%x)\n", offset); - break; - case CXLDEV_CAP_CAP_ID_MEMDEV: - dev_dbg(dev, "found Memory Device capability (0x%x)\n", offset); - cxlm->regs.memdev = register_block; - break; - default: - dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset); - break; - } - } + cxl_setup_device_regs(dev, cxlm->base, ®s->device_regs); - if (!cxlm->regs.status || !cxlm->regs.mbox || !cxlm->regs.memdev) { + if (!regs->status || !regs->mbox || !regs->memdev) { dev_err(dev, "registers not found: %s%s%s\n", - !cxlm->regs.status ? "status " : "", - !cxlm->regs.mbox ? "mbox " : "", - !cxlm->regs.memdev ? "memdev" : ""); + !regs->status ? "status " : "", + !regs->mbox ? "mbox " : "", + !regs->memdev ? "memdev" : ""); return -ENXIO; } From patchwork Fri May 7 22:51:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 432165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A0EEC433B4 for ; Fri, 7 May 2021 22:51:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 478E160FF2 for ; Fri, 7 May 2021 22:51:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230124AbhEGWwz (ORCPT ); Fri, 7 May 2021 18:52:55 -0400 Received: from mga05.intel.com ([192.55.52.43]:33170 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbhEGWwy (ORCPT ); Fri, 7 May 2021 18:52:54 -0400 IronPort-SDR: /IceLBQd1ZAF3JAfBgJIABnEFsjXEtLedAqqAfmwgmk9L2KRjrvFPFxwNTj0ejKvF3a/X6Cyit hS5HDR2er7NQ== X-IronPort-AV: E=McAfee;i="6200,9189,9977"; a="284290002" X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="284290002" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:53 -0700 IronPort-SDR: tvmuAuxoalOBgvgxiNZhhRRdhxOJ63QJbPSu5nIQWw49+1+TZhctp5f9Z4WcuAPX4Ywdz4Bq2+ j0AtepNVX9Cw== X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="465900959" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.25]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:51:53 -0700 Subject: [PATCH 6/8] cxl/Kconfig: Default drivers to CONFIG_CXL_BUS From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org Date: Fri, 07 May 2021 15:51:53 -0700 Message-ID: <162042791307.1202325.2513845748708305095.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> References: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org CONFIG_CXL_BUS is default 'n' as expected for new functionality. When that is enabled do not make the end user hunt for all the expected sub-options to enable. For example CONFIG_CXL_BUS without CONFIG_CXL_MEM is an odd/expert configuration, so is CONFIG_CXL_MEM without CONFIG_CXL_ACPI (on ACPI capable platforms). Default CONFIG_CXL_MEM and CONFIG_CXL_ACPI to CONFIG_CXL_BUS. Acked-by: Ben Widawsky Signed-off-by: Dan Williams Acked-by: Jonathan Cameron --- drivers/cxl/Kconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index fb282af84afd..1da7970a5e55 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -15,6 +15,7 @@ if CXL_BUS config CXL_MEM tristate "CXL.mem: Memory Devices" + default CXL_BUS help The CXL.mem protocol allows a device to act as a provider of "System RAM" and/or "Persistent Memory" that is fully coherent @@ -54,6 +55,7 @@ config CXL_MEM_RAW_COMMANDS config CXL_ACPI tristate "CXL ACPI: Platform Support" depends on ACPI + default CXL_BUS help Enable support for host managed device memory (HDM) resources published by a platform's ACPI CXL memory layout description. From patchwork Fri May 7 22:52:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 432164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D89FC43460 for ; Fri, 7 May 2021 22:52:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F6B0613D9 for ; Fri, 7 May 2021 22:52:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230235AbhEGWxF (ORCPT ); Fri, 7 May 2021 18:53:05 -0400 Received: from mga05.intel.com ([192.55.52.43]:33183 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230099AbhEGWxE (ORCPT ); Fri, 7 May 2021 18:53:04 -0400 IronPort-SDR: 7RDnDXHSQcdh0tofuQsEkU/IN5kxPnej1jCVYWxv0/RDTWo/Jf6A+YpwMDaY8wSwM24eT/1G8Q mYZBmyjBLaHw== X-IronPort-AV: E=McAfee;i="6200,9189,9977"; a="284290023" X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="284290023" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:52:04 -0700 IronPort-SDR: FeJlpeY3CRjRO5ViQBB6nJ6Cd1Q2PdnSs3BBGVDNM0nOq006Aw8R3EpHvIiX9aRpl2zFD4+Nrl vRd4ZPVTICBg== X-IronPort-AV: E=Sophos;i="5.82,282,1613462400"; d="scan'208";a="459953572" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.25]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2021 15:52:04 -0700 Subject: [PATCH 8/8] cxl/acpi: Add module parameters to stand in for ACPI tables From: Dan Williams To: linux-cxl@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org Date: Fri, 07 May 2021 15:52:03 -0700 Message-ID: <162042792370.1202325.11876776878497740843.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> References: <162042787450.1202325.5718541949681409566.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org [debug / to-be-replaced / not-for-upstream] Given ACPICA support is needed before drivers can integrate ACPI functionality add some module parameters as proxies. --- drivers/cxl/acpi.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index bc2a35ae880b..2a48a728f3e0 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -4,10 +4,84 @@ #include #include #include +#include #include #include #include "cxl.h" +/* + * TODO: Replace all of the below module parameters with ACPI CXL + * resource descriptions once ACPICA makes them available. + */ +static unsigned long chbcr[4]; +module_param_named(chbcr0, chbcr[0], ulong, 0400); +module_param_named(chbcr1, chbcr[1], ulong, 0400); +module_param_named(chbcr2, chbcr[2], ulong, 0400); +module_param_named(chbcr3, chbcr[3], ulong, 0400); + +/* TODO: cross-bridge interleave */ +static struct cxl_address_space cxl_space[] = { + [0] = { .range = { 0, -1 }, .targets = 0x1, }, + [1] = { .range = { 0, -1 }, .targets = 0x1, }, + [2] = { .range = { 0, -1 }, .targets = 0x1, }, + [3] = { .range = { 0, -1 }, .targets = 0x1, }, +}; + +static int set_range(const char *val, const struct kernel_param *kp) +{ + unsigned long long size, base; + struct cxl_address_space *space; + unsigned long flags; + char *p; + int rc; + + size = memparse(val, &p); + if (*p != '@') + return -EINVAL; + + base = memparse(p + 1, &p); + if (*p != ':') + return -EINVAL; + + rc = kstrtoul(p + 1, 0, &flags); + if (rc) + return rc; + if (!flags || flags > CXL_ADDRSPACE_MASK) + return rc; + + space = kp->arg; + *space = (struct cxl_address_space) { + .range = { + .start = base, + .end = base + size - 1, + }, + .flags = flags, + }; + + return 0; +} + +static int get_range(char *buf, const struct kernel_param *kp) +{ + struct cxl_address_space *space = kp->arg; + + if (!range_len(&space->range)) + return -EINVAL; + + return sysfs_emit(buf, "%#llx@%#llx :%s%s%s%s\n", + (unsigned long long)range_len(&space->range), + (unsigned long long)space->range.start, + space->flags & CXL_ADDRSPACE_RAM ? " ram" : "", + space->flags & CXL_ADDRSPACE_PMEM ? " pmem" : "", + space->flags & CXL_ADDRSPACE_TYPE2 ? " type2" : "", + space->flags & CXL_ADDRSPACE_TYPE3 ? " type3" : ""); +} + +module_param_call(range0, set_range, get_range, &cxl_space[0], 0400); +module_param_call(range1, set_range, get_range, &cxl_space[1], 0400); +module_param_call(range2, set_range, get_range, &cxl_space[2], 0400); +module_param_call(range3, set_range, get_range, &cxl_space[3], 0400); + static int match_ACPI0016(struct device *dev, const void *host) { struct acpi_device *adev = to_acpi_device(dev); @@ -67,13 +141,16 @@ static int cxl_acpi_register_ports(struct device *dev, struct acpi_device *root, struct cxl_port *port, int idx) { struct acpi_pci_root *pci_root = acpi_pci_find_root(root->handle); + resource_size_t chbcr_base = ~0ULL; struct cxl_walk_context ctx; if (!pci_root) return -ENXIO; /* TODO: fold in CEDT.CHBS retrieval */ - port = devm_cxl_add_port(dev, port, &root->dev, idx, ~0ULL); + if (idx < ARRAY_SIZE(chbcr)) + chbcr_base = chbcr[idx]; + port = devm_cxl_add_port(dev, port, &root->dev, idx, chbcr_base); if (IS_ERR(port)) return PTR_ERR(port); dev_dbg(dev, "%s: register: %s\n", dev_name(&root->dev), @@ -99,7 +176,7 @@ static int cxl_acpi_probe(struct platform_device *pdev) struct cxl_root *cxl_root; int rc, i = 0; - cxl_root = devm_cxl_add_root(dev, NULL, 0); + cxl_root = devm_cxl_add_root(dev, cxl_space, ARRAY_SIZE(cxl_space)); if (IS_ERR(cxl_root)) return PTR_ERR(cxl_root); dev_dbg(dev, "register: %s\n", dev_name(&cxl_root->port.dev));