| Message ID | 168193569717.1178687.294428469465996536.stgit@djiang5-mobl3 |
|---|---|
| State | New |
| Headers | show |
| Series | cxl: Add support for QTG ID retrieval for CXL subsystem | expand |
On Wed, 19 Apr 2023 13:21:37 -0700 Dave Jiang <dave.jiang@intel.com> wrote: > Provide a callback to parse the Device Scoped Latency and Bandwidth > Information Structure (DSLBIS) in the CDAT structures. The DSLBIS > contains the bandwidth and latency information that's tied to a DSMAS > handle. The driver will retrieve the read and write latency and > bandwidth associated with the DSMAS which is tied to a DPA range. > > Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and > Bandwidth Information Structure (DSLBIS) > > Signed-off-by: Dave Jiang <dave.jiang@intel.com> > One comment inline. > +/* Flags for DSLBIS subtable */ > +#define DSLBIS_MEM_MASK GENMASK(3, 0) > +#define DSLBIS_MEM_MEMORY 0 > + > int devm_cxl_port_enumerate_dports(struct cxl_port *port); > struct cxl_dev_state; > int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, > @@ -136,5 +164,9 @@ cdat_table_parse(dsmas); > cdat_table_parse(dslbis); > cdat_table_parse(sslbis); > > -int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg); > +#define cxl_parse_entry(x) \ > +int cxl_##x##_parse_entry(struct cdat_entry_header *header, void *arg) I'm not sure this is worthwhile. What was your reasoning for it? Also wrecks typing that arg argument as I suggested earlier... > + > +cxl_parse_entry(dsmas); > +cxl_parse_entry(dslbis);
On 4/20/23 4:40 AM, Jonathan Cameron wrote: > On Wed, 19 Apr 2023 13:21:37 -0700 > Dave Jiang <dave.jiang@intel.com> wrote: > >> Provide a callback to parse the Device Scoped Latency and Bandwidth >> Information Structure (DSLBIS) in the CDAT structures. The DSLBIS >> contains the bandwidth and latency information that's tied to a DSMAS >> handle. The driver will retrieve the read and write latency and >> bandwidth associated with the DSMAS which is tied to a DPA range. >> >> Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and >> Bandwidth Information Structure (DSLBIS) >> >> Signed-off-by: Dave Jiang <dave.jiang@intel.com> >> > > One comment inline. > >> +/* Flags for DSLBIS subtable */ >> +#define DSLBIS_MEM_MASK GENMASK(3, 0) >> +#define DSLBIS_MEM_MEMORY 0 >> + >> int devm_cxl_port_enumerate_dports(struct cxl_port *port); >> struct cxl_dev_state; >> int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, >> @@ -136,5 +164,9 @@ cdat_table_parse(dsmas); >> cdat_table_parse(dslbis); >> cdat_table_parse(sslbis); >> >> -int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg); >> +#define cxl_parse_entry(x) \ >> +int cxl_##x##_parse_entry(struct cdat_entry_header *header, void *arg) > I'm not sure this is worthwhile. What was your reasoning for it? > Also wrecks typing that arg argument as I suggested earlier... I can remove the macros. They are patterned after the code in ACPI. > >> + >> +cxl_parse_entry(dsmas); >> +cxl_parse_entry(dslbis); >
Dave Jiang wrote: > Provide a callback to parse the Device Scoped Latency and Bandwidth > Information Structure (DSLBIS) in the CDAT structures. The DSLBIS > contains the bandwidth and latency information that's tied to a DSMAS > handle. The driver will retrieve the read and write latency and > bandwidth associated with the DSMAS which is tied to a DPA range. > > Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and > Bandwidth Information Structure (DSLBIS) > > Signed-off-by: Dave Jiang <dave.jiang@intel.com> > > --- > v3: > - Added spec section in commit header. (Alison) > - Remove void * recast. (Alison) > - Rework comment. (Alison) > - Move CDAT parse to cxl_endpoint_port_probe() > - Convert to use 'struct node_hmem_attrs' > > v2: > - Add size check to DSLIBIS table. (Lukas) > - Remove unnecessary entry type check. (Jonathan) > - Move data_type check to after match. (Jonathan) > - Skip unknown data type. (Jonathan) > - Add overflow check for unit multiply. (Jonathan) > - Use dev_warn() when entries parsing fail. (Jonathan) > --- > drivers/cxl/core/cdat.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++ > drivers/cxl/cxlpci.h | 34 +++++++++++++++++++++++- > drivers/cxl/port.c | 11 +++++++- > 3 files changed, 111 insertions(+), 2 deletions(-) > > diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c > index 6f20af83a3ed..e8b9bb99dfdf 100644 > --- a/drivers/cxl/core/cdat.c > +++ b/drivers/cxl/core/cdat.c > @@ -1,5 +1,6 @@ > // SPDX-License-Identifier: GPL-2.0-only > /* Copyright(c) 2023 Intel Corporation. All rights reserved. */ > +#include <linux/overflow.h> > #include "cxlpci.h" > #include "cxl.h" > > @@ -124,3 +125,70 @@ int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg) > return 0; > } > EXPORT_SYMBOL_NS_GPL(cxl_dsmas_parse_entry, CXL); > + > +static void cxl_hmem_attrs_set(struct node_hmem_attrs *attrs, > + int access, unsigned int val) > +{ > + switch (access) { > + case HMAT_SLLBIS_ACCESS_LATENCY: > + attrs->read_latency = val; > + attrs->write_latency = val; > + break; > + case HMAT_SLLBIS_READ_LATENCY: > + attrs->read_latency = val; > + break; > + case HMAT_SLLBIS_WRITE_LATENCY: > + attrs->write_latency = val; > + break; > + case HMAT_SLLBIS_ACCESS_BANDWIDTH: > + attrs->read_bandwidth = val; > + attrs->write_bandwidth = val; > + break; > + case HMAT_SLLBIS_READ_BANDWIDTH: > + attrs->read_bandwidth = val; > + break; > + case HMAT_SLLBIS_WRITE_BANDWIDTH: > + attrs->write_bandwidth = val; > + break; > + } > +} > + > +int cxl_dslbis_parse_entry(struct cdat_entry_header *header, void *arg) > +{ > + struct cdat_dslbis *dslbis = (struct cdat_dslbis *)header; > + struct list_head *dsmas_list = arg; > + struct dsmas_entry *dent; > + > + if (dslbis->hdr.length != sizeof(*dslbis)) { > + pr_warn("Malformed DSLBIS table length: (%lu:%u)\n", > + (unsigned long)sizeof(*dslbis), dslbis->hdr.length); > + return -EINVAL; > + } > + > + /* Skip unrecognized data type */ > + if (dslbis->data_type >= HMAT_SLLBIS_DATA_TYPE_MAX) > + return 0; > + > + list_for_each_entry(dent, dsmas_list, list) { > + u64 val; > + int rc; > + > + if (dslbis->handle != dent->handle) > + continue; Oh, now I see why the list is needed. Update the changelog of the previous patch to indicate that the entries are cached to a list so they can be cross referenced during dslbis parsing. At least that would have saved me from picking on it. > + > + /* Not a memory type, skip */ > + if ((dslbis->flags & DSLBIS_MEM_MASK) != DSLBIS_MEM_MEMORY) > + return 0; > + > + rc = check_mul_overflow(le64_to_cpu(dslbis->entry_base_unit), > + le16_to_cpu(dslbis->entry[0]), &val); > + if (unlikely(rc)) Don't use likely() / unlikely() without performance numbers. The compiler generally does a better job and this is not a hot path.
On 4/24/23 3:46 PM, Dan Williams wrote: > Dave Jiang wrote: >> Provide a callback to parse the Device Scoped Latency and Bandwidth >> Information Structure (DSLBIS) in the CDAT structures. The DSLBIS >> contains the bandwidth and latency information that's tied to a DSMAS >> handle. The driver will retrieve the read and write latency and >> bandwidth associated with the DSMAS which is tied to a DPA range. >> >> Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and >> Bandwidth Information Structure (DSLBIS) >> >> Signed-off-by: Dave Jiang <dave.jiang@intel.com> >> >> --- >> v3: >> - Added spec section in commit header. (Alison) >> - Remove void * recast. (Alison) >> - Rework comment. (Alison) >> - Move CDAT parse to cxl_endpoint_port_probe() >> - Convert to use 'struct node_hmem_attrs' >> >> v2: >> - Add size check to DSLIBIS table. (Lukas) >> - Remove unnecessary entry type check. (Jonathan) >> - Move data_type check to after match. (Jonathan) >> - Skip unknown data type. (Jonathan) >> - Add overflow check for unit multiply. (Jonathan) >> - Use dev_warn() when entries parsing fail. (Jonathan) >> --- >> drivers/cxl/core/cdat.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++ >> drivers/cxl/cxlpci.h | 34 +++++++++++++++++++++++- >> drivers/cxl/port.c | 11 +++++++- >> 3 files changed, 111 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c >> index 6f20af83a3ed..e8b9bb99dfdf 100644 >> --- a/drivers/cxl/core/cdat.c >> +++ b/drivers/cxl/core/cdat.c >> @@ -1,5 +1,6 @@ >> // SPDX-License-Identifier: GPL-2.0-only >> /* Copyright(c) 2023 Intel Corporation. All rights reserved. */ >> +#include <linux/overflow.h> >> #include "cxlpci.h" >> #include "cxl.h" >> >> @@ -124,3 +125,70 @@ int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg) >> return 0; >> } >> EXPORT_SYMBOL_NS_GPL(cxl_dsmas_parse_entry, CXL); >> + >> +static void cxl_hmem_attrs_set(struct node_hmem_attrs *attrs, >> + int access, unsigned int val) >> +{ >> + switch (access) { >> + case HMAT_SLLBIS_ACCESS_LATENCY: >> + attrs->read_latency = val; >> + attrs->write_latency = val; >> + break; >> + case HMAT_SLLBIS_READ_LATENCY: >> + attrs->read_latency = val; >> + break; >> + case HMAT_SLLBIS_WRITE_LATENCY: >> + attrs->write_latency = val; >> + break; >> + case HMAT_SLLBIS_ACCESS_BANDWIDTH: >> + attrs->read_bandwidth = val; >> + attrs->write_bandwidth = val; >> + break; >> + case HMAT_SLLBIS_READ_BANDWIDTH: >> + attrs->read_bandwidth = val; >> + break; >> + case HMAT_SLLBIS_WRITE_BANDWIDTH: >> + attrs->write_bandwidth = val; >> + break; >> + } >> +} >> + >> +int cxl_dslbis_parse_entry(struct cdat_entry_header *header, void *arg) >> +{ >> + struct cdat_dslbis *dslbis = (struct cdat_dslbis *)header; >> + struct list_head *dsmas_list = arg; >> + struct dsmas_entry *dent; >> + >> + if (dslbis->hdr.length != sizeof(*dslbis)) { >> + pr_warn("Malformed DSLBIS table length: (%lu:%u)\n", >> + (unsigned long)sizeof(*dslbis), dslbis->hdr.length); >> + return -EINVAL; >> + } >> + >> + /* Skip unrecognized data type */ >> + if (dslbis->data_type >= HMAT_SLLBIS_DATA_TYPE_MAX) >> + return 0; >> + >> + list_for_each_entry(dent, dsmas_list, list) { >> + u64 val; >> + int rc; >> + >> + if (dslbis->handle != dent->handle) >> + continue; > > Oh, now I see why the list is needed. Update the changelog of the > previous patch to indicate that the entries are cached to a list so they > can be cross referenced during dslbis parsing. At least that would have > saved me from picking on it. Jonathan had the same comment. It'll be updated for the next rev to make the connection. > > >> + >> + /* Not a memory type, skip */ >> + if ((dslbis->flags & DSLBIS_MEM_MASK) != DSLBIS_MEM_MEMORY) >> + return 0; >> + >> + rc = check_mul_overflow(le64_to_cpu(dslbis->entry_base_unit), >> + le16_to_cpu(dslbis->entry[0]), &val); >> + if (unlikely(rc)) > > Don't use likely() / unlikely() without performance numbers. The > compiler generally does a better job and this is not a hot path.
diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index 6f20af83a3ed..e8b9bb99dfdf 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2023 Intel Corporation. All rights reserved. */ +#include <linux/overflow.h> #include "cxlpci.h" #include "cxl.h" @@ -124,3 +125,70 @@ int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg) return 0; } EXPORT_SYMBOL_NS_GPL(cxl_dsmas_parse_entry, CXL); + +static void cxl_hmem_attrs_set(struct node_hmem_attrs *attrs, + int access, unsigned int val) +{ + switch (access) { + case HMAT_SLLBIS_ACCESS_LATENCY: + attrs->read_latency = val; + attrs->write_latency = val; + break; + case HMAT_SLLBIS_READ_LATENCY: + attrs->read_latency = val; + break; + case HMAT_SLLBIS_WRITE_LATENCY: + attrs->write_latency = val; + break; + case HMAT_SLLBIS_ACCESS_BANDWIDTH: + attrs->read_bandwidth = val; + attrs->write_bandwidth = val; + break; + case HMAT_SLLBIS_READ_BANDWIDTH: + attrs->read_bandwidth = val; + break; + case HMAT_SLLBIS_WRITE_BANDWIDTH: + attrs->write_bandwidth = val; + break; + } +} + +int cxl_dslbis_parse_entry(struct cdat_entry_header *header, void *arg) +{ + struct cdat_dslbis *dslbis = (struct cdat_dslbis *)header; + struct list_head *dsmas_list = arg; + struct dsmas_entry *dent; + + if (dslbis->hdr.length != sizeof(*dslbis)) { + pr_warn("Malformed DSLBIS table length: (%lu:%u)\n", + (unsigned long)sizeof(*dslbis), dslbis->hdr.length); + return -EINVAL; + } + + /* Skip unrecognized data type */ + if (dslbis->data_type >= HMAT_SLLBIS_DATA_TYPE_MAX) + return 0; + + list_for_each_entry(dent, dsmas_list, list) { + u64 val; + int rc; + + if (dslbis->handle != dent->handle) + continue; + + /* Not a memory type, skip */ + if ((dslbis->flags & DSLBIS_MEM_MASK) != DSLBIS_MEM_MEMORY) + return 0; + + rc = check_mul_overflow(le64_to_cpu(dslbis->entry_base_unit), + le16_to_cpu(dslbis->entry[0]), &val); + if (unlikely(rc)) + pr_warn("DSLBIS value overflowed.\n"); + + cxl_hmem_attrs_set(&dent->hmem_attrs, dslbis->data_type, val); + break; + } + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_dslbis_parse_entry, CXL); diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 9a2468a93d83..0f36fb23055c 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -2,6 +2,7 @@ /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ #ifndef __CXL_PCI_H__ #define __CXL_PCI_H__ +#include <linux/node.h> #include <linux/pci.h> #include "cxl.h" @@ -104,10 +105,21 @@ struct cdat_subtable_entry { enum cdat_type type; }; +enum { + HMAT_SLLBIS_ACCESS_LATENCY = 0, + HMAT_SLLBIS_READ_LATENCY, + HMAT_SLLBIS_WRITE_LATENCY, + HMAT_SLLBIS_ACCESS_BANDWIDTH, + HMAT_SLLBIS_READ_BANDWIDTH, + HMAT_SLLBIS_WRITE_BANDWIDTH, + HMAT_SLLBIS_DATA_TYPE_MAX +}; + struct dsmas_entry { struct list_head list; struct range dpa_range; u8 handle; + struct node_hmem_attrs hmem_attrs; }; /* Sub-table 0: Device Scoped Memory Affinity Structure (DSMAS) */ @@ -120,6 +132,22 @@ struct cdat_dsmas { __le64 dpa_length; } __packed; +/* Sub-table 1: Device Scoped Latency and Bandwidth Information Structure (DSLBIS) */ +struct cdat_dslbis { + struct cdat_entry_header hdr; + u8 handle; + u8 flags; + u8 data_type; + u8 reserved; + __le64 entry_base_unit; + __le16 entry[3]; + __le16 reserved2; +} __packed; + +/* Flags for DSLBIS subtable */ +#define DSLBIS_MEM_MASK GENMASK(3, 0) +#define DSLBIS_MEM_MEMORY 0 + int devm_cxl_port_enumerate_dports(struct cxl_port *port); struct cxl_dev_state; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, @@ -136,5 +164,9 @@ cdat_table_parse(dsmas); cdat_table_parse(dslbis); cdat_table_parse(sslbis); -int cxl_dsmas_parse_entry(struct cdat_entry_header *header, void *arg); +#define cxl_parse_entry(x) \ +int cxl_##x##_parse_entry(struct cdat_entry_header *header, void *arg) + +cxl_parse_entry(dsmas); +cxl_parse_entry(dslbis); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 3022bdd52439..ab584b83010d 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -141,8 +141,17 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) rc = cdat_table_parse_dsmas(port->cdat.table, cxl_dsmas_parse_entry, (void *)&dsmas_list); - if (rc < 0) + if (rc > 0) { + rc = cdat_table_parse_dslbis(port->cdat.table, + cxl_dslbis_parse_entry, + (void *)&dsmas_list); + if (rc <= 0) { + dev_warn(&port->dev, + "Failed to parse DSLBIS: %d\n", rc); + } + } else { dev_warn(&port->dev, "Failed to parse DSMAS: %d\n", rc); + } dsmas_list_destroy(&dsmas_list); }
Provide a callback to parse the Device Scoped Latency and Bandwidth Information Structure (DSLBIS) in the CDAT structures. The DSLBIS contains the bandwidth and latency information that's tied to a DSMAS handle. The driver will retrieve the read and write latency and bandwidth associated with the DSMAS which is tied to a DPA range. Coherent Device Attribute Table 1.03 2.1 Device Scoped Latency and Bandwidth Information Structure (DSLBIS) Signed-off-by: Dave Jiang <dave.jiang@intel.com> --- v3: - Added spec section in commit header. (Alison) - Remove void * recast. (Alison) - Rework comment. (Alison) - Move CDAT parse to cxl_endpoint_port_probe() - Convert to use 'struct node_hmem_attrs' v2: - Add size check to DSLIBIS table. (Lukas) - Remove unnecessary entry type check. (Jonathan) - Move data_type check to after match. (Jonathan) - Skip unknown data type. (Jonathan) - Add overflow check for unit multiply. (Jonathan) - Use dev_warn() when entries parsing fail. (Jonathan) --- drivers/cxl/core/cdat.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlpci.h | 34 +++++++++++++++++++++++- drivers/cxl/port.c | 11 +++++++- 3 files changed, 111 insertions(+), 2 deletions(-)