| Message ID | 20231211-cxl-cper-v2-0-c116900ba658@intel.com |
|---|---|
| Headers | show |
| Series | efi/cxl-cper: Report CPER CXL component events through trace events | expand |
Ira Weiny wrote: > If the firmware has configured CXL event support to be firmware first > the OS can process those events through CPER records. The CXL layer has > unique DPA to HPA knowledge and standard event trace parsing in place. > > CPER records contain Bus, Device, Function information which can be used > to identify the PCI device which is sending the event. > > Change pci driver registration to include registration for a CXL CPER > callback to process the events through the trace subsystem. > > Signed-off-by: Ira Weiny <ira.weiny@intel.com> > > --- > Changes from v1: > [djbw: use single registration function] > --- > drivers/cxl/core/mbox.c | 31 ++++++++++++++++++++++++----- > drivers/cxl/cxlmem.h | 6 ++++++ > drivers/cxl/pci.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++- > 3 files changed, 83 insertions(+), 6 deletions(-) > > diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c > index a67161f8764a..da262bbc3519 100644 > --- a/drivers/cxl/core/mbox.c > +++ b/drivers/cxl/core/mbox.c > @@ -854,9 +854,30 @@ static const uuid_t dram_event_uuid = CXL_EVENT_DRAM_UUID; > */ > static const uuid_t mem_mod_event_uuid = CXL_EVENT_MEM_MODULE_UUID; > > -static void cxl_event_trace_record(const struct cxl_memdev *cxlmd, > - enum cxl_event_log_type type, > - struct cxl_event_record_raw *record) > +void cxl_event_trace_record(const struct cxl_memdev *cxlmd, > + enum cxl_event_log_type type, > + enum cxl_event_type event_type, > + union cxl_event *event) > +{ > + switch (event_type) { > + case CXL_CPER_EVENT_GEN_MEDIA: > + trace_cxl_general_media(cxlmd, type, &gen_media_event_uuid, > + &event->gen_media); > + break; > + case CXL_CPER_EVENT_DRAM: > + trace_cxl_dram(cxlmd, type, &dram_event_uuid, &event->dram); > + break; > + case CXL_CPER_EVENT_MEM_MODULE: > + trace_cxl_memory_module(cxlmd, type, &mem_mod_event_uuid, > + &event->mem_module); > + break; > + } > +} > +EXPORT_SYMBOL_NS_GPL(cxl_event_trace_record, CXL); > + > +static void __cxl_event_trace_record(const struct cxl_memdev *cxlmd, > + enum cxl_event_log_type type, > + struct cxl_event_record_raw *record) > { > union cxl_event *evt = &record->event; > uuid_t *id = &record->id; > @@ -979,8 +1000,8 @@ static void cxl_mem_get_records_log(struct cxl_memdev_state *mds, > break; > > for (i = 0; i < nr_rec; i++) > - cxl_event_trace_record(cxlmd, type, > - &payload->records[i]); > + __cxl_event_trace_record(cxlmd, type, > + &payload->records[i]); > > if (payload->flags & CXL_GET_EVENT_FLAG_OVERFLOW) > trace_cxl_overflow(cxlmd, type, payload); > diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h > index e5d770e26e02..7a891b4641cc 100644 > --- a/drivers/cxl/cxlmem.h > +++ b/drivers/cxl/cxlmem.h > @@ -481,6 +481,8 @@ struct cxl_memdev_state { > struct cxl_security_state security; > struct cxl_fw_state fw; > > + struct notifier_block cxl_cper_nb; > + > struct rcuwait mbox_wait; > int (*mbox_send)(struct cxl_memdev_state *mds, > struct cxl_mbox_cmd *cmd); > @@ -802,6 +804,10 @@ void set_exclusive_cxl_commands(struct cxl_memdev_state *mds, > void clear_exclusive_cxl_commands(struct cxl_memdev_state *mds, > unsigned long *cmds); > void cxl_mem_get_event_records(struct cxl_memdev_state *mds, u32 status); > +void cxl_event_trace_record(const struct cxl_memdev *cxlmd, > + enum cxl_event_log_type type, > + enum cxl_event_type event_type, > + union cxl_event *event); > int cxl_set_timestamp(struct cxl_memdev_state *mds); > int cxl_poison_state_init(struct cxl_memdev_state *mds); > int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len, > diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c > index 0155fb66b580..30a98399d013 100644 > --- a/drivers/cxl/pci.c > +++ b/drivers/cxl/pci.c > @@ -1,5 +1,6 @@ > // SPDX-License-Identifier: GPL-2.0-only > /* Copyright(c) 2020 Intel Corporation. All rights reserved. */ > +#include <asm-generic/unaligned.h> > #include <linux/io-64-nonatomic-lo-hi.h> > #include <linux/moduleparam.h> > #include <linux/module.h> > @@ -969,6 +970,55 @@ static struct pci_driver cxl_pci_driver = { > }, > }; > > +#define CXL_EVENT_HDR_FLAGS_REC_SEVERITY GENMASK(1, 0) > +static void cxl_cper_event_call(struct cxl_cper_event_data *ev_data) > +{ > + struct cper_cxl_event_devid *device_id = &ev_data->rec->hdr.device_id; > + struct cxl_dev_state *cxlds = NULL; > + enum cxl_event_log_type log_type; > + struct pci_dev *pdev; > + unsigned int devfn; > + u32 hdr_flags; > + > + devfn = PCI_DEVFN(device_id->device_num, device_id->func_num); > + pdev = pci_get_domain_bus_and_slot(device_id->segment_num, > + device_id->bus_num, devfn); What if pci_get_domain_bus_and_slot() returned NULL? > + > + guard(mutex)(&pdev->dev.mutex); Lets not open code this since device_lock() is so prevalent it deserves its own guard() type: DEFINE_GUARD(device, struct device *, device_lock(_T), device_unlock(_T)) > + if (pdev->driver == &cxl_pci_driver) > + cxlds = pci_get_drvdata(pdev); > + if (!cxlds) > + goto out; Lets not mix usage of cleanup.h helpers with usage of goto. The helpers are there to eliminate goto errors. Just add a new helper: DEFINE_FREE(pci_dev_put, struct pci_dev *, if (_T) pci_dev_put(_T)) ...and declare @pdev as: struct pci_dev *pdev __free(pci_dev_put) = NULL; > + > + /* Fabricate a log type */ > + hdr_flags = get_unaligned_le24(ev_data->rec->event.generic.hdr.flags); > + log_type = FIELD_GET(CXL_EVENT_HDR_FLAGS_REC_SEVERITY, hdr_flags); > + > + cxl_event_trace_record(cxlds->cxlmd, log_type, ev_data->event_type, > + &ev_data->rec->event); > +out: > + pci_dev_put(pdev); > +} > + > +static int __init cxl_pci_driver_init(void) > +{ > + int rc; > + > + rc = pci_register_driver(&cxl_pci_driver); > + if (rc) > + return rc; > + > + cxl_cper_register_notifier(cxl_cper_event_call); > + return 0; > +} > + > +static void __exit cxl_pci_driver_exit(void) > +{ > + cxl_cper_unregister_notifier(); > + pci_unregister_driver(&cxl_pci_driver); > +} > + > +module_init(cxl_pci_driver_init); > +module_exit(cxl_pci_driver_exit); > MODULE_LICENSE("GPL v2"); > -module_pci_driver(cxl_pci_driver); > MODULE_IMPORT_NS(CXL); > > -- > 2.43.0 >
Dan Williams wrote: > Ira Weiny wrote: [snip] > > diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h > > index e5d770e26e02..7a891b4641cc 100644 > > --- a/drivers/cxl/cxlmem.h > > +++ b/drivers/cxl/cxlmem.h > > @@ -481,6 +481,8 @@ struct cxl_memdev_state { > > struct cxl_security_state security; > > struct cxl_fw_state fw; > > > > + struct notifier_block cxl_cper_nb; delete this... [snip] > > > > +#define CXL_EVENT_HDR_FLAGS_REC_SEVERITY GENMASK(1, 0) > > +static void cxl_cper_event_call(struct cxl_cper_event_data *ev_data) > > +{ > > + struct cper_cxl_event_devid *device_id = &ev_data->rec->hdr.device_id; > > + struct cxl_dev_state *cxlds = NULL; > > + enum cxl_event_log_type log_type; > > + struct pci_dev *pdev; > > + unsigned int devfn; > > + u32 hdr_flags; > > + > > + devfn = PCI_DEVFN(device_id->device_num, device_id->func_num); > > + pdev = pci_get_domain_bus_and_slot(device_id->segment_num, > > + device_id->bus_num, devfn); > > What if pci_get_domain_bus_and_slot() returned NULL? :-/ yep. > > > + > > + guard(mutex)(&pdev->dev.mutex); > > Lets not open code this since device_lock() is so prevalent it deserves > its own guard() type: > > DEFINE_GUARD(device, struct device *, device_lock(_T), device_unlock(_T)) Good idea. > > > + if (pdev->driver == &cxl_pci_driver) > > + cxlds = pci_get_drvdata(pdev); > > + if (!cxlds) > > + goto out; > > Lets not mix usage of cleanup.h helpers with usage of goto. The helpers > are there to eliminate goto errors. Just add a new helper: > > DEFINE_FREE(pci_dev_put, struct pci_dev *, if (_T) pci_dev_put(_T)) > > ...and declare @pdev as: > > struct pci_dev *pdev __free(pci_dev_put) = NULL; > Even better idea. Done. Thanks, Ira
Series status/background ======================== Smita has been a great help with this series. This includes testing the last RFC version.[1] Unfortunately, Dan had a better idea for how to register the call between the efi and cxl subsystems so this is reworked for V2. [1] https://lore.kernel.org/all/7ec6d2af-c860-9b05-7eaf-c82f50f8e66e@amd.com/ Cover letter ============ CXL Component Events, as defined by EFI 2.10 Section N.2.14, wrap a mostly CXL event payload in an EFI Common Platform Error Record (CPER) record. If a device is configured for firmware first CXL event records are not sent directly to the host. The CXL sub-system uniquely has DPA to HPA translation information. It also already has event format tracing. Restructure the code to make sharing the data between CPER/event logs most efficient. Then send the CXL CPER records to the CXL sub-system for processing. With event logs the events interrupt the driver directly. In the EFI case events are wrapped with device information which allows the CXL subsystem to identify the PCI device. Previous version considered matching the memdev differently. However, the most robust was to find the PCI device via Bus, Device, Function and use the PCI device to find the driver data. CPER records are identified with GUID's while CXL event logs contain UUID's. The UUID is reported for all events. While the UUID is redundant for the known events the UUID's are already used by rasdaemon. To keep compatibility UUIDs are injected for CPER records based on the record type. In addition this series cleans up the UUID defines used between the event processing and cxl_test code. Signed-off-by: Ira Weiny <ira.weiny@intel.com> --- Changes in v2: - djbw: Clarify GUID defines - djbw: Clarify UUID defines - djbw: Use a single event callback to the CXL subsystem - iweiny: Minor function name clean ups - Link to v1: https://lore.kernel.org/r/20230601-cxl-cper-v1-0-d19f1ac18ab6@intel.com --- Ira Weiny (7): cxl/trace: Pass uuid explicitly to event traces cxl/events: Promote CXL event structures to a core header cxl/events: Create common event UUID defines cxl/events: Separate UUID from event structures cxl/events: Create a CXL event union firmware/efi: Process CXL Component Events cxl/memdev: Register for and process CPER events drivers/cxl/core/mbox.c | 77 ++++++++++--------- drivers/cxl/core/trace.h | 32 ++++---- drivers/cxl/cxlmem.h | 112 +++++++-------------------- drivers/cxl/pci.c | 52 ++++++++++++- drivers/firmware/efi/cper.c | 15 ++++ drivers/firmware/efi/cper_cxl.c | 45 +++++++++++ drivers/firmware/efi/cper_cxl.h | 29 +++++++ include/linux/cxl-event.h | 162 +++++++++++++++++++++++++++++++++++++++ tools/testing/cxl/test/mem.c | 163 +++++++++++++++++++++++----------------- 9 files changed, 481 insertions(+), 206 deletions(-) --- base-commit: 7475e51b87969e01a6812eac713a1c8310372e8a change-id: 20230601-cxl-cper-26ffc839c6c6 Best regards,