| Message ID | 20250115-netcon_cpu-v2-3-95971b44dc56@debian.org |
|---|---|
| State | New |
| Headers | show |
| Series | [net-next,v2,1/5] netconsole: Rename userdata to extradata | expand |
On Fri, 17 Jan 2025 03:02:40 -0800 Breno Leitao wrote: > > Looks like previously all the data was on the stack, now we have a mix. > > Not sure I followed. The data ({userdata,extradata}_complete) was always > inside nt field, which belongs to target_list. I mean the buffer we use for formatting. Today it's this: static char buf[MAX_PRINT_CHUNK]; /* protected by target_list_lock */ int header_len, msgbody_len; const char *msgbody; right? I missed that "static" actually so it's not on the stack, it's in the .bss section. > > Maybe we can pack all the bits of state into a struct for easier > > passing around, but still put it on the stack? > > It depends on what state you need here. We can certainly pass runtime > (aka sysdata in this patchset) data in the stack, but doing the same for > userdata would require extra computation in runtime. In other words, the > userdata_complete and length are calculated at configfs update time > today, and only read during runtime, and there is no connection between > configfs and runtime (write_ext_msg()) except through the stack. > > > On the other side, if we want to have extradata_complete in the stack, I > still think that userdata will need to be in the stack, and create a > buffer in runtime's frame and copy userdata + sysdata at run time, doing > an extra copy. > > Trying to put this in code, this is what I thought: > > /* Copy to the stack (buf) the userdata string + sysdata */ > static void append_runtime_sysdata(struct netconsole_target *nt, char *buf) { > if (!(nt->sysdata_fields & CPU_NR)) > return; > > return scnprintf(buf, MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS, > "%s cpu=%u\n", nt->userdata_complete, raw_smp_processor_id()); > } > > /* Move complete string in the stack and send from there */ > static void send_ext_msg_udp(struct netconsole_target *nt, const char *msg, > int msg_len) { > ... > #ifdef CONFIG_NETCONSOLE_DYNAMIC > struct char buf[MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS]; > extradata_len = append_runtime_sysdata(nt, buf); > #endif > > send_msg_{no}_fragmentation(nt, msg, buf, extradata_len, release_len) > ... > } My thinking was to handle it like the release. Print it at the send_msg_no_fragmentation() stage directly into the static buffer. Does that get hairy coding-wise?
On Fri, Jan 17, 2025 at 06:35:20PM -0800, Jakub Kicinski wrote: > On Fri, 17 Jan 2025 03:02:40 -0800 Breno Leitao wrote: > > > Looks like previously all the data was on the stack, now we have a mix. > > > > Not sure I followed. The data ({userdata,extradata}_complete) was always > > inside nt field, which belongs to target_list. > > I mean the buffer we use for formatting. Today it's this: > > static char buf[MAX_PRINT_CHUNK]; /* protected by target_list_lock */ > int header_len, msgbody_len; > const char *msgbody; > > right? I missed that "static" actually so it's not on the stack, > it's in the .bss section. Since you raised this topic, I don't think buf needs to be static for a functional perspective, since `buf` is completely overwritten every time send_msg functions are called. > > > Maybe we can pack all the bits of state into a struct for easier > > > passing around, but still put it on the stack? > > > > It depends on what state you need here. We can certainly pass runtime > > (aka sysdata in this patchset) data in the stack, but doing the same for > > userdata would require extra computation in runtime. In other words, the > > userdata_complete and length are calculated at configfs update time > > today, and only read during runtime, and there is no connection between > > configfs and runtime (write_ext_msg()) except through the stack. > > > > > > On the other side, if we want to have extradata_complete in the stack, I > > still think that userdata will need to be in the stack, and create a > > buffer in runtime's frame and copy userdata + sysdata at run time, doing > > an extra copy. > > > > Trying to put this in code, this is what I thought: > > > > /* Copy to the stack (buf) the userdata string + sysdata */ > > static void append_runtime_sysdata(struct netconsole_target *nt, char *buf) { > > if (!(nt->sysdata_fields & CPU_NR)) > > return; > > > > return scnprintf(buf, MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS, > > "%s cpu=%u\n", nt->userdata_complete, raw_smp_processor_id()); > > } > > > > /* Move complete string in the stack and send from there */ > > static void send_ext_msg_udp(struct netconsole_target *nt, const char *msg, > > int msg_len) { > > ... > > #ifdef CONFIG_NETCONSOLE_DYNAMIC > > struct char buf[MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS]; > > extradata_len = append_runtime_sysdata(nt, buf); > > #endif > > > > send_msg_{no}_fragmentation(nt, msg, buf, extradata_len, release_len) > > ... > > } > > My thinking was to handle it like the release. > Print it at the send_msg_no_fragmentation() stage directly > into the static buffer. Does that get hairy coding-wise? I suppose the advantage of doing this approach is to reduce a memcpy/strcpy, right? If this is what your motivation, I think we cannot remove it from the fragmented case. Let me share my thought process: 1) sysdata needs to be appended to both send_msg_fragmented() and send_msg_no_fragmentation(). The fragmented case is the problem. 2) It is trivially done in send_msg_fragmented() case. 3) For the send_msg_no_fragmentation() case, there is no trivial way to get it done without using a secondary buffer and then memcpy to `buf`. Let's suppose sysdata has "cpu=42", and original `buf` has only 5 available chars, thus it needs to have 2 msgs to accommodate the full message. Then the it needs to track that `cpu=4` will be sent in a msg and create another message with the missing `2`. The only way to do it properly is having a extra buffer where we have `cpu=42` and copy 5 bytes from there, and then copy the last one in the next iteration. I am not sure we can do it in one shot. On top of that, I am planning to increase other features in sysdata (such as current task name, modules and even consolidate the release as sysdata), which has two implications: 1) Average messages size will become bigger. Thus, memcpy will be needed one way or another. 2) Unless we can come up with a smart solution, this solution will be harder to reason about. If you want to invest more time in this direction, I am more than happy to create a PoC, so we can discuss more concretely. Thanks, --breno
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index 108ec4f836b62860832c601768546c0ecbdb1153..d89b4ef23965ac0a25090d0a1e461de4e56c5fa7 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -97,6 +97,15 @@ struct netconsole_target_stats { struct u64_stats_sync syncp; }; +/* Features enabled in sysdata. Contrary to userdata, this data is populated by + * the kernel. The fields are designed as bitwise flags, allowing multiple + * features to be set in sysdata_fields. + */ +enum sysdata_feature { + /* Populate the CPU that sends the message */ + CPU_NR = BIT(0), +}; + /** * struct netconsole_target - Represents a configured netconsole target. * @list: Links this target into the target_list. @@ -104,6 +113,8 @@ struct netconsole_target_stats { * @userdata_group: Links to the userdata configfs hierarchy * @extradata_complete: Cached, formatted string of append * @userdata_length: String length of usedata in extradata_complete. + * @sysdata_length: String length of sysdata in extradata_complete. + * @sysdata_fields: Sysdata features enabled. * @stats: Packet send stats for the target. Used for debugging. * @enabled: On / off knob to enable / disable target. * Visible from userspace (read-write). @@ -130,7 +141,9 @@ struct netconsole_target { struct config_group group; struct config_group userdata_group; char extradata_complete[MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS]; - size_t userdata_length; + size_t userdata_length; /* updated at update_userdata() */ + size_t sysdata_length; /* updated at every message */ + u32 sysdata_fields; /* bit-wise with sysdata_feature bits */ #endif struct netconsole_target_stats stats; bool enabled; @@ -396,6 +409,19 @@ static ssize_t transmit_errors_show(struct config_item *item, char *buf) return sysfs_emit(buf, "%llu\n", xmit_drop_count + enomem_count); } +/* configfs helper to display if cpu_nr sysdata feature is enabled */ +static ssize_t sysdata_cpu_nr_show(struct config_item *item, char *buf) +{ + struct netconsole_target *nt = to_target(item->ci_parent); + bool cpu_nr_enabled; + + mutex_lock(&dynamic_netconsole_mutex); + cpu_nr_enabled = nt->sysdata_fields & CPU_NR; + mutex_unlock(&dynamic_netconsole_mutex); + + return sysfs_emit(buf, "%d\n", cpu_nr_enabled); +} + /* * This one is special -- targets created through the configfs interface * are not enabled (and the corresponding netpoll activated) by default. @@ -666,7 +692,15 @@ static ssize_t remote_ip_store(struct config_item *item, const char *buf, */ static size_t count_extradata_entries(struct netconsole_target *nt) { - return list_count_nodes(&nt->userdata_group.cg_children); + size_t entries; + + /* Userdata entries */ + entries = list_count_nodes(&nt->userdata_group.cg_children); + /* Plus sysdata entries */ + if (nt->sysdata_fields & CPU_NR) + entries += 1; + + return entries; } static ssize_t remote_mac_store(struct config_item *item, const char *buf, @@ -790,7 +824,67 @@ static ssize_t userdatum_value_store(struct config_item *item, const char *buf, return ret; } +/* disable_sysdata_feature - Disable sysdata feature and clean sysdata + * @nt: target that is diabling the feature + * @feature: feature being disabled + */ +static void disable_sysdata_feature(struct netconsole_target *nt, + enum sysdata_feature feature) +{ + nt->sysdata_fields &= ~feature; + nt->sysdata_length = 0; + /* extradata_complete might have sysdata appended after + * userdata. Clean everything, and it will be re-appened + * when a new message is sent. + */ + nt->extradata_complete[nt->userdata_length] = 0; +} + +/* configfs helper to sysdata cpu_nr feature */ +static ssize_t sysdata_cpu_nr_store(struct config_item *item, const char *buf, + size_t count) +{ + struct netconsole_target *nt = to_target(item->ci_parent); + bool cpu_nr_enabled, curr; + ssize_t ret; + + ret = kstrtobool(buf, &cpu_nr_enabled); + if (ret) + return ret; + + mutex_lock(&dynamic_netconsole_mutex); + curr = nt->sysdata_fields & CPU_NR; + if (cpu_nr_enabled == curr) + /* no change requested */ + goto unlock_ok; + + if (cpu_nr_enabled && + count_extradata_entries(nt) >= MAX_EXTRADATA_ITEMS) { + /* user wants the new feature, but there is no space in the + * buffer. + */ + ret = -ENOSPC; + goto unlock; + } + + if (cpu_nr_enabled) + nt->sysdata_fields |= CPU_NR; + else + /* This is special because extradata_complete might have + * remaining data from previous sysdata, and it needs to be + * cleaned. + */ + disable_sysdata_feature(nt, CPU_NR); + +unlock_ok: + ret = strnlen(buf, count); +unlock: + mutex_unlock(&dynamic_netconsole_mutex); + return ret; +} + CONFIGFS_ATTR(userdatum_, value); +CONFIGFS_ATTR(sysdata_, cpu_nr); static struct configfs_attribute *userdatum_attrs[] = { &userdatum_attr_value, @@ -850,6 +944,7 @@ static void userdatum_drop(struct config_group *group, struct config_item *item) } static struct configfs_attribute *userdata_attrs[] = { + &sysdata_attr_cpu_nr, NULL, }; @@ -1025,6 +1120,35 @@ static void populate_configfs_item(struct netconsole_target *nt, init_target_config_group(nt, target_name); } +/* + * append_runtime_sysdata - append sysdata at extradata_complete in runtime + * @nt: target to send message to + */ +static void append_runtime_sysdata(struct netconsole_target *nt) +{ + size_t userdata_len = nt->userdata_length; + size_t sysdata_len; + + if (!(nt->sysdata_fields & CPU_NR)) + return; + + /* Append cpu=%d at extradata_complete after userdata str */ + sysdata_len = scnprintf(&nt->extradata_complete[userdata_len], + MAX_EXTRADATA_ENTRY_LEN, " cpu=%u\n", + raw_smp_processor_id()); + + WARN_ON_ONCE(userdata_len + sysdata_len > + MAX_EXTRADATA_ENTRY_LEN * MAX_EXTRADATA_ITEMS); + + /* nt->sysdata_length will be used later to decide if the message + * needs to be fragmented. + * userdata_len cannot be used for it, once next sysdata append should + * start from the same userdata_len location, and only overwrite old + * sysdata. + */ + nt->sysdata_length = sysdata_len; +} + #endif /* CONFIG_NETCONSOLE_DYNAMIC */ /* Handle network interface device notifications */ @@ -1297,7 +1421,8 @@ static void send_ext_msg_udp(struct netconsole_target *nt, const char *msg, int release_len = 0; #ifdef CONFIG_NETCONSOLE_DYNAMIC - extradata_len = nt->userdata_length; + append_runtime_sysdata(nt); + extradata_len = nt->userdata_length + nt->sysdata_length; #endif if (nt->release)
Add infrastructure to automatically append kernel-generated data (sysdata) to netconsole messages. As the first use case, implement CPU number population, which adds the CPU that sent the message. This change introduces three distinct data types: - extradata: The complete set of appended data (sysdata + userdata) - userdata: User-provided key-value pairs from userspace - sysdata: Kernel-populated data (e.g. cpu=XX) The implementation adds a new configfs attribute 'cpu_nr' to control CPU number population per target. When enabled, each message is tagged with its originating CPU. The sysdata is dynamically updated at message time and appended after any existing userdata. The CPU number is formatted as "cpu=XX" and is added to the extradata buffer, respecting the existing size limits. Signed-off-by: Breno Leitao <leitao@debian.org> --- drivers/net/netconsole.c | 131 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 128 insertions(+), 3 deletions(-)