| Message ID | 20230613042422.5344-16-ricardo.neri-calderon@linux.intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | sched: Introduce classes of tasks for load balance | expand |
On Tue, Jun 13, 2023 at 6:23 AM Ricardo Neri <ricardo.neri-calderon@linux.intel.com> wrote: > > Implement the arch_get_ipcc_score() interface of the scheduler. Use the > performance capabilities of the extended Hardware Feedback Interface table > as the IPC score. > > Use the cached per-CPU IPCC scores. A seqcount provides lockless access and > the required memory ordering to avoid stale data. > > Cc: Ben Segall <bsegall@google.com> > Cc: Daniel Bristot de Oliveira <bristot@redhat.com> > Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> > Cc: Ionela Voinescu <ionela.voinescu@arm.com> > Cc: Joel Fernandes (Google) <joel@joelfernandes.org> > Cc: Len Brown <len.brown@intel.com> > Cc: Lukasz Luba <lukasz.luba@arm.com> > Cc: Mel Gorman <mgorman@suse.de> > Cc: Perry Yuan <Perry.Yuan@amd.com> > Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> > Cc: Steven Rostedt <rostedt@goodmis.org> > Cc: Tim C. Chen <tim.c.chen@intel.com> > Cc: Valentin Schneider <vschneid@redhat.com> > Cc: Zhao Liu <zhao1.liu@linux.intel.com> > Cc: x86@kernel.org > Cc: linux-pm@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > --- > Changes since v3: > * Ordered memory loads using a seqcount. > * Removed local variable hfi_class from intel_hfi_get_ipcc_score(). > (Rafael). > > Changes since v2: > * None > > Changes since v1: > * Adjusted the returned HFI class (which starts at 0) to match the > scheduler IPCC class (which starts at 1). (PeterZ) > * Used the new interface names. > --- > arch/x86/include/asm/topology.h | 4 ++++ > drivers/thermal/intel/intel_hfi.c | 40 +++++++++++++++++++++++++++++-- > 2 files changed, 42 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h > index 7d2ed7f7bb8c..8dd328cdb5cf 100644 > --- a/arch/x86/include/asm/topology.h > +++ b/arch/x86/include/asm/topology.h > @@ -237,13 +237,17 @@ void init_freq_invariance_cppc(void); > > #ifdef CONFIG_INTEL_HFI_THERMAL > int intel_hfi_read_classid(u8 *classid); > +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu); > #else > static inline int intel_hfi_read_classid(u8 *classid) { return -ENODEV; } > +static inline unsigned long > +intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) { return -ENODEV; } > #endif > > #ifdef CONFIG_IPC_CLASSES > void intel_update_ipcc(struct task_struct *curr); > #define arch_update_ipcc intel_update_ipcc > +#define arch_get_ipcc_score intel_hfi_get_ipcc_score > #endif > > #endif /* _ASM_X86_TOPOLOGY_H */ > diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c > index d822ed0bb5c1..fec2c01fda1b 100644 > --- a/drivers/thermal/intel/intel_hfi.c > +++ b/drivers/thermal/intel/intel_hfi.c > @@ -199,6 +199,42 @@ static int alloc_hfi_ipcc_scores(void) > return hfi_ipcc_scores ? 0 : -ENOMEM; > } > > +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) > +{ > + int *scores, score; > + unsigned long seq; > + > + scores = per_cpu_ptr(hfi_ipcc_scores, cpu); > + if (!scores) > + return -ENODEV; > + > + if (cpu < 0 || cpu >= nr_cpu_ids) > + return -EINVAL; > + > + if (ipcc == IPC_CLASS_UNCLASSIFIED) > + return -EINVAL; > + > + /* > + * Scheduler IPC classes start at 1. HFI classes start at 0. > + * See note intel_hfi_update_ipcc(). > + */ > + if (ipcc >= hfi_features.nr_classes + 1) > + return -EINVAL; > + > + /* > + * The seqcount implies load-acquire semantics to order loads with > + * lockless stores of the write side in set_hfi_ipcc_score(). It > + * also implies a compiler barrier. > + */ > + do { > + seq = read_seqcount_begin(&hfi_ipcc_seqcount); > + /* @ipcc is never 0. */ > + score = scores[ipcc - 1]; > + } while (read_seqcount_retry(&hfi_ipcc_seqcount, seq)); > + > + return score; > +} > + > static void set_hfi_ipcc_scores(struct hfi_instance *hfi_instance) > { > int cpu; > @@ -213,8 +249,8 @@ static void set_hfi_ipcc_scores(struct hfi_instance *hfi_instance) > raw_spin_lock_irq(&hfi_instance->table_lock); > /* > * The seqcount implies store-release semantics to order stores with > - * lockless loads from the seqcount read side. It also implies a > - * compiler barrier. > + * lockless loads from the seqcount read side in > + * intel_hfi_get_ipcc_score(). It also implies a compiler barrier. > */ > write_seqcount_begin(&hfi_ipcc_seqcount); > for_each_cpu(cpu, hfi_instance->cpus) { > -- > 2.25.1 >
diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index 7d2ed7f7bb8c..8dd328cdb5cf 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -237,13 +237,17 @@ void init_freq_invariance_cppc(void); #ifdef CONFIG_INTEL_HFI_THERMAL int intel_hfi_read_classid(u8 *classid); +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu); #else static inline int intel_hfi_read_classid(u8 *classid) { return -ENODEV; } +static inline unsigned long +intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) { return -ENODEV; } #endif #ifdef CONFIG_IPC_CLASSES void intel_update_ipcc(struct task_struct *curr); #define arch_update_ipcc intel_update_ipcc +#define arch_get_ipcc_score intel_hfi_get_ipcc_score #endif #endif /* _ASM_X86_TOPOLOGY_H */ diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index d822ed0bb5c1..fec2c01fda1b 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -199,6 +199,42 @@ static int alloc_hfi_ipcc_scores(void) return hfi_ipcc_scores ? 0 : -ENOMEM; } +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) +{ + int *scores, score; + unsigned long seq; + + scores = per_cpu_ptr(hfi_ipcc_scores, cpu); + if (!scores) + return -ENODEV; + + if (cpu < 0 || cpu >= nr_cpu_ids) + return -EINVAL; + + if (ipcc == IPC_CLASS_UNCLASSIFIED) + return -EINVAL; + + /* + * Scheduler IPC classes start at 1. HFI classes start at 0. + * See note intel_hfi_update_ipcc(). + */ + if (ipcc >= hfi_features.nr_classes + 1) + return -EINVAL; + + /* + * The seqcount implies load-acquire semantics to order loads with + * lockless stores of the write side in set_hfi_ipcc_score(). It + * also implies a compiler barrier. + */ + do { + seq = read_seqcount_begin(&hfi_ipcc_seqcount); + /* @ipcc is never 0. */ + score = scores[ipcc - 1]; + } while (read_seqcount_retry(&hfi_ipcc_seqcount, seq)); + + return score; +} + static void set_hfi_ipcc_scores(struct hfi_instance *hfi_instance) { int cpu; @@ -213,8 +249,8 @@ static void set_hfi_ipcc_scores(struct hfi_instance *hfi_instance) raw_spin_lock_irq(&hfi_instance->table_lock); /* * The seqcount implies store-release semantics to order stores with - * lockless loads from the seqcount read side. It also implies a - * compiler barrier. + * lockless loads from the seqcount read side in + * intel_hfi_get_ipcc_score(). It also implies a compiler barrier. */ write_seqcount_begin(&hfi_ipcc_seqcount); for_each_cpu(cpu, hfi_instance->cpus) {
Implement the arch_get_ipcc_score() interface of the scheduler. Use the performance capabilities of the extended Hardware Feedback Interface table as the IPC score. Use the cached per-CPU IPCC scores. A seqcount provides lockless access and the required memory ordering to avoid stale data. Cc: Ben Segall <bsegall@google.com> Cc: Daniel Bristot de Oliveira <bristot@redhat.com> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Ionela Voinescu <ionela.voinescu@arm.com> Cc: Joel Fernandes (Google) <joel@joelfernandes.org> Cc: Len Brown <len.brown@intel.com> Cc: Lukasz Luba <lukasz.luba@arm.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Perry Yuan <Perry.Yuan@amd.com> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Tim C. Chen <tim.c.chen@intel.com> Cc: Valentin Schneider <vschneid@redhat.com> Cc: Zhao Liu <zhao1.liu@linux.intel.com> Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> --- Changes since v3: * Ordered memory loads using a seqcount. * Removed local variable hfi_class from intel_hfi_get_ipcc_score(). (Rafael). Changes since v2: * None Changes since v1: * Adjusted the returned HFI class (which starts at 0) to match the scheduler IPCC class (which starts at 1). (PeterZ) * Used the new interface names. --- arch/x86/include/asm/topology.h | 4 ++++ drivers/thermal/intel/intel_hfi.c | 40 +++++++++++++++++++++++++++++-- 2 files changed, 42 insertions(+), 2 deletions(-)