[v2,2/2] cpufreq: Wire-up arch-flavored freq info into cpufreq_verify_current_freq

From: Sumit Gupta <sumitg@nvidia.com>

When available, use arch_freq_get_on_cpu to obtain current frequency
(usually an average reported over given period of time)
to better align the cpufreq's view on the current state of affairs.
This also automatically pulls in the update for cpuinfo_cur_freq sysfs
attribute, aligning it with the scaling_cur_freq one, and thus providing
consistent view on relevant platforms.

Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
[BM: Subject & commit msg]
Signed-off-by: Beata Michalska <beata.michalska@arm.com>
---
 drivers/cpufreq/cpufreq.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Hi Sumit,

On Friday 01 Dec 2023 at 18:32:10 (+0530), Sumit Gupta wrote:
> Hi Ionela,
> 
> > > --- a/drivers/cpufreq/cpufreq.c
> > > +++ b/drivers/cpufreq/cpufreq.c
> > > @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
> > >   {
> > >        unsigned int new_freq;
> > > 
> > > -     new_freq = cpufreq_driver->get(policy->cpu);
> > > +     new_freq = arch_freq_get_on_cpu(policy->cpu);
> > > +     new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
> > 
> > Given that arch_freq_get_on_cpu() is an average frequency, it does not
> > seem right to me to trigger the sync & update process of
> > cpufreq_verify_current_freq() based on it.
> > 
> > cpufreq_verify_current_freq() will at least modify the internal state of
> > the policy and send PRE and POST notifications, if not do a full frequency
> > update, based on this average frequency, which is likely different from
> > the current frequency, even beyond the 1MHz threshold.
> > 
> > While I believe it's okay to return this average frequency in
> > cpuinfo_cur_freq, I don't think it should be used as an indication of
> > an accurate current frequency, which is what
> > cpufreq_verify_current_freq() expects.
> > 
> > Sumit, can you give more details on the issue at [1] and why this change
> > fixes it?
> > 
> > [1] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
> > 
> > Thank you,
> > Ionela.
> > 
> cpufreq_verify_current_freq() also updates 'policy->cur' in POST
> notification if the frequency from hardware has more delta (out of sync).
> 
> As the value from 'cpufreq_driver->get()' is not reliable due to [1],
> calling the 'get' hook can update the 'policy->cur' with a wrong value when
> governor starts in cpufreq_start_governor().
> And if the frequency is never changed after the governor starts during
> boot e.g. when performance governor is set as default, then
> 'scaling_cur_freq' always returns wrong value.
> 
> Instead, the arch_freq_get_on_cpu() API updates 'policy->cur' with a more
> stable freq value.
> 
> [1] https://lore.kernel.org/lkml/20230418113459.12860-7-sumitg@nvidia.com/

Got it, many thanks! 

As the code is right now in v2, arch_freq_get_on_cpu() is called on
show_scaling_cur_freq(), so the problem you describe would not show up.
policy->cur would still be incorrect, but 'scaling_cur_freq' would
return the value from arch_freq_get_on_cpu().

Would it be enough if arch_freq_get_on_cpu() gets also called from
show_cpuinfo_cur_freq() instead of cpufreq_verify_current_freq()?

Thanks,
Ionela.

> 
> Best regards,
> Sumit Gupta
> 
> > >        if (!new_freq)
> > >                return 0;
> > > 
> > > --
> > > 2.25.1
> > >

On 05/12/23 16:35, Ionela Voinescu wrote:
> External email: Use caution opening links or attachments
> 
> 
> Hi Sumit,
> 
> On Friday 01 Dec 2023 at 18:32:10 (+0530), Sumit Gupta wrote:
>> Hi Ionela,
>>
>>>> --- a/drivers/cpufreq/cpufreq.c
>>>> +++ b/drivers/cpufreq/cpufreq.c
>>>> @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
>>>>    {
>>>>         unsigned int new_freq;
>>>>
>>>> -     new_freq = cpufreq_driver->get(policy->cpu);
>>>> +     new_freq = arch_freq_get_on_cpu(policy->cpu);
>>>> +     new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
>>>
>>> Given that arch_freq_get_on_cpu() is an average frequency, it does not
>>> seem right to me to trigger the sync & update process of
>>> cpufreq_verify_current_freq() based on it.
>>>
>>> cpufreq_verify_current_freq() will at least modify the internal state of
>>> the policy and send PRE and POST notifications, if not do a full frequency
>>> update, based on this average frequency, which is likely different from
>>> the current frequency, even beyond the 1MHz threshold.
>>>
>>> While I believe it's okay to return this average frequency in
>>> cpuinfo_cur_freq, I don't think it should be used as an indication of
>>> an accurate current frequency, which is what
>>> cpufreq_verify_current_freq() expects.
>>>
>>> Sumit, can you give more details on the issue at [1] and why this change
>>> fixes it?
>>>
>>> [1] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
>>>
>>> Thank you,
>>> Ionela.
>>>
>> cpufreq_verify_current_freq() also updates 'policy->cur' in POST
>> notification if the frequency from hardware has more delta (out of sync).
>>
>> As the value from 'cpufreq_driver->get()' is not reliable due to [1],
>> calling the 'get' hook can update the 'policy->cur' with a wrong value when
>> governor starts in cpufreq_start_governor().
>> And if the frequency is never changed after the governor starts during
>> boot e.g. when performance governor is set as default, then
>> 'scaling_cur_freq' always returns wrong value.
>>
>> Instead, the arch_freq_get_on_cpu() API updates 'policy->cur' with a more
>> stable freq value.
>>
>> [1] https://lore.kernel.org/lkml/20230418113459.12860-7-sumitg@nvidia.com/
> 
> Got it, many thanks!
> 
> As the code is right now in v2, arch_freq_get_on_cpu() is called on
> show_scaling_cur_freq(), so the problem you describe would not show up.
> policy->cur would still be incorrect, but 'scaling_cur_freq' would
> return the value from arch_freq_get_on_cpu().
> 
> Would it be enough if arch_freq_get_on_cpu() gets also called from
> show_cpuinfo_cur_freq() instead of cpufreq_verify_current_freq()?
> 
> Thanks,
> Ionela.
> 

Yes.
I am not sure if making both the nodes 'scaling_cur_freq' and 
'cpuinfo_cur_freq' same is fine?

Best Regards,
Sumit Gupta

On Mon, Nov 27, 2023 at 5:09 PM Beata Michalska <beata.michalska@arm.com> wrote:
>
> From: Sumit Gupta <sumitg@nvidia.com>
>
> When available, use arch_freq_get_on_cpu to obtain current frequency
> (usually an average reported over given period of time)
> to better align the cpufreq's view on the current state of affairs.

And why is this a good idea?

Any problem statement?

> This also automatically pulls in the update for cpuinfo_cur_freq sysfs
> attribute, aligning it with the scaling_cur_freq one, and thus providing
> consistent view on relevant platforms.

I have no idea what the above is supposed to mean, sorry.

> Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
> [BM: Subject & commit msg]
> Signed-off-by: Beata Michalska <beata.michalska@arm.com>
> ---
>  drivers/cpufreq/cpufreq.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> index 8c4f9c2f9c44..109559438f45 100644
> --- a/drivers/cpufreq/cpufreq.c
> +++ b/drivers/cpufreq/cpufreq.c
> @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
>  {
>         unsigned int new_freq;
>
> -       new_freq = cpufreq_driver->get(policy->cpu);
> +       new_freq = arch_freq_get_on_cpu(policy->cpu);
> +       new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);

Please don't use ?: in general and it is not even useful here AFAICS.

What would be wrong with

new_freq = arch_freq_get_on_cpu(policy->cpu);
if (!new_freq)
        new_freq = cpufreq_driver->get(policy->cpu);

?

>         if (!new_freq)
>                 return 0;
>
> --

On Wednesday 06 Dec 2023 at 18:58:17 (+0530), Sumit Gupta wrote:
> 
> 
> On 05/12/23 16:35, Ionela Voinescu wrote:
> > External email: Use caution opening links or attachments
> > 
> > 
> > Hi Sumit,
> > 
> > On Friday 01 Dec 2023 at 18:32:10 (+0530), Sumit Gupta wrote:
> > > Hi Ionela,
> > > 
> > > > > --- a/drivers/cpufreq/cpufreq.c
> > > > > +++ b/drivers/cpufreq/cpufreq.c
> > > > > @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
> > > > >    {
> > > > >         unsigned int new_freq;
> > > > > 
> > > > > -     new_freq = cpufreq_driver->get(policy->cpu);
> > > > > +     new_freq = arch_freq_get_on_cpu(policy->cpu);
> > > > > +     new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
> > > > 
> > > > Given that arch_freq_get_on_cpu() is an average frequency, it does not
> > > > seem right to me to trigger the sync & update process of
> > > > cpufreq_verify_current_freq() based on it.
> > > > 
> > > > cpufreq_verify_current_freq() will at least modify the internal state of
> > > > the policy and send PRE and POST notifications, if not do a full frequency
> > > > update, based on this average frequency, which is likely different from
> > > > the current frequency, even beyond the 1MHz threshold.
> > > > 
> > > > While I believe it's okay to return this average frequency in
> > > > cpuinfo_cur_freq, I don't think it should be used as an indication of
> > > > an accurate current frequency, which is what
> > > > cpufreq_verify_current_freq() expects.
> > > > 
> > > > Sumit, can you give more details on the issue at [1] and why this change
> > > > fixes it?
> > > > 
> > > > [1] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
> > > > 
> > > > Thank you,
> > > > Ionela.
> > > > 
> > > cpufreq_verify_current_freq() also updates 'policy->cur' in POST
> > > notification if the frequency from hardware has more delta (out of sync).
> > > 
> > > As the value from 'cpufreq_driver->get()' is not reliable due to [1],
> > > calling the 'get' hook can update the 'policy->cur' with a wrong value when
> > > governor starts in cpufreq_start_governor().
> > > And if the frequency is never changed after the governor starts during
> > > boot e.g. when performance governor is set as default, then
> > > 'scaling_cur_freq' always returns wrong value.
> > > 
> > > Instead, the arch_freq_get_on_cpu() API updates 'policy->cur' with a more
> > > stable freq value.
> > > 
> > > [1] https://lore.kernel.org/lkml/20230418113459.12860-7-sumitg@nvidia.com/
> > 
> > Got it, many thanks!
> > 
> > As the code is right now in v2, arch_freq_get_on_cpu() is called on
> > show_scaling_cur_freq(), so the problem you describe would not show up.
> > policy->cur would still be incorrect, but 'scaling_cur_freq' would
> > return the value from arch_freq_get_on_cpu().
> > 
> > Would it be enough if arch_freq_get_on_cpu() gets also called from
> > show_cpuinfo_cur_freq() instead of cpufreq_verify_current_freq()?
> > 
> > Thanks,
> > Ionela.
> > 
> 
> Yes.
> I am not sure if making both the nodes 'scaling_cur_freq' and
> 'cpuinfo_cur_freq' same is fine?

That would happen anyway if arch_freq_get_on_cpu() is called from
cpufreq_verify_current_freq().

In principle, according to [1], it would be correct to use it for
'cpuinfo_cur_freq' and not 'scaling_cur_freq'. But the call from
show_scaling_cur_freq() is already there before these patches,
introduced a long time ago for x86.

The topic was discussed at [2] and the agreement so far was that it
would be best to keep the behaviour the same for both x86 and arm.

I don't like going against the user-guide, but these patches don't
actually go against the user-guide. The old call to
arch_freq_get_on_cpu() from show_scaling_cur_freq() goes against it.
But I agree that's something necessary to keep, as legacy for x86.
Additionally, you also mentioned that you'd prefer to have a more
accurate frequency returned for 'scaling_cur_freq'.

[1] https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt
[2] https://lore.kernel.org/lkml/20230609043922.eyyqutbwlofqaddz@vireshk-i7/

Thanks,
Ionela.

> 
> Best Regards,
> Sumit Gupta

>>>>>> --- a/drivers/cpufreq/cpufreq.c
>>>>>> +++ b/drivers/cpufreq/cpufreq.c
>>>>>> @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
>>>>>>     {
>>>>>>          unsigned int new_freq;
>>>>>>
>>>>>> -     new_freq = cpufreq_driver->get(policy->cpu);
>>>>>> +     new_freq = arch_freq_get_on_cpu(policy->cpu);
>>>>>> +     new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
>>>>>
>>>>> Given that arch_freq_get_on_cpu() is an average frequency, it does not
>>>>> seem right to me to trigger the sync & update process of
>>>>> cpufreq_verify_current_freq() based on it.
>>>>>
>>>>> cpufreq_verify_current_freq() will at least modify the internal state of
>>>>> the policy and send PRE and POST notifications, if not do a full frequency
>>>>> update, based on this average frequency, which is likely different from
>>>>> the current frequency, even beyond the 1MHz threshold.
>>>>>
>>>>> While I believe it's okay to return this average frequency in
>>>>> cpuinfo_cur_freq, I don't think it should be used as an indication of
>>>>> an accurate current frequency, which is what
>>>>> cpufreq_verify_current_freq() expects.
>>>>>
>>>>> Sumit, can you give more details on the issue at [1] and why this change
>>>>> fixes it?
>>>>>
>>>>> [1] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
>>>>>
>>>>> Thank you,
>>>>> Ionela.
>>>>>
>>>> cpufreq_verify_current_freq() also updates 'policy->cur' in POST
>>>> notification if the frequency from hardware has more delta (out of sync).
>>>>
>>>> As the value from 'cpufreq_driver->get()' is not reliable due to [1],
>>>> calling the 'get' hook can update the 'policy->cur' with a wrong value when
>>>> governor starts in cpufreq_start_governor().
>>>> And if the frequency is never changed after the governor starts during
>>>> boot e.g. when performance governor is set as default, then
>>>> 'scaling_cur_freq' always returns wrong value.
>>>>
>>>> Instead, the arch_freq_get_on_cpu() API updates 'policy->cur' with a more
>>>> stable freq value.
>>>>
>>>> [1] https://lore.kernel.org/lkml/20230418113459.12860-7-sumitg@nvidia.com/
>>>
>>> Got it, many thanks!
>>>
>>> As the code is right now in v2, arch_freq_get_on_cpu() is called on
>>> show_scaling_cur_freq(), so the problem you describe would not show up.
>>> policy->cur would still be incorrect, but 'scaling_cur_freq' would
>>> return the value from arch_freq_get_on_cpu().
>>>
>>> Would it be enough if arch_freq_get_on_cpu() gets also called from
>>> show_cpuinfo_cur_freq() instead of cpufreq_verify_current_freq()?
>>>
>>> Thanks,
>>> Ionela.
>>>
>>
>> Yes.
>> I am not sure if making both the nodes 'scaling_cur_freq' and
>> 'cpuinfo_cur_freq' same is fine?
> 
> That would happen anyway if arch_freq_get_on_cpu() is called from
> cpufreq_verify_current_freq().
> 
Yes, that will happen in both the cases.

> In principle, according to [1], it would be correct to use it for
> 'cpuinfo_cur_freq' and not 'scaling_cur_freq'. But the call from
> show_scaling_cur_freq() is already there before these patches,
> introduced a long time ago for x86.
> 
> The topic was discussed at [2] and the agreement so far was that it
> would be best to keep the behaviour the same for both x86 and arm.
> 
Looking at the previous discussion in [2], seems to be fine.

Best Regards,
Sumit Gupta

> I don't like going against the user-guide, but these patches don't
> actually go against the user-guide. The old call to
> arch_freq_get_on_cpu() from show_scaling_cur_freq() goes against it.
> But I agree that's something necessary to keep, as legacy for x86.
> Additionally, you also mentioned that you'd prefer to have a more
> accurate frequency returned for 'scaling_cur_freq'.
> 
> [1] https://www.kernel.org/doc/Documentation/cpu-freq/user-guide.txt
> [2] https://lore.kernel.org/lkml/20230609043922.eyyqutbwlofqaddz@vireshk-i7/
> 
> Thanks,
> Ionela.
> 
>>
>> Best Regards,
>> Sumit Gupta

Apologies for extremely late reply, juggling too many things at time ....

On Wed, Dec 06, 2023 at 09:41:05PM +0100, Rafael J. Wysocki wrote:
> On Mon, Nov 27, 2023 at 5:09 PM Beata Michalska <beata.michalska@arm.com> wrote:
> >
> > From: Sumit Gupta <sumitg@nvidia.com>
> >
> > When available, use arch_freq_get_on_cpu to obtain current frequency
> > (usually an average reported over given period of time)
> > to better align the cpufreq's view on the current state of affairs.
> 
> And why is this a good idea?
Apart from being problematic with an issue pointed at [1] (which will result
in dropping the change in cpufreq) this was to keep the cpufreq core aware of
potential frequency changes and take appropriate action (informing the governor)
case it got out of sync.
> 
> Any problem statement?
The problem has been raised here [2]
> 
> > This also automatically pulls in the update for cpuinfo_cur_freq sysfs
> > attribute, aligning it with the scaling_cur_freq one, and thus providing
> > consistent view on relevant platforms.
> 
> I have no idea what the above is supposed to mean, sorry.
Bad wording I guess. With this change both 'cpuinfo_cur_freq' and
'scaling_cur_freq' will use the arch_freq_get_on_cpu if available, and will use
the same source of information (one depending on a platform).

> 
> > Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
> > [BM: Subject & commit msg]
> > Signed-off-by: Beata Michalska <beata.michalska@arm.com>
> > ---
> >  drivers/cpufreq/cpufreq.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> > index 8c4f9c2f9c44..109559438f45 100644
> > --- a/drivers/cpufreq/cpufreq.c
> > +++ b/drivers/cpufreq/cpufreq.c
> > @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
> >  {
> >         unsigned int new_freq;
> >
> > -       new_freq = cpufreq_driver->get(policy->cpu);
> > +       new_freq = arch_freq_get_on_cpu(policy->cpu);
> > +       new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
> 
> Please don't use ?: in general and it is not even useful here AFAICS.
> 
> What would be wrong with
> 
> new_freq = arch_freq_get_on_cpu(policy->cpu);
> if (!new_freq)
>         new_freq = cpufreq_driver->get(policy->cpu);
> 
> ?
Nothing wrong with that.

---
[1] https://lore.kernel.org/all/ZWXy0h%2FfFfQh+Rhy@arm.com/
[2] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
---
BR
Beata
> 
> >         if (!new_freq)
> >                 return 0;
> >
> > --

Hi Ionela,

So sorry for relpying so late, lost if from my rader for a while ...
On Tue, Nov 28, 2023 at 02:01:54PM +0000, Ionela Voinescu wrote:
> Hi Beata, Sumit,
> 
> On Monday 27 Nov 2023 at 16:08:38 (+0000), Beata Michalska wrote:
> > From: Sumit Gupta <sumitg@nvidia.com>
> > 
> > When available, use arch_freq_get_on_cpu to obtain current frequency
> > (usually an average reported over given period of time)
> > to better align the cpufreq's view on the current state of affairs.
> > This also automatically pulls in the update for cpuinfo_cur_freq sysfs
> > attribute, aligning it with the scaling_cur_freq one, and thus providing
> > consistent view on relevant platforms.
> > 
> > Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
> > [BM: Subject & commit msg]
> > Signed-off-by: Beata Michalska <beata.michalska@arm.com>
> > ---
> >  drivers/cpufreq/cpufreq.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
> > index 8c4f9c2f9c44..109559438f45 100644
> > --- a/drivers/cpufreq/cpufreq.c
> > +++ b/drivers/cpufreq/cpufreq.c
> > @@ -1756,7 +1756,8 @@ static unsigned int cpufreq_verify_current_freq(struct cpufreq_policy *policy, b
> >  {
> >  	unsigned int new_freq;
> >  
> > -	new_freq = cpufreq_driver->get(policy->cpu);
> > +	new_freq = arch_freq_get_on_cpu(policy->cpu);
> > +	new_freq = new_freq ?: cpufreq_driver->get(policy->cpu);
> 
> Given that arch_freq_get_on_cpu() is an average frequency, it does not
> seem right to me to trigger the sync & update process of
> cpufreq_verify_current_freq() based on it.
> 
> cpufreq_verify_current_freq() will at least modify the internal state of
> the policy and send PRE and POST notifications, if not do a full frequency
> update, based on this average frequency, which is likely different from
> the current frequency, even beyond the 1MHz threshold.
> 
Noted, will drop this change.

---
BR
Beata
> While I believe it's okay to return this average frequency in
> cpuinfo_cur_freq, I don't think it should be used as an indication of
> an accurate current frequency, which is what
> cpufreq_verify_current_freq() expects.
> 

> Sumit, can you give more details on the issue at [1] and why this change
> fixes it?
> 
> [1] https://lore.kernel.org/lkml/6a5710f6-bfbb-5dfd-11cd-0cd02220cee7@nvidia.com/
> 
> Thank you,
> Ionela.
	> 
> >  	if (!new_freq)
> >  		return 0;
> >  
> > -- 
> > 2.25.1
> >